There is actually two answers to this.... the short and long.

The short is that USB ports are only designed to trickle charge, whereas a wall charger is specifically for charging at maximum speed.

The long, official USB specification says the current drain from the USB port must be somewhere between 100-500mA, this is generally on the lower end (100-200) for laptops... as they have to be more power conscious, whereas a desktop generally will see a little better (200-500), this actually depends a fair bit on there you plug it in also, plugging directly into the back of a PC will see a significantly more amount of current compared to the convenient ones at the front.

A wall charger, however, works with the premise it can drain as much power as needed. The Nokia AC-10 wall charger, for example, puts out 1200mA of charging output. As you can obviously do basic maths... thats more than 10 times the charging power.

This is why I'm always careful to get the best possible charger... as a lot of the Chinese ones will only ever put out 500-800 (haven't really looked to deeply into this) which limits me to a charger for a lot longer than I need be.

Just did, gave it to my father and he won 5 times in a row (he plays FICS almost 3 hours a day, he's obsessed with online chess) :P