The issue is that from USB you are not allowed to draw more than 100mA without negotiating with the host first, asking permission to draw full 500mA. If the host computer give permission for this, the device can then start consuming more power.
For "dumb" chargers, I believe they added a way to signal to the device, that the other end is a charger, by shorting the data lines together. In that case the device can pull as much power as it can get.

There's probably a million metric tonnes of chargers and devices manufactured before USB Charging was standardized. They are not USB Compliant, and might or might not work, might fry your computer's USB port, etc...