When the internal battery is down to 3.7 V the device has probably auto-shutdown already. My Palm shuts down at 3.79 V (on the Palm there's software available to tell you which voltage it shuts down, which voltage it starts to give you warnings, which voltage it stops charging etc. Some of these can even be changed with software).

A fully loaded battery will be somewhere between 4.10V and 4.20V, depending on the device and its configuration. My Palm goes up to 4.24V while charging, but not above.
Chargers typically provide around 5.2V, the charging circuitry in the device does the rest.