It actually boots into act_dead (or something like that) mode when you connect charger. Then when you press powerbutton it only has to load the GUI.

It's probably starting the kernel and getting the root jffs2 filesystem mounted that's taking time. jffs2 is kinda notorious for long mount times.