It may be faulty or stuck power button itself. When you attach charger the device boots 'normally' but in the end it hides the UI from you and just shows charging battery icon. So in fact the device is 99% booted and when you press power button in this state it does the final 1% and shows the desktop to you.

This does not help you much, just wanted to say that the device is not dead but works almost fine except the power button. So if you can return it, you probably should but if not, it may be possible to find workaround and power it up by charger and then force the last 1% via some software modification.

Plugging charger is simply another way to power the device on. The boot sequence then checks boot reason (file /proc/bootreason) anf if it is not 'charger' it boots normally and shows the desktop, otherwise it shows the battery icon.