OK, I'm still playing with this thing before I RMA it. Right now the device still won't power-on on it's own. However, if I plug it in it is now consistently going into Battery Recharge mode (after sitting plugged in for 30 or so sec), and can then be powered on from there.
IMO This is the way to "solve" the problem (while you can)