Mine has trouble turning on also when it's not plugged in. About 30% of the time it will not turn on unless I do one of the following:
- Repeatedly press and hold power for as long as 30 seconds.
- Take out battery, put back in and do step one a couple of times
- Press the Home button and Power at the same time and keep holding them

Trying these steps over and over eventually brings it back to life. The longest it's taking me to power it on is 5 minutes of trying. Usually it's about 1-2 minutes...