Yes, you should be able to turn off the unit with the power button.

You did take the battery out once to see if the problem went away, right? I hope so.

There is a way to configure how the power button works, but I don't know it.