My N9 seems to think I am pressing the power button even when I am not touching it. This means that it will randomly wake up from sleep, and occasionally think I am holding it for more than ten seconds or so and power off. The result is that it is no longer usable as a primary phone - and due to having ubiboot on there, this has led to several times that the alarm didn't even go off (as it randomly rebooted while plugged in and ended up in the telnet screen).

Is there any way to fix this? I know it's probably no longer possible to ask Nokia for help.

Get your power button repaired from a phone repair shop. The button may have gone in too much hence the constant pressing. There is a good video on disassembling the N9 which you may want to take to the repair shop in case they are not too familiar with the N9.