Ah, managed to get it working now. For future reference: the missing library that has to be installed in red pill mode is libg2c. Then you can run octave with ./octave. Works like a charm.

Would it be possible to get octave 3.0 running (was released this December)? Would be awesome!