Another update of MPlayer which works with the second version of N800 firmware is available (mplayer_1.0rc1-maemo.12.800). It uses Xv for video output, some tweaks were required to fix stability issues, but now it seems to run ok.