I tried some different MPEG-4 qualities, coming from a HD-264 film (.m2ts), using Pinnacle 12. Highest one was the 720X400, 2000 kbits/sec, 25 images/sec and audio 128 kbits/sec 16 bits stereo, 48 khz,. It lookes great on the machine, TV-out is bearable. Of course the real 1080p on TV using blu-ray is a different league.

Using a higher bitrate (4000 kbits/sec) gave some stuttering in camera movements.

It will work if you set the quality to DVD and then manually change the resolution to: 848x480.

This will work on N900.
take a look at the screenshot
http://img339.imageshack.us/img339/2381/68312864.png

I thought I'd bump this topic. Surely more people can post their results...

I've tried:

Format: MPEG-4 Visual
Codec ID: XVID
Resolution: 848x480
Video Bitrate: 7 000 Kbps
FPS: 25
Audio Format: MP3
Audio Bitrate: 320 Kbps
Sample Rate: 48 Khz

...but the audio is delayed. Video plays fluently though. Still amazing the N900 can play this.

I've also tried:

Format: MPEG-4 Visual
Codec ID: XVID
Resolution: 848x480
Video Bitrate: 4 500 Kbps
FPS: 60
Audio Format: MP3
Audio Bitrate: 320 Kbps
Sample Rate: 48 Khz

...and this plays without any problem and looks stunning! 60fps ftw!

I think 5 000 Kbps is the maximum the N900 can handle.