Hello,
I've been testing video players in my N800, and mplayer is by far the fastest video player in this tablet.

Now, I'm looking for the best video quality that mplayer can run smoothly in the tablet.

Most of the topics in this forum discussing video encoding for N8x0 are old or are based on the build-in video player.

So far, what I've found, is that I can play 640*360 without any lags.

Container: AVI
Width 640
Height 640/Aspect Ratio
Video Codec: Xvid (with 1 pass)
Video bitrate: 800
Audio Codec: MP3
Audio Bitrate: 128

This works like a charm for 16x9 videos, even with subtitles, I'm looking forward to see what can you get from the N8x0 processor.

EDIT:
Finally, here is what I think the best quality mplayer (MPlayer 1.0rc1-maemo.29.n8x0) can play on N8x0:
Container: AVI
Width: 480 for 4/3, and 640 for 16/9
Height: 360
Video Codec: Xvid (with 1 pass only)
Video Bitrate: 800
Framerate: 24 f/s
Audio Codec: MP3
Audio bitrate: 128
SampleRate: 44.1 Khz

Encoder used: HandBrake

Again, these values are based on my experience, If you think they should be corrected, feel free to post.