I am also playing with H264/AVC. As I don't want to go back to MPEG-4 ASP or Divx.

For certain, there is no way the N810 can keep up with PSP files encoded using level 3 main profile. You must re-encode.

I have some success with 400x240 resolution and let the LCD h/w scaler do the image doubling. For audio, HE-AAC at 64Kbps. You want to give all the processor power to handle the h264 decoding, which is most time consuming.

Right now I am encoding my movies using level 1.2 baseline profile, and doing a 2-pass encoding. But I have some problem with the 2-pass encoding, the max video bit rate produced is still too high for the N810 to handle, I tried to lower the level to 1.1, but the built-in player won't accept that profile.

From what I have observed so far. With audio set to 64Kbps. If the video bit rate reaches above 450Kbps, you are starting to drop frames, and the video will look choppy. So I need to play with my encoder and to put a constraint on the output bit rate. If I can do that, then I think I can produce very good quality AVC video, much better quality than MPEG-4 ASP.