I already mentioned it in this very thread how I do it I think.

1. I use xvid4psp 6 daily build from www.winnydows.com
2. Set it to MP4, set resolution to 854x480 maximum, AVC level to 3.1 (can save this as a new profile)
3. Set video to H264, then baseline profile in its settings.
4. I prefer constant bitrate. I use 2000 for high quality source. 1500 for lower quality source. (then can save this as a new video profile)
5. AAC-LC for audio. 192kbps stereo. (I like my audio fairly high quality or at least not too lossy)
- I don't really touch anything else. But you can experiment to your hearts content if you like to.

My encoded videos look perfect and plays butter smooth with default video player or OVP. =)

Edit: Updated with more info...