1- Well, HQ will only be so good on a 640x360 screen. There might be something else going on though.

2 and 3- often depends what codec is used to create the video in either container. The videos may have been encoded with a codec that the N8 can't decode. I can't remember off the top of my head which ones it can do so I'd check the manual and check your files.