If you can't open an xvid in maemo mplayer, it is probably too "big" (too high resolution or too high bitrate) for the tablet. The ideal xvid for the tablet is 400x240, 800 kbps. Use Tablet Encode or some other encoding program (like VirtualDub in Windows) to re-encode your video.

There is a maemo vlc, but it isn't as good as maemo mplayer.