Transcode your DVD VOBs to a more modern format. MPEG-2 isn't really state of the art anymore, there are more space-efficient (and higher quality, which means no/not much quality loss compared to the MPEG-2 source) codecs out there. Also, there's little use in having AC-3 sound (maybe that's why Media Player can't play your files?) with a stereo speaker device / stereo headphones / stereo line-out.

For example, there's Handbrake which is an easy-to-use DVD ripping utility, and I'm sure you get good encoding settings for your N900 if you search this forum (on the other hand, the N900 plays almost any "700MB movie" that you throw at it, so a off-the-shelf 700MB DVD movie rip should be no problem at all).

Apart from compatibilty with your N900, you will also gain more space on your UPnP share and an easier file structure (multiple strangely-named VOBs versus a single .avi/.mkv/.ogv file).