The N900 begins to behave badly when you load it with lots of media. Tracker will begin to work hard indexing your stuff, while the thumbnailer will eat CPU producing thumbnail images while you don't need them at all.
Combine that with frequently swapping to a slow flash drive, and pulseaudio taking away the last bits of CPU resources, and you'll get a really bad experience.