Music (read-only) and documents (small files, very few writes) shouldn't cause much trouble if they are on the SD card with the swap partition. Of course there may still be conflicts but the device should be able to handle them.
I have swap and (mostly) music on my SD card and almost never experience any hickups.