Do you use a tile based navigation program like mappero? Those tile files are pretty small, usually between 5k and 12k, but because of the fat32 filesystem, each file hogs a 64k cluster. So 1 GB worth of maps can easily occupy 10 GB worth of disk space or more.

I suggest to try emptying your map folder before you wipe the whole partition.