With FAT, it should be safe to pull as long as there are no write operations in progress. Keep in mind, just because there isn't a dialog telling you something is going on, there could still be something in the background you aren't aware of (such as cached writes). Ejecting syncs any pending operations to disk before letting you know it's safe. Also, some file systems don't take well to not being unmounted (they assume it means something bad happened). Personally, I play it safe and eject - I can spare a few seconds to save a few hours of recovery.