The card doesn't use magnetism for storage, so no, that won't damage the card. But if the cover is not on then it won't mount (the card is automatically mounted when the cover goes on, and dismounted when off. Likewise for that little flap over the external card. This is important, particularly for the unmount because as part of the automatic unmount the system will need to flush non-written data to the card. By ripping out the card too fast you could end up with inconsistencies (just as when you unplug a USB stick from a Windows PC without going through the 'safe to remove USB device' sequence thingy)).