To swap batteries quickly, I have the replacement battery ready in one hand (in the correct orientation) and pull the old battery out with the other hand.

If the battery is out for longer than about 2 seconds, I lose the time & date. It's always been that way for me.