Might not be the USB port that's loose, could just be getting worn out.
Not sure why but sometimes I get the problem where wiggleing the connector will make the phone go in and out of charge mode, but not often. After a quick blast of compressed air it's fine for weeks afterwards.
I'm using a Lumia charger and USB cable for mine, which fares better than the old original N900 cable.