Same here, This doesn't worry me so bad, I think a little bit of play here is a good thing (to prevent additional stress on the housing)

I tried applying very small amounts of pressure in the same directions to the port housing itself with the stylus and the port itself does not seem to wiggle.