having a sharp eye for these kind of build issues, I have pointed out my mistrust of the usb connector (noticing it sitting kinda lonely in a big hole) to the sales person back in Januari. Like he cared.

Having always pulled and inserted the usb cables straight and gently, l'm now sure I won't likely meet an issue.

I have bought 2 extra second hand N900's which seem to have more easy to pull usb cable fits (due to usage by non nerdy persons?) , and both have still firmly fixed USB ports.
Finally I don't see a problem if you will be the single careful user of the device.