If you write a new partition table and reboot, nothing will be left, so this reboot loop is "normal" behavior.
To execute the mkfs/format commands one possibility would be to flash a temporary rescue system to the ram.

http://wiki.maemo.org/User:Nin101/N900_Rescue_OS
http://wiki.maemo.org/Repartitioning...rom_a_linux_PC
http://wiki.meego.com/ARM/N900/Using_Rescue_Initrd

To get the files back you can use flasher, backupmenu, or PC Suite (I never used it).

I did use gparted and don't did it per hand like you. The advantage is, all files remain untouched as I did only resize the partitions with gparted. It takes some time to shrink/grow, so better only do this with charged battery.