I do generic ARM embedded systems development running a Debian-based environment (well, just some randomly selected Debian virtual appliance from the VMware store and a CodeSourcery toolchain installed in /usr/local) under VMware at work.

It's not too bad. I ssh to in from Terminal and have the home directory on OS X side available to Debian, i.e. all development files are outside the virtual machine. I use Xcode as an editor and project manager. X applications will probably work too if X is installed on OS X (installed by default on Leopard, separate free download in Tiger).

One small bug with this setup: Debian does not know when the machine has been sleeping, so I need to manually run "sudo ntpdate-debian" after resuming from sleep. Otherwise make is not happy.