After some use, my 11.04 + gnome really works well. Fast and smooth, stable, not candy but sexy enough. I am completely satisfied. (some issue with bluetooth, but it was worse in Vista).

Xfce on the other hand does not seem that responsive, and is still down on features.