The x server you *want* to use is the xomap x server. xfce4-session just starts xfce in the running x server. So you can say 'DISPLAY=:0 xfce4-session' and have a running xfce desktop as long as you already have an x server running.
Here are the errors I got when I tried to run xfce4 directly on top of the XOmap x-server (I didn't shut down the XOmap server): Code: /usr/bin/startxfce4: X server already running on display :0 Agent pid 3377 The program 'xfce4-session' received an X Window System error. This probably reflects a bug in the program. The error was 'BadName (named color or font does not exist)'. (Details: serial 91 error_code 15 request_code 45 minor_code 0) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the --sync command line option to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) Agent pid 3377 killed
/usr/bin/startxfce4: X server already running on display :0 Agent pid 3377 The program 'xfce4-session' received an X Window System error. This probably reflects a bug in the program. The error was 'BadName (named color or font does not exist)'. (Details: serial 91 error_code 15 request_code 45 minor_code 0) (Note to programmers: normally, X errors are reported asynchronously; that is, you will receive the error a while after causing it. To debug your program, run it with the --sync command line option to change this behavior. You can then get a meaningful backtrace from your debugger if you break on the gdk_x_error() function.) Agent pid 3377 killed