That should work, yes. Which VNC server are you using? And is there any firewall on the desktop which might be blocking things?