Which vnc server are you using? I had some problem with ultraVNC, but works fine with tightVNC. No special setting on client side just enter xxx.xxx.xxx.xxx:5900. You are entering IP address instead of hostname right?