I think it is just the controller which is not well configured (still binded to your PS3).

I tried long time to configure my controller through Windows (and through VMWare too) and I gave up.

The best solution is to use a Linux distribution (for example, a Ubuntu live CD) and to do it through this operating system.