I get this problem immediately upon loading penguinbait's joystick.ko kernel module, too, whether I have anything attached or not.