kernel module or X extension could probably implement a keylogger although neither is exactly trivial to write. Using standard command line tools I could dump passwords for IM, Mail, and network access trivially. Grabbing cookies might allow for attacks on several sites like google as well.

But the user still has to install and run this malicious software so some amount of social engineering is required.