I've been doing a bit of research and it seems that GTalk doesn't work with a Google account which doesn't use a GMail address. I guess that's my problem as I've been using a Yahoo email address.

It'd be nice if Google documented that anywhere but nevermind.