I had the same problem. Basically it appears that GAIM spawns a bunch of tasks and quickly fills up all the RAM. The solution was easy for me though:

Use the other package from the Maemo page. There are two versions of GAIM out there. One is very buggy, and one seems to work reasonably well (and can even co-habitate with the browser...barely). The good one is version "v 1.5-dk-1". If you have the other one, here is the link to "v 1.5-dk-1": Working GAIM Port

Uninstall the other one first.