I haven't tried geocaching, but I have been screwing around with the GPS.

If you want to see your GPS drift, use Maemo Mapper.
Make sure you have tracking turned on.
Go somewhere with a good satellite lock. Now sit there and twiddle your thumbs or something for a moment.
Look at the "track".
There, visualized "drift".


A note, the compass only works if you're moving.