Should be easy enough to do, record data at full rate and perform a 2D correlation on each frame to work out which way it's moved (on average).

Try using the example code for the camera, or the maemo-barcode code as a starting point. I'm happy to advise/contribute to see if it can be made to work.