Yeah, but all the things you'd want to use it for are already hogging CPU.

My advice: Window a small chunk from the center of the previous frame, and do a correlation of that on the whole window, and keep resolution small, and it might work. But a basic approach is gonna be glitchy as imaginable when the CPU binds and you get no frames for a half second. Trying to accurately return results when there was no jump, and handle real jumps like that gracefully, is not gonna be easy.

If I were just looking for an arbitrary way of conveniently scrolling down webpages, I'd go with overall image brightness. Scroll up when there's lots of light, down when there's no light, and somehow (lux sensor?) detect when the user's thumb is not there and don't scroll at all. Modulate light with your thumb like one of those optical "theremins" on eBay.