I should have realised. So thats why the jitter (it wasn't bad anyway) felt quite constant across the zoom levels and didn't get noticeably amplified by narrowing FOV. Filtering is doing it's work.

[EDIT] Now that I've used Stellarium-N9 bit more I'm coinvinced that the filtering actually works very well. One can even zoom up to Saturn's moons without losing track of them and lag doesn't seem to prevent finely adjusting the view. Maybe on the first try my hands were just shaking from the excitement of trying this wonderful app.

(Also removed the ugly icon attachment from this post as the new ones few posts down are somewhat better)