The "Hand" icon (soon to be remove as it is offensive in some countries in favor of a different one) now switches to PAN mode.
In a near future in PAN mode you will also be able to move between pages by flipping with your finger to the left or the right.

I don't think I will bind panning to anything different than the hardware cursor keys.

I am also thinking at this solution:

I like to hide the toolbar when you are drawing on the screen, so that you have maximum space. When you stop drawing and after a very short timeout (0.2-0.3 secs [configurable]) it will appear again.. I can also make appear two big arrow overlaid buttons (a la PDF reader) so that you can go to next/prev pages and, may be, I can make appear something you can drag to pan around?