Augmented reality based on gestures. It's not easy to build but not impossible. Also having just a single camera may be tricky (since only one camera can't analyze the depth of field).

Now here is how I think it works. In good light conditions, the hand color have a certain colour. The phone being pointed at a screen it can't be tricked by anything else. So it scans for objects with that color and if they find something like they search for something like this shape:



Because i think if you keep your hand like this:



holding 2 fingers instead of one, the soft gets confused and won't work anymore.

It would be nice to implement stuff like this, but also unconfortable. because you can have to stay very close to the screen. Another idea was to hang this to your neck, hock it up to a pico projector and do all that from whenever you are staying!