I'm visually impaired (Aniridia in both eyes), and I'd really love such an app
There already is an app which I cannot recall the name of currently that can translate signs and similar things using OCR. You might want to have a look at that too.

My biggest gripe about Maemo 5''s UI is how incosistent the font size is in the different aspects.
i.e. for me the numers in the phone app or the details in the app manager are unreadable