[Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

http://mhsueh.com/mir/mir-translator_large.png

MIR Translator is an application for translating text in photos taken by the N900 camera. It experiments with novel interactive techniques to perform text recognition. Currently the app supports translation to and from over 30 different languages. Video Demo: http://www.youtube.com/watch?v=VT-XsUPXCFA

Requires internet connection to work.

Package is available in extras-devel. Be sure to restart after installation if FCam drivers were not previously installed.
Maemo packages URL: http://maemo.org/packages/view/mir-translator/

The app relies on the text recognition happening off the device and in the cloud. This allows the application to support the maximum number of languages possible while keeping the install size as small as possible (~1.5 MB). It also improves response times, given a fast connection and server side.

Features:
-A viewfinder is used to capture images with the camera. The user selects text to translate.
-If the recognition is erroneous, the user is able to correct the recognition via manual segmentation, manual thresholding, and manual candidate selection.
-Supports more than 30 input and output languages.

Limitations:
-Somewhat limited performance. This is because the recognition service resides in the cloud as a web service. I am hosting the service myself, so server capacity / speed is as a result limited.
-Recognition accuracy is currently quite dependent on the quality of the input image. That is, images should be well and evenly lit and have good contrast. It's best if the text is sharp, with typographic-like fonts and non-complex backgrounds.

This project fell out of a project I was doing in school (its actually my first maemo app!). I've recently adapted it to be deployed publicly, so its in an early development phase. Naturally there will probably be some bugs, and there is much room for optimization. I don't expect the application to be powerful enough for the general case of translating arbitrary text-- but simpler cases should be do-able. Deploying it publicly is fun and I'm hoping it speeds feedback + development.

I'm keeping the web service up and running on my own student funds -- its not cheap! Any and all small donations are greatly appreciated (Donate Link: Donate). Hope you enjoy the app!

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Will try this out! Thanks for your efforts!!

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

do you plan to take part at the MeeGo Coding Competition 2011?
You really should!

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Ummm.. WOW! Very nice looking app. Can't wait to give this a try.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Is it possible to make the app work offline with no internet?

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

How much space would it take up for all the languages? Just english?

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

As far as recognition, It will take some work, though not overly much, to make the app run completely locally. It might be ~2-3MB per language, so around 90MB total, plus an additional 150MB or so for the Asian languages, which require much more data to support. Alternatively, one might just choose a few languages to support and then download support for additional ones as needed.

I'm not sure about the translation part though, right now that depends on the web...

One of the things I might try next for the app is to enable completely local processing as an option.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

This looks fantastic.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

maybe somehow link it with Mstardict\qstardict dictionaries?

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

It has broken my camera twice. Both times I needed to reboot to get it working again :/

I have kernel-power and CSSU.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Thanks for the feedback. Am definitely going to look into it.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Observation of user interface from watching the video:

Current flow:
Click Send>Translating label>Done label>Click Result>Result screen

Why not have the UI go directly to the Result screen without user input when Translating is complete?
Click Send>Translating label>Result screen

This would increase speed & reduce user frustration from not noticing that translating was done and results could have been seen sooner.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

rotoflex, that's a good idea. I actually thought of that myself and am going to get around to it in the next few iterations.

Regarding the app crashing the regular camera app. A quick investigation revealed the problem. It was actually a bug in some FCam API example code. It doesn't release/close /dev/fb properly when the app shuts down. Should be an easy fix I'll get to right away.:)

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

It works until I crop to the translation area but when i tap on "send" it crashes... :(

I just realized it crashes when the phone is not connected to the internet instead of requesting connection.

It also crashed during editing of badly captured letters

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

This is really a good application!

I agree with users asking for an offline app. If you're in a foreign country and there's no wifi it'll be very expensive to use it.

Have you seen the iphone 4 app World lens? Is it possible to do that with the N900?

You must take part at the Meego coding competition.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Weeeee more apps ^_^ .

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Really cool program! Thanks!

I know this is just personal, but It would be lovely to have hindi on here too. :)

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Very impressive work... workes great for me - i mostly use it for japanese translations.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

For some reason this app doesn't work for me. It recognizes the text properly, but the box which is supposed to show the translation is always completely blank. Has anyone else seen this problem? Does anyone know a solution? I'm running the plain Nokia kernel, and CSSU.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I have had the same issue. I have a plain PR1.3 install so doubt CSSU is to blame. I have just installed it though, was going to play with it a bit more before saying anything.

BTW this is both useful and cool. Thanks for creating it.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

This looks really good and a boon to the n900 - especially if an off-line option becomes available. "Photo-translator alpha" from OVI although promising, is buggy and needs one to wait each time as it reconstitutes image icons.

Thanks to dev for effort!

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Implemented your suggestion.

Also fixed the problem with the application breaking the normal camera app after use.

New version is uploaded.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Would it be possible for you to add an 'export text' to text file feature?

I've been looking for some sort of OCR software so I can take a photo of a document and edit, and then email the text; your app can obviously do the OCR side, so presumably it wouldn't take much to implement :)

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Thanks for the feedback. Just uploaded a fix for crashing on sending without an internet connection. The manual correction mode might still be a bit rough, especially for crops that extend beyond image. Will look into it.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Great app!

Option to load an image file, as alternative to take a photograph would be fantabulous.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

This looks completely marvellous! It makes me proud belonging to this community.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I have a few suggestions to improve this great application:

1. Allow dictionaries download (trasnlation available offline) - i believe many people will use this application while travelling, which means in most cases won't have internet connection.

2. Allow translation from a previously captured image.

3. Allow for manual image correction before the send for translation - sometimes it says there was no text recognized after taking a photo - especially if the photograph is not black text on white page.

Great work :)

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I am not able to find this app in the repositories and also nowhere on the internet.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I made some jaws drop with your app! Excellent work! Thank you very much :D

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I'm also not seeing it :S I have all the repo's enabled. It was there earlier too! Bleh.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Enable extra-devel and you are good to go

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

It would be good if the user could correct the result text in the "Selected Text" area of the results which show in the 3rd step. I have tried the program in a number of languages and it seems to have trouble with semicolons and colons. If I could simply put a cursor in the "Selected Text" box and eliminate (or edit) the colon/semicolon (or any other error) the translation may then work.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I not able to install it, i get conflict with power kermel.
does this app uses Fcam?

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Would it be possible to split this app into OCR app and translator app? That way, it could be way more versatile.

For example, those who want only an OCR app (and rarely, if ever uses translation), could have a lightweight OCR application. Those who want the translation also, should be able to download a seperate plugin/applcation that can be called directly from the first ocr app.

The advantages to this are:
a. reduction in size/footprint of the OCR app, if translation not required.
b. For those who have the translation plugin/app installed, it functions just like one joint app which takes images>OCRs it>translates it.
c. For those who want only translation (for eg, text from an ebook/pdf file/webpage/etc, they can install and run just the translator app without OCR plugin. again, lower memory footprint.

Please consider this methodology. Also, you may be able to enter the Meego Coding Competition 2011 with two seperate apps :D

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Anyone has problem with the scroll bar? When the texts to translate need several lines to display, there are scroll bars there but I can't use them. Maybe a bug?

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I wasn't able to see it too even after regularly checking updates from the dev repositories .

So i went over here : http://maemo.org/packages/package_in...nslator/0.0.9/

And downloaded the .deb file manually . After installing the 0.07 version i got an update for the 0.0.9 version the next time in application manager .

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Yes, this is a known issue I haven't been able to resolve. For now you can click inside the window and use the keyboard arrow keys to scroll up or down. Same with highlighting: use the shift key.

Thanks.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Mine eventually showed, I was updating my catalogues over Three Uk's 3G and then updated them on home wifi, not sure if this was a coincidence or a fix. Thanks anyway!

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

I have troubles getting the mir translator to work.
Currently installed verion 0.0.9 and FCamera 0.1.7-1

I've restarted my N900 after installing above software (and several times since that). When I start mir translator, I get the camera output with just one button visible: the close button.
Even the close button does not respond easily. Looking at "top" from xterminal, the load average is over 4.
After executing mir translator I can no longer use the "regular" camera application until I have restarted the N900 as the camera will crash.

Re: [Announce] MIR Translator - Translate text captured by the Nokia N900 mobile camera
 

Japa-fi,

Thank you for your patience. The beginning screen with just the close button is correct. If you can see the viewfinder being displayed, try using the camera shutter button to take a picture... but I feel maybe you aren't talking about this.

Unfortunately, I am having difficulty reproducing your other two problems. For now, I can only suggest you try uninstalling both FCam drivers as well as MIR Translator and reinstalling. I have an idea about what can be causing these issues, but it's just a guess. I will include a fix in the next update.