Maemo & augmented reality
 

As i was daydreaming and reading the newspaper i stumbled on an article on augmented reality applications.
I've seen some example movies on youtube as well on this augmented reality.

In this one the augm. rlty. browser Layar detects celebrity houses and this guy finds Brad Pitt somewhere in Amsterdam. (watch to the end to see brad pitt)
http://www.youtube.com/watch?v=6BybSbY9NR4

Now i do not see the fun in augmented reality myself, since it is not much more than google maps or whatever. (as it works with the gps)

What i'm about to say might be far stretched and i don't even see a practical use for it.
I was just wondering if anyone here has the knowledge to provide the insight if it would be theoretically possible to create an app to actually analyze objects which are snapped by the camera of the N900 or filmed by it. For example, when you film/fotograph a football, that the application would be able to say: object: football. brand: nike. available at: www.nike.com.
Since maemo is open source this would be THE operating system to think of.

this way you could create a way to analyze someones face or what might be even cooler, to capture a text on paper, and then recognize the characters to transfer this to a word document.
just as some scanners have text recognition.

Re: Maemo & augmented reality
 

Don't quote me on this but object recognition on the level you are talking about is afaik still on the research table. Very complicated to identify things like that based only on images, especially intelligently without building a database of "known" objects to compare with. Barcodes and rfids are two ways of identifying stuff (a camera can be used to identify barcodes, think there is something like that for s60 for instance) but that's beside the point...

Text recognition on the other hand sounds semi-feasible to me, but don't know what technologies there are. Most documents are really available electronically today anyway...?

Re: Maemo & augmented reality
 

thing is, the stuff the professor writes on the schoolboards isnt always :D

Re: Maemo & augmented reality
 

Good point. There are probably several other cases where it would be handy.

Actually I had another thought about the text-recognition in conjunction with AR. Suppose you are abroad and don't know the language. There are a number of occasions when you would need to read signs, menus and what not. How about if you could point your camera at the text and get a translated version of it in your display?

Now, text translation is not easy either but there are solutions (google, babelfish etc) and they will only get better. What is remaining is basically to process the image, identify what is text in it and process that into "real" text as input to the translation algorithm. There are techniques to identify 'glyphs' in an image and associate them with virtual objects. I wonder if that technique could be refined to treat each letter in an image as a 'glyph' and associate it with corresponding "real letter" to build up a text in a software..

Does anyone have thoughts on this..?

Re: Maemo & augmented reality
 

Semi-feasible was probably a fair assessment:

http://en.wikipedia.org/wiki/Optical...er_recognition

I actually think a text-translating AR app is not entirely far-fetched for "easy" signs with say roman letters and simple fonts (personally I think this would be really cool and actually usable..). Once you can capture text successfully you could not only translate it or transfer it to a document but also do things like automatically search for it on the web and whatever..

Re: Maemo & augmented reality
 

GOCR, a GNU open-source character recognition program in development:

http://jocr.sourceforge.net/

Re: Maemo & augmented reality
 

I think there could be a future for this.

http://en.wikipedia.org/wiki/Automat...ion#Algorithms

Re: Maemo & augmented reality
 

That one caught my eye as well. :)

Re: Maemo & augmented reality
 

Re: Maemo & augmented reality
 

intel showed off a demo of street sign translation close to the beijing olympics, but that was powered by a backstage stack of computers doing the actual translation (not that google could not provide such a backend with its translation service, just get from image to text, and dump it to them over a data connection of some sort).

Re: Maemo & augmented reality
 

Yep, there is even a language API from google.

Re: Maemo & augmented reality
 

Good find!
For those too lazy to click the link, it's a blog post about wordsnap OCR, a character recognition software for android.
Quote from the link:
The source is available here and there is a project log with among other things two youtube clips of it in action.

Re: Maemo & augmented reality
 

Is it possible to create an app like this

http://www.youtube.com/watch?v=8EA8x...eature=related
http://www.youtube.com/watch?v=tpaJB...eature=related

for the n900 or is it impossible because the N900 has no digital compass?

Re: Maemo & augmented reality
 

this is a link showing an idea for navigation with the same principal.
You can take a snapshot of an address and then the software recognizes the text, sending this to the navigation system. Sounds like a good application for the ability to use OCR on your smart phone.

http://www.infuse.be/v2/user-system-...avigation.html

"NOOO... iPhone has already done it!!" :rolleyes:

Re: Maemo & augmented reality
 

a few great resources for AR:
http://gamesalfresco.com/

and ISMAR conference (which is coming):
http://www.ismar09.org/

N900 is actually a perfect device for AR so far. Although many concepts has been developed on iPhone, they are never actually being able to put it on apps store yet. N900 will not have such hassle. It's all by community itself.

Re: Maemo & augmented reality
 

As far as I understand this is not possible without a compass, no. This is one of the reasons a lot of people where disappointed that it was not included in the specs.

There is a whole thread dedicated to how to hook up an external compass to the N900:
http://talk.maemo.org/showthread.php?t=31861

Re: Maemo & augmented reality
 

There is an open-source OCR enginge called Tesseract which seems relatively well accepted and documented. Might be a good place to start for someone interested in playing with text recognition in maemo..

It's been used in various licence plate recognition experiments:
http://www.youtube.com/watch?v=9vzR_H-jjHk
http://www.youtube.com/watch?v=BTpIpn7w1lg

Re: Maemo & augmented reality
 

you do know about the augmented reality subway app/poi app for iphone, right? it's very impressive
called: Bionic Eye

Re: Maemo & augmented reality
 

It is cool, but again one of those that need digital compass right?

Re: Maemo & augmented reality
 

sadly yes... hey Nokia, why ??? :(

Re: Maemo & augmented reality
 

There is already some cool ocr stuff going on on maemo:
http://www.joaquimrocha.com/2009/10/...maemo-preview/
(check out the video)

Thanks for the find, Capt'n Corrupt!

Re: Maemo & augmented reality
 

Hahaha.. No need to thank me, all I did was link to it (but thanks anyway)! The big win goes to Joaquim and his/her outstanding work! Kudos!

I agree that this is a very cool peice of tech. I'd love to 'scan' a page of text by taking a picture of it, and having it in a usable digital form (convert to HTML, for example, and view it in browser).

}:^)~

Re: Maemo & augmented reality
 

Well, finder's fee (?) ya'know. Would thank Joaquim too if I could. :)

Re: Maemo & augmented reality
 

Indeed. Or do this (the bold text).

Re: Maemo & augmented reality
 

N910/N1000 sales...

While i think there are some great things that can be done with Augmented Reality, we're "not there" yet. A friend of mine with little computer science background was all fired up about AR back in 2000 and i told him it was a great idea, but that by the time it could be easily done, it would no longer be his own baby.

Seems that is true.

Edit: Oops. Anyway, what i meant to say is that the lack of a compass isn't too huge considering the N900 is still just a stepping stone, and by the time AR is mainstream in ways that are really useful, new maemo devices will be out.

Re: Maemo & augmented reality
 

Well, you can do AR without a compass. Check the link in post #24 for instance. I don't think that is unreasonable at all, at least on a demo level.

Re: Maemo & augmented reality
 

That's not augmented reality; that's a camera app combined with an OCR app combined with a language API. It is an interesting convergence app, but not AR.

Augmented Reality would be if you could view the scene through the display and the text of the signs would all be translated and overlaid on the actual signs. As i said before, by the time AR is mainstream in ways that are really useful, we'll be past the N900. Doing what i just described in real time and displaying it in a useful way is just "not there yet".

It's "interesting" how you repeatedly cite yourself.

Re: Maemo & augmented reality
 

That is what I was talking about but...
...you are most certainly right. My point was rather obsolete since when it "is there" we will probably have compasses in the maemo devices anyway. I just meant many people are equating AR with the need for a compass, which is a misnomer.
Do I repeatedly do that? Haven't realized, thanks for pointing it out... If I do it is neither meant to bang my own drum, insinuate that what I say is the truth nor offend anyone. My apologies if it comes over that way and I will try to tone it down. I just do it because I remember what I have written, feel it is suitable in the context and don't want to repeat myself.

Re: Maemo & augmented reality
 

Ok, i see what you were saying.

One thing that has come to mind while browsing the download options here and writing that previous post is that AR, as the apex of convergence of both hardware and software design has incredible potential, but equally incredible barriers. Think about all the little apps that are just great at one thing--there are quite a few. Then, as the functionality becomes more complex and there are more combinations of ways to do things, it's harder and harder to put together an app to do things in a slick and user-friendly way.

By the time you are capturing video, doing OCR and object recognition on the stream, manipulating the data, creating virtual objects to correspond with it, positioning those objects in the 3D space of the video image, and projecting the image in a way the user can interact with, and providing a useful UI to do so, there are multiples of world-class functionality all going on in a real-time, seamless way. That's a lot of complexity for any one developer or team to tackle.

All of that digression is a way of saying that we can start the Maemo AR revolution in stages, even right now, by building up the blocks and refining them. When the hardware is there, the optimized components can be integrated.

Make your apps modular! :)

Re: Maemo & augmented reality
 

http://www.youtube.com/watch?v=aMoesX4POfk

Already, AR is getting to the point where it's already useful.

Re: Maemo & augmented reality
 

hmm, now i got thinking about street signs with ir 2d barcodes that allow a computer to look up the sign online and overlay a image of it in your native language, thanks to other ir markers in the corners to get the orientatioon right.

watch gits:sac and see how barcodes are displayed next to or replace text...

Re: Maemo & augmented reality
 

How about this

http://www.linuxfordevices.com/c/a/News/PABRcom-WXHMD/

http://www.vuzix.com/iwear/products_camar.html

Re: Maemo & augmented reality
 

biggest problem there is that you loose the ability to have everything in focus at the same time.

what i would love to see is those new LG or samsung transparent LED screens, with a camera aligned in the direction of the view (so that you line up a crosshair/mark of some sort to photo something).

that way, you can create a subtext or overlay, without having to convert everything into video like current android/iphone AR do it.

Re: Maemo & augmented reality
 

"see through" displays are great for glasses (hmd) or car windshields, but I fear in a handheld the field of view is a problem. Much better to display an image and overlay things there. Not only you can use a a larger FOV, but you don't force the user to hold the thing right ahead of him, and you can also guarantee the artificial overlay will be properly aligned to the objects being displayed.

Think about small camcorders... Today the trend is having an lcd display that you can move around. It's much more comphortable. But I repeat: see-through AR displays have their place in the HMD world.:cool:

Re: Maemo & augmented reality
 

Here's an approach to doing a semi-augmented reality approach with IKEA: http://mobilebehavior.com/2009/12/01...d-reality-app/

The lessons learned here should be sufficient enough for anyone looking to do augmented reality apps with Maemo, and what kinds of education that would need to go along with the application/service itself.

Re: Maemo & augmented reality
 

is it possible to port Layar to the n900?

Re: Maemo & augmented reality
 

Looks like it's closed source so you'd need to ask the app developers. It does look rather cool though - nice to have a single app that can combine multiple sets of overlay data.

I wonder if we could reverse engineer the layer data and write our own app to do this (obviously no compass is a pita, but proof of concept and all).

In fact it might not even need to be reverse engineered as there's a developer page here: http://layar.pbworks.com/w/page/7783228/FrontPage

Re: Maemo & augmented reality
 

Right, I just had a play with Layar on my wife's old Android phone, and it does seem to be quite cool.

Re: Maemo & augmented reality
 

like most augmented reality apps, Layar uses a compass, which we don't have on our N900s.
so even if somebody were to port it, it wouldn't be of much use.