Software emulation of dualtouch input possible?
 

I've come up with a few algorithms for translating unitouch input, as well as a combo of unitouch input and accelerometer input, into specific uses of dualtouch input (each is more suited to one or another situation).


But for those algorithms to be of any use with programs made for multitouch, we need a scriptable way to provide multitouch input, somthing that would trick programs into bdelieving true multitouch is happening, and which you could program to suit the different use cases. Do you know of anything like that?

Re: Software emulation of dualtouch input possible?
 

I think Lcuk was fooling around with this for awhile and he actually had a workable demo.

Re: Software emulation of dualtouch input possible?
 

Lcuk's youtube multitouch demo:

http://www.youtube.com/watch?v=dJWvvn_cePM

Re: Software emulation of dualtouch input possible?
 

You can achieve the same result with Qt with really few lines of code, but you have to insert multitouch emulation directly into the code of your application.

@Tiago
Since Maemo's X server does not support multitouch I don't know if what you want to do is possible. I mean, you can send two input signals simultaneously, but I think they will be processed one after the other.
Atm, I think that the only possible solution is to implement multitouch emulation directly into applications' code.

Re: Software emulation of dualtouch input possible?
 

I imagine it would involve a driver for the virtual multitouch device

Re: Software emulation of dualtouch input possible?
 

Question: The way Stantum screens detect multitouch is they are able to get contact data from every grid point on the screen individually and based on pressure/distribution are pretty easily able to figure out what spot is one touch, and which is another.

Is there something about the actual physical make up of the N900 screen that makes it incapable of this? Or is it just that whatever hardware does the processing of the inputs isn't meant to figure this stuff out? If the latter, what level of access, if any, can the operating system get to the raw outputs of the screen hardware? (Now, stantum screens have their own preprocessor that does the calculations/whatever to tell the device where the touches are, so if this was even remotely possible, it would probably rape the CPU and be practically useless. But I am still curious, is there something inherent about how most resistive screens are built that makes them incapable of detecting presses the same way, at the very very fundamental level? Or is it that Stantum was the only one that bothered to figure out how to calculate separate touches from all the inputs?)

Re: Software emulation of dualtouch input possible?
 

I think that there are different methodes to formulate dualtouch on a singletouch device. (My propositon is still touch&swipe).

The problem for the moment is only how to make believe the n900 that he is a multitouch device.
Then you can send emulated multitouch events to the n900.

Probably you need to find a multitouch driver for the n900.
That should be no problem. Because there exist usb-multitouch touchpads that can be used with arm-linux.

Re: Software emulation of dualtouch input possible?
 

Like i said, there are different algorithms, some better for some situations and others for others.

Re: Software emulation of dualtouch input possible?
 

Imagine the Stantum resistive screen as a lot of small finger-sized N900-like resistive screens layered out on a grid.

You wouldn't be far from the real thing. Also, you can guess why this is different from what the N900 has.

Re: Software emulation of dualtouch input possible?
 

fingersized? I thought they had hyperpixel resolution sensing of multitouch...though from what i remember from the videos perhaps they just did a bit of plausibly deniable slight of hand, in a moment showing all ten fingers touching the screen at once all enough apart from each other to not trigger artifacts and then switching to talking about the high precision of a single pointy thing against the screen....dunno what to believe...

Re: Software emulation of dualtouch input possible?
 

Well, I guess the cell's size would make for a nice variable to change between price points (cheaper ones having bigger cells).

Individually, each cell has the same precision as your N900 screen... which means that as long as not any two fingers fall on the same cell, the screen will be very, very precise. When two fingers fall on a single cell, that cell will report the average point much like the N900 does.
I do not think that would be a big problem since:
a) With small cell sizes, you would need to have very small fingers to be able to trigger this,
b) You might be able to do more accurate fake multitouch on such a smaller cell.

Re: Software emulation of dualtouch input possible?
 

I think that the ideal solution could be:

• Modify N900 screen driver and insert multitouch emulation directly into it (btw, is it open source?)
• Patch X server for multitouch support

What do you think?

Re: Software emulation of dualtouch input possible?
 

I think you gave yourself the answer.

My proposition is:
A second device in Xorg (like a USB multitouch touchpad based on uTouch).
This device could receive emulated multitouch events.
All the events of the single touch driver will be replicated on that driver.
And and all the rest can stay the same.

Re: Software emulation of dualtouch input possible?
 

Still need a way to script the different algorithms (I have one for a virtual thumbstick + multiple individual buttons (one thumb on the stick and the other pressing and releasing buttons one at time), one for typing fast on a virtual keyboard, multiple buttons (kinda an hybrid between the joypad one and the fast typing one), then there are those for faking the classic gestures, like tilting the device side to side while holding a modifier key to do the twist and fwd/back for pinching, holding a modifier key to turn regular dragging into two finger dragging (with parallel and direction changing variations), for "raw" twofingering use the screen for one finger and the accell+ proximity sensor for the second finger, and so on)

Re: Software emulation of dualtouch input possible?
 

Does the WebOS progs that people have been playing with preenv use a standard way to read multitouch input? And what standard for multitouch input does MeeGo use?

Re: Software emulation of dualtouch input possible?
 

Basically, "different ones". The Meego one probably being the sanest one, depending on what the N900+1 ships with.

My gripes with this thread are:
a) Unaccuracy/uselessness of fake multitouch
b) Lack of applications that would use such input

If you think you've solved both problems, you might be on to something. :) But I suggest you firstly implement a demo (as a standalone application) or something like that to "polish" the ideas, much like lcuk did.

Re: Software emulation of dualtouch input possible?
 

My software solution (touch&swipe) doesn't use fake multitouch.
And my driver solution is based on standard multitouch drivers and frameworks.

Re: Software emulation of dualtouch input possible?
 

:D

Interesting idea...

Re: Software emulation of dualtouch input possible?
 

Don't look at what happens on the surface (where your fingers are interacting with the screen) but understand what's going on underneath all that...

The system still only sees one point, and that's a big big difference compared to true multitouch devices.

Re: Software emulation of dualtouch input possible?
 

The system sees one point at different times at different locations.
That's very much and enough to emulate all multitouch gestures in a specific way.


Multitouch is not useful on little devices. The screen is to little. And the precision of a finger is nothing against the precision of a stylus.

There are good single touch emulations of all multitouch gestures. We have only to implement them.

Re: Software emulation of dualtouch input possible?
 

The joypad algorithm for example, would work like this:



On one side, lets pick the right one, you draw a thumbstick, with your right thumb you can drag it to anywhere you want within it's range (the circle or square area representing the range of motion of the stick), depending on the desired mechanics it will either snap to center or rest where you left it when you raise your finger.


On the left side, you draw several buttons (i imagine around 9 might already be pushing the algorithm to it's limits), the distribution of the buttons and the positioning and size of the group would matter, some configurations would make it too hard for tunning the algorithm. The buttons you must never slide from one to another (perhaps if they were just a vertical line of a few buttons sliding can work), you should lift your thumb and then press down on the next button.


The two sides can be used concurrently (that means you use each one without needing to interfeer with the use of the other.


The key part is the algorithm to separate the two inputs. With the N900's sensor, two touches result in the cursor being placed in point between two touches, and if you touch them one and then the other, the cursor will jet from the position of the first towards the midpoint. This is the building block of the pure screen algorithms.


If it is just the joypad being used the cursor stays in the joypad area.
If it is just the buttons being used, the cursor stays over one of the buttons.

And now the interesting part:

If the joypad is being used and then a button is pressed, the cursor will jump from the last joypad to the point midway between that and the pressed button, which gives you which button is pressed.

If a button is being held down and then you pull on the joypad, the cursor will jump to the position between the button and the current joypad position.

And when a button is pressed and you wiggle the joypad around, you already know which button is pressed, so it's just a matter of extrapolating the joypad position.


Questions?





ps:it might involve a little bit of practice depending on how you usually touch the screen with your thumbs, the position of the cursor with two fingers on is the average of the two contact points, but it's an average weighted by the pressure of each thumb (with a little practice you can wiggle the cursor back and forth between the thumbs)

Re: Software emulation of dualtouch input possible?
 

Here's a common multitouch gesture with just 2 fingers (most multitouch screens are capable of processing more than 2 points at a time), let's see how you emulate this:

2 touch points on screen = zoom, rotate and pan an image.

Yes, all 3 can be done together at the same time:

- When the two fingers are moved closer/apart: the image will zoom in/out)
- When the two fingers are rotated against the other: the image will rotate according to the direction
- When the two fingers are moved around screen (maintaining their relation to one another): the image will pan.

This is commonly used in image manipulation apps, while the single finger gesture is used for the main interaction:

For photo browsing: single finger actions are used for changing photos.

For drawing apps: single finger actions are used for drawing.

Notice that the system does not need to change into different modes via UI buttons or key combinations, it just recognizes the different types of multitouch and applies the operations immediately.

Re: Software emulation of dualtouch input possible?
 

Tiago:

Let's keep this simple:

A pt1 X
B pt2 Y
C pt3 Z


A and Z will be the same as C and X. They both register as pt2.

MVP (Multiple virtual pointers)
 

This is only ONE very simple emulation of multitouch:
(I call it Multiple virtual pointers MVP).

Instead of a single mouse pointer you have several (say 7) virtual pointers.

The principle is very easy (I call it "Jump to the next pointer" (JNP)):

If you tap on the screen only the virtual pointer next to the tapped location is moved to that place.

@ysss:
I didn't see your posts:
This problem is solved by my tap@swipe emulation:
If you tap on the screen followed by a swiping this is recognized as a combined gesture with the following interpretation.
The tapped spot is the center of the transformation.
The beginning and the end of the swiping define the transformation (rotation, zoom)

Re: MVP (Multiple virtual pointers)
 

I'm not sure if you read my post (I call it INSIYRMP).

And what would 7 stationary pointers are used for on screen, when you can only move 1 at a time? (I call that WTF).

Also, I was mainly responding to this claim of yours...


I have a feeling that you're missing the point of multitouch.

No pan and the device has to change modes via a button/other gestures/triggers if you would like to do other thing than just rotate and zoom.

The difference is subtle on writing, but when you play with a multitouch tablet then the intuitive feel comes through very clearly.

It's even more pronounced when you combine those capabilities. Left hand with 2 fingers manipulating the image (zoom, pan, rotate, all at the same time) while your right hand is drawing. For example.

Re: Software emulation of dualtouch input possible?
 

_____________x
____________u y v
a b c d _______z


@TiagoTiago

I had the same idea some time ago, it's definitely possible to achieve this result, although obtaining thumbstick position while pressing a button could be a bit tricky.

Re: Software emulation of dualtouch input possible?
 

I already tried this concept long ago in a nonreleased build of DrNokSnes. It worked very well... as long as you used two stylus. With both fingers I found no way to be able to make it precise enough for game usage.

Re: MVP (Multiple virtual pointers)
 

There is no need that all movements are done on the same time.
You can pinch by moving one edge and then after this moving the other edge.


The pan is a translation. It doesn't need any center of transformation. It is a transformation that is done only by swiping on the n900, without any change of mode.

You can't paint on a multitouch device at all. The fingers are to thick. You can only do the drawing of a 4 year old child.
And if you do serious painting you don't zoom, pan and draw at the same time.

An intelligent person does one thing after the other one. If you do everything at the same time everything will be bad.

Re: MVP (Multiple virtual pointers)
 

Well, maybe you don't. But the multitouch bearing competitors have been exploring these combinations and coming up with very interesting new interaction paradigms.


Sure there is. There is pan mode and rotate+zoom mode.

Here is something to open your mind. It doesn't hurt to see what the competitors are doing.

http://www.youtube.com/watch?v=jYSRQDxeGZY

Re: MVP (Multiple virtual pointers)
 

Of course, we know that we will never be able to transform our N900 into an IPad, we only want to cover some simple use cases, nothing more.

Re: Software emulation of dualtouch input possible?
 

The problem there is you made it too simple, lemme try to squiggle out a drawing of what i mean:

http://img233.imageshack.us/img233/1...algosketch.png


Your example there is more of a job for the multiple buttons on each side algo; either way, you forget about the time dimension, at any given moment you always know at least one side for sure because of where the cursor moved to before.



Regarding the accuracy of the thumbstick while combined with buttons, i don't know how to program well enough to actually code a proof of concept to test the limitations, very preliminary tests using the accuracy thingy in Healthcheck seems promising, some sort of filtering will probably be needed, but it looks like it's worth further investigations.

Re: MVP (Multiple virtual pointers)
 

Sorry, I was trying to make a point to gerbich, who made a claim that he could emulate all multitouch gestures with his touch and swipe.

Re: Software emulation of dualtouch input possible?
 

I prefere this one:
http://www.youtube.com/watch?v=66RBfrBgL2E

Your Video is on a iPad. On an iPhone with the same size of the n900 you couldn't even draw 1 line.

The user of the iPad needs special wearing and and special stylus.
He can't draw details because the screen is not pressure sensitive.

What he does is very dangerous. If the pinch gesture is not correctly recognized he will have a false line instead of a pinch gesture. I preferre the buttons on my n900 that are less dangerous for such an exercise.

In this case a mode change would be very useful. But the iPad has not enough buttons.

It's better to paint on the little n900 than on the big iPad. Because the screen and the buttons are better for such a work.

But for panning and zooming there is no mode change needed. Panning is done by the swipe gesture. Zooming and Rotationg are done by tap&swipe.

If you want to work without buttons like the user on the iPad I propose a standard tap&hold for mode change: Between drawing and moving.

The n900 is much better for painting than the iPad that is much bigger because the concept of the n900 is more intelligent.

Re: Software emulation of dualtouch input possible?
 

@gerdich, I think you've successfully trolled yourself. Just reread the replies above and come to your own conclusions.

If that didn't help, perhaps you'll need to wait to own a multitouch device yoruself to understand.

Re: Software emulation of dualtouch input possible?
 

Hi,

I think software emulation of dualtouch is possible (dualtouch will be very useful for example in palm pre games).


Videos about resistive screen emulation

http://www.youtube.com/watch?v=ld_kIpLWa3Y
http://www.youtube.com/watch?v=XndW7YlvYjU

Maybe this will be helpful http://lii-enac.fr/en/architecture/l...ndex.html#xorg

I think easiest way (to emulate multitouch in palm pre games) is write software to emulate points on screen from input keybord, for example coordinate point (100, 100) will be "A" key, coordinate point (200,300) will be "B" key etc. This software should have configurable coordinate points assigned to input key's.

Sorry for my bad Englisch :p

Re: Software emulation of dualtouch input possible?
 

Okay, for once I'm skipping most of the thread before saying anything, I just wanted to reply to the initial point early on about Stantum screens specifically:

The way stantum screens are able to pin-point a touch location with more precision than they have sensors for is simple math (well, calculus, if you understand where the basic math transits over into calculus, but whatever) - you detect "this cell is reporting contacts on [set of points], with (amount) of pressure; this cell is reporting contacts on [other set of points] with (other amount) of pressure". And so on for every "cell". Even with only four different cells (assuming they're arranged as squares) you can calculate where in between those cells the touch is.

And I'm not 100% sure the response javispedro gave me is accurate, but it makes sense. The layman's explanation on the Stantum site shows that the two 'layers' of the screen have a bunch of tiny transparent 'separator' dots. Logically, I can understand how this can compare to having a 'bunch' of mini-screens. I can see how this prevents the normal all-touches-register-as-one-big-touch problem. Imagine a giant cellophane layer stretched over a rectangle. If you press it down, it bulges in. If you press it down in two places, simple physics makes it the two inward bulges blend together into one elongated bulge (though this also suggests that with proper mathmatic calculations, you may be able to distinguish single 'point/circle touch' from 'blob/odd-shape touch', if those calculations don't force too much resource use constantly.

Anyway, if you stick a bar along the middle of the empty space over which the cellophane is stretched, the bulge from a press across one 'section' can't reach across the other. So you can now detect two touches so long as they are in separate sections, AND, if you press down both sections at once with something really really big, you can calculate the center of the really big touch by using the positions/pressure of touches detected in both sections.

Now expand the concept to practically microscopic 'sections', and you have an infinitely-expandable multitouch screen. (Unlike modern capacitive screens, which have to be built in a way that lets them distinguish a certain amount of current changes, as I understand it.)

And, I just had another revelation while typing this: When you scale these 'sections' to the point where they're small enough to be barely distinguishable visibly, something else happens - you can make the distance between the two screen layers smaller, and the actual screen material more flexible: Because doing that on a normal resistive screen would make it possible for the screen to just 'sag' on itself under it's own weight, or would make it too imprecise - however, having these microsections means that you can afford a more 'flexible' screen material, and a smaller gap between the two screens. Which means that suddenly, creating resistive multitouch technology simultaneously allows you to increase the sensitivity of the screen as a side effect. Which more than explains, in my mind, how Stantum can claim that their screens can both process infinite touches (they can process however many touches as they have cells on the screen, and that's in the thousands on a modern screen, which is more than you'll ever realistically be able touch the screen with simultaneously), and how they can claim great sensitivity - making the resistive screen support multitouch also makes it practical to significantly increase sensitivity.

...Actually, the whole thing's kinda genius. Like, I'm really sitting here going "holy **** that's brilliant".

Re: Software emulation of dualtouch input possible?
 

Finished reading thread. Most of the discussion has flown naturally without anything for me to add, however, I REALLY want to point this out, because so few people seem to get it:

The time aspect Tiago mentions is right on. If you have:

A 1 X
B 2 Y
C 3 Z

...then yes, dumb point-of-press analysis will say C+X and A+Z is the same thing. HOWEVER, if your input method is hacked to buffer the last few milliseconds of contact, and checks where the last press was (INCLUDING noting if the last few milliseconds had NO pressure on screen, and having a couple other exceptions which can be logically deduced), then it's pretty easy to tell between person was JUST pressing A, then pressed 2, or person was just pressing A, then also started pressing Z (while not letting go of A); vs person was just pressing C, then pressed 2, or person was just pressing C, then also started pressing X (while not letting go of C).

This STILL has flaws. And unless you have a LOT of spare processor idle time, you probably can't do this well unless you design your own preprocessor hardware that does this (which is how Stantum screens keep the device from having to do all the calculations I alluded to above - they have a chip that does that part, and outputs the specific 'touches (coordinates, pressure, etc)'.

However, this does mean that if someone is willing to sit down and code a robust system like this, at least SOME basic (two joysticks/control pads) methods can be implemented, even if some of the 'center' areas are shared by multiple combinations of individual touches.

A similar concept can easily be used by someone who understands calculus and physics well enough into using accelerometer values for gyroscope emulation.

Much like the above, there's still instances where doing something not-multitouchy/not-gyroscope-y will result in this emulation interpreting it as such. But if you're at all logical about it, and can spare the extra memory, processor load, and tiny amount of (ideally not-human-perceptible) lag, you can figure out how to narrow the amount of such problem instances down to very rare situations.

Re: Software emulation of dualtouch input possible?
 

Probably this can be interesting for you

http://www.fcl.fujitsu.com/en/releas...0101109-3.html

Re: Software emulation of dualtouch input possible?
 

I am going to be infinitely pissed if this gains traction, but Stantum's long-existing ones don't.

Re: Software emulation of dualtouch input possible?
 

Indeed. Since there's nothing specific other than it being a "software based solution" I'd say they're trying to sell fake multitouch.