touchLessToys – touchless is better than multitouch
for whom: I started this interface experiment as a personal project, but people seemed to like it, so it’s now being used in a few commission works.
when: started in 2008, going on when needed.
what: the basic idea was to be able to interact with digital tools in a natural, immersive way: my touchLessToys give you the possibility to drag, stretch, punch virtual objects just moving your hands in the air. No need for controllers, touch sensitive surfaces or minority-report-like gloves.
Technically speaking, I use a webcam, the user’s body and some good ol’ math. The first toys were coded in Java, so they were platform independent; in later scenarios, when performance issues became more important, i switched to C++.
For the computer vision geeks out there, I can say my main ingredient is optical flow: I like it because it can work out of the box everywhere (outside, inside, any background is just fine) and for anyone (human or not if he/she has a body, then he/she can play).
The video on the right is a simple window manager demo: every window can be moved, resized, locked or closed (by squeezing it in your hands or pushing it away). While i ‘d still wait some more time before dumping my mouse in the nearest trash bin, this kind of tool can be easily adapted to prepare killer presentations.
The videos below, instead, show a more ludic approach: one is a soccer ball bouncing minigame, the other is a mixed reality version of arkanoid.