who: the performance is a collaboration between Spectre (Stefano Moscardini + me) and Lukas Zpira. I created the software and the visuals, Stefano was responsible for the sound design and Lukas is the main performer on stage.

when: the whole performance was conceived and realized during the summer of 2011, while the three of us were constantly traveling in different parts of the world; fortunately we did share the same vision and come from very compatible cultural backgrounds, so we managed to create something organic working from different places and syncing with a few occasional chats.

where: Danse Neurale was realized for the 2011 edition of Borderline Biennale; after just a few days it was performed again at Souterrain Porte VI and it’s now ready to travel to new places.

what: the central aspect in Danse Neurale is to bring flesh, plastic and steel on stage together, make them complementary and let them express together as if they were a single organism.
Lukas’ body is equipped with a series of wireless sensors: an EEG headset, a breath microphone and a modified stethoscope; moreover two flesh hooks are inserted under his skin and connected to a winch. As the biometric signals are collected, they are sent to the software, where they are parsed in real time and used to drive both the audio and the visuals; when the winch starts to pull Lukas’ flesh, it generates a neurological response that, in turn, becomes a biomechanical feedback: two huge wings made with his own EEG graph grow from his shoulders, start to flap and lift his body off the ground.

where: Milano

when: from 20th september 2011 to 20th november 2011

what: Democratizing Technology is an exhibit sponsored by Ford and aimed at making new car embedded intelligent systems easily understandable, through a series of hands-on experiences.
I was responsible to create two interactive installations that could educate kids (and grown ups) in a playful way.

The first one tries to show in the simplest possible way how a computer can parse vocal commands. Visitors can visualize the sound spectrum of different commands, pass their data buffer to the sound card and hear their sound. Then they are asked to repeat the word they heard and, while their voice gets recorded, they can visualize in real time the “shape” of their voice. Visitors are asked to visually compare the “shapes” of the two different sounds and look for similarities and differences; finally the software explains how specialized techniques can be used to efficiently automate this kind of comparison.

The second one gives an explanation of how traffic sign recognition systems work. Visitors are asked to imagine and draw their own traffic sign on a blank sticker and to place their creation under a camera. The software takes a snapshot of the drawing, applies a set of feature extraction techniques and visualizes the interest points it finds (ie: edges, corners, gradients and their relative distribution inside the image). As Visitors begin to understand how these detected features “describe” the image they have just created, they are asked to give a name to the “traffic sign” they created and to challenge the computer vision software: they are asked to move, rotate, partially hide their sticker, while the software continues to track it and recognize it. Finally, visitors are asked to choose one of the available backgrounds depicting different scenarios (urban, seaside, desert, etc) and to glue their sticker on it: just like in real world scenarios, the software still finds and recognizes the drawing, even between all the other elements in the picture.

Here’s a couple screencasts that show how the two softwares work:

website: http://www.democratizingtechnology.com/

what else: given the name of the exhibit, it was a natural choice to use only open source libraries; if somebody is curious to look under the hood, the traffic sign recognition software is licensed GPL and its source code is here.

for whom: Marcia Lyons

where: David Richard Contemporary, Santa Fe

when: late spring 2011

what: when I was contacted by Marcia, she needed someone that could transform her performance piece into an independent installation. In order to achieve this goal, I created a modular software that (remaining faithful to the author’s sensibility) could control on its own the projections in the exposition space: variations were triggered by a balanced mix of seismic data from online databases, computer vision filtered camera input and physics based particle systems.
Here’s an excerpt from the exhibit catalogue:

RED Force Fields (2011) is a two part interactive installation activated by live data and viewer’s movement. Bubbling up as large pulsating red blots, the projection field is triggered by remote earthly events and bodies in motion that might alter the work – without knowing it – thorough the use of cameras and sensors. Exploring a dual networked affect -’interacting force fields’- streaming seismic sites are wirelessly translated into massive ‘magmatic’ color field projections on adjacent walls of the gallery space. The artwork is activated by live events (feeding in by the hour, day, week) as the viewer moves, the piece is moved, while both the work and the viewer un-earth an empathic with a pulse. This anthropomorphic nature is enhanced by an audible hearthrobing surge synched to the live data and bleeding red trails as viewers ‘bump’ into ’sore spots’. RED Force Fields ultimately allows the visitor to comprehend the environment in a more ephemeral and sensual manner, breaking free of the limitations of ‘logic’ and ‘reality’. Marcia Lyons elaborates, “I’m interested in how these limits disappear and how the performance body takes over.”

when: december 2010

what: ofxMotionTemplates is an OpenFrameworks addon that enables developers to simply create computer vision interfaces controlled by motion; it makes use of MIT’s Media Lab Motion Templates algorithm to find moving objects inside a video stream and keeps track of the motion history of every found object. The data recorded by the library can be used to track movement inside a video, use natural body motion to control software, understand gestures performed in front of a camera, etc…

how it’s licensed: LGPL

where to get it: ofxMotionTemplates.googlecode.com

what: ofxOpenSurf -as the name suggests- is an OpenFrameworks addon wrapping the capabilities of Chris Evan’s OpenSURF. The SURF (achronym of Speeded Up Robust Features) algorithm is able to find significant regions inside an image and to track them in a distorted (i.e. rotated, scaled,…) version of the same image, or in a different image portraying the same kind of object; this means it can be used in scenarios like markerless augmented reality, object recognition, 3D scene reconstruction, etc…
The current version of ofxOpenSurf is thought to make OpenSURF easily accessible from OF and to make it easy to perform common tasks like:

• - finding SURF points in images and video streams:



• - matching objects in different images:



• - transforming any image into an AR marker and tracking it in a video stream:



• - using SURF points to detect motion (useful for 3D scene reconstruction):

how it’s licensed: GPL

where to get it: ofxOpenSurf.googlecode.com

for whom: SolarExpo / GreenBuilding

where: Verona, Italy

when: 5-7 May 2010

with whom: I worked together with two amazing teams:

• bonsaininja, who created the 3D assets
• SSg Creative Factory who dealt with marker printing, totem setup and pr with client

what: SolarExpo is an international conference & exhibition on renewable energy and energy efficiency, held once a year in Verona.
The client wanted to involve visitors in some kind of augmented reality experience, that could entertain them, underline the innovative nature of the solutions presented at the fair, and give useful informations at the same time: in order to accomplish these goals we created an augmented map.
The exhibition catalogues contained a graphic marker that could be read by custom created multimedia kiosks equipped with cameras that were capturing the surrounding area all the time and played back the footage on huge 60″ screens in real time. When someone would show his marker near one of the kiosks, people looking at the screen could see a small cartoonish version of the fair grow out of the catalogue.
This animated model looked like a small planet, powered by the renewable energy solutions presented at SolarExpo: there were wind turbines, sun following solar panels and small electric cars. Furthermore the buildings on the planet were a scaled down version of the exhibition area, so people could actually find their way through the expo playing with this 3D virtual toy.

where to get it: this project is opensource (GPL license) so you can grab your own copy of the software here if you want to try it at home or you can download the source code from here if you want to give a look under the hood. The project was developed on a mac, so, in order to run it, you will need:

• a recent mac, with a recent version of MacOsX
• a decent video card
• a good amount of RAM (>3G)
• a working webcam
• this marker

Last note for programmers: the code is stardard C++ and makes use of multiplatform libraries (Openframeworks, ARToolkit Plus), so you should be able to port it to win or linux with no particular problems, if you feel so inclined.

where does it run: on the iPhone

when: the app was developed between december 2009 and january 2010

what: Squiddy is a minimal, digital pet that swims on your iPhone with a unique, primary need: feeding on Pink Thingies. He will not pay attention to you and your fingers poking inside his little world, because he’s a primitive little predator, not a cute puppy asking for love. This doesn’t mean you can’t have fun with him: if you give him a few Pink Thingies he will become pretty entertaining (even if in a bloody way :) )…

how does it work: it’s very easy: tapping on the screen you drop a Pink Thingie inside Squiddy’s world; as soon as the Pink Thingie appears, Squiddy will swim in its direction, attack it and eat it. Pink Thingies aren’t sentient creatures and can’t swim around on their own, so, if you want to make Squiddy’s hunt more challenging, you can rotate and tilt your iphone in order to let the gravity do its job, or your can drag around your Pink Thingie with your finger.
Give a look at the video to see Squiddy’s bad temperament in all its glory.

where to get it: here or, if you don’t have itunes, just search for “squiddy” on the appstore.

]]> http://naufolio.augmentedrealityag.com/software/squiddy-bloody-mess/feed/ 1