Experiment using the Kinect, Processing, the OpenKinect
Libraries, and a video projector during a massage session.
Projecting interactive video directly on the client’s body during massage? Images that follow the therapist’s hands? Motion tracking in low light? It’s all here…
Depth information from the Kinect allows for easy background removal regardless of lighting conditions. Rather than keying out a color (as in green screening), depth information is used to key out distances. In this demonstration, I set the depth threshold just above the height of the client on the massage table. Anything below this threshold, the computer ignores, which makes motion tracking above the threshold much more effective. Rather than tracking the motion of a bright spot or a certain color, the depth-sensing camera allows the computer to track the motion of depth changes. By moving my hands, arms, and body I can create changes in the depth field, which the computer then translates into corresponding forces known as a flow field. The strength and direction of the forces in this flow field are a function of the rate and direction of my motion. The result is a depth-informed flow field map of my movements.
During the initial calibration of the threshold level, the forces in the flow field (caused by the direction and magnitude of my movements) are represented by the direction and length of red vector lines. A particle system is introduced once the threshold level is set to the height of the client on the massage table. Adding a particle system to this flow field is like throwing dust in the wind, the particles flow according to the whims of the local forces. The resulting pattern of particle behavior is a physical manifestation of the flow field’s underlying order. Thanks to the brilliance of Trent Brooks, the underlying forces in this demonstration have the added bonus of noise and randomness, which accounts for the organic fluidity of the particle system.