This is a video of a real conductor using virtual telekinetic forces (via an Xbox Kinect) in order to control the way a piece of music is played through a synthesizer:
The conductor controls multiple orbs in 2D space using hand movements to attract the different orbs; thus using a virtual telekinetic force the user can control these orbs. The position of these orbs then gets converted into midi signals that can be used to control a synthesizer. In the video attached a midi track is being played into NI’s Massive synth and all 8 dials on the synthesizer are being controlled at once, using different controllers it is quite possible to control up to 20 parameters at once.
Being able to control hundreds of parameters at once is all well and good; however this device adds something extra. When someone uses this device they automatically find themselves dancing or conducting (dependent on the music type) in order to control the music properly, in fact if you don’t dance you don’t get the right timing.
The functionality is not limited to one machine either; each component connects to a server via TCP/IP so collaboration / jamming can take place over the internet.
This system is not at product stage yet; but I am especially interested in getting this system in front of charities / social enterprises that using music production to help handicapped or disaffected people. Would you be able to help realize this by putting an article out somewhere? People can get in contact via the website:
Thanks Alistair de Blacquiere Clarkson for sharing