Final Proposal  - 听画
Nianqi, Yunze and me came up(finalized) our idea tonight(Nov, 13rd, 2018). And this project is about conducting musicalization visual. Let people feel the different visual worlds by hearing them, also be able to conduct the music. (medium of the visual worlds will be paintings with different styles) 
There will be paintings, a music stand and a baton on the music stand in front of the user, and the instruction of how to conduct the painting for playing the music will be put on the music stand(music sheet). 
By picking up the baton, users will know they can conducting the painting to "sing". By pointing the elements on the painting, users can hear the music played by that element. By changing their gestures, users will trigger different sound effect of that music which singing by that element. 
How it works
The scenes will look like that:

Tools

Sensors: 
Accelerometer sensor/ Gyro sensor for detecting the movement of the user   ----- it will change the sound effect that song from the painting
IR sensors for detecting if the elements in the paintings are being pointed at with the baton or not.  ------ it will trigger the music that represent the element. also will trigger the light behind the elements of the painting, for giving feedback to the user.
Some hidden button for detecting if the baton on the music table or not   ------ when the baton is off the music table, the light that above the painting will be light up.

Arduino 
Ableton live /or Max/MSP

Back to Top