event
Synthux Hackathon 2024

Synesthetic Dinner

Experience dining on a whole new sensorial level
TouchDesigner
Camera feed
prompt

Coaches & collaborators

Roey Tsemah
institute
Design Academy Eindhoven
Synesthetic Dinner is an interactive installation which allows people to experience eating food on a whole different level with all of their senses involved. When someone is hovering over a plate with a chosen ingredient, a specific audio and an animation following their hand movement will shortly show up. If 4 of the chosen ingredients are put on the mixing plate and then eaten, a combined full experience will play. Even if you don't have synesthesia, this project will allow you to feel what it's like.
No items found.
the tech

How does it work?

The base animations were created in After Effects and the sounds were made in Ableton with the help of the SynthGPT plugin. Then, everything was combined and mapped in Touch Designer, using optical flow and blob tracking TOPs. By using only these two operators, we were able to accurately track and map the graphics using only a simple camera input. The sounds and the visuals get triggered with a 1.5-4 second sustain every time there is movement detected over a specific plate. What is more, by reading the X and Y coordinates of the moving pixels over the plate, we were also able to make the animations on the screen follow the hand gestures of the user.
Open source code
Click here to grab this project's code
TouchDesigner
Node based visual programming language for real time interactive multimedia content
Camera feed
About the instrument
we're using cookies to run our site. privacy policy.