top of page

The Theremuino

A MIDI controller to play music by moving hands in thin air. The Theremuino has an Arduino at its heart and can play virtually every sound possible. 

Historical context

The Theremin was invented by Leon Theremin in 1919 and is the first electronic musical instrument invented in human history. Played by the relative position of hands in to control the pitch and volume the theremin never made it to popular instruments of the era because of the sheer difficulty to play it.

The right-hand controls oscillators for pitch near the antenna and the left-hand controls volume over a metal hoop. Theremin presents an excellent example of using spatial interactions to create music.

Leon Theremin playing the original Theremin 

Myself playing the Theremuino prototype

Frame 2608201.png

The Problem

The Theremin costs around 500$ and is extremely difficult to play. The challenge was to simplify the music-creating interactions and create more versatile sounds in under 50$.

The Goal

The project aims to explore spatial interactions and physical computing for the application of music creation.

Focus

Physical Computing | Electronics Tinkering | Interaction Design

Project Type

Capstone Project for ‘An Introduction to programming IoT Devices’ online specialisation’ by University of California Irvine

Timeline

December 2022

Design Process

Group 10832.png

Requirement Specification

System Design

Detail Design

Prototype

Test

Requirement Specifications

Designer’s Perspective: How does The Theremuino work?

The Theremuino project re-envisions the original theremin using IR (Infra red) distance sensors and an ATmega32U4 microcontroller on an Arduino Leonardo Pro Micro.

It is a MIDI controller to create music. The IR sensors map the relative distance of the player’s hands from the sensors to produce MIDI (Musical Instrument Digital Interface) signals. The MIDI signals are read by a DAW (Digital Audio Workstation) to play the sounds of any and every instrument by using a VST (Virtual studio Technology) plugin. The device will map the physical distance pitch and volume level. Moving close to the sensors will increase the pitch and volume and moving away will lower the pitch and volume

The hardware also features programmable buttons to change octaves(-2 to +2), keys (C, C#, D, D#, E, F, F#, G, G#, A, A#, B ) select MIDI channels, scales, tones, play modes and control the DAW like a conventional MIDI controller.

User’s Perspective: How can a user play The Theremuino?

A user can start playing the Theremuino in three easy steps:

  1. Plug the Theremiuno into a laptop/pc running a DAW like Ableton Live, ProTools, and Reaper.

  2. Select the MIDI input setting in the DAW preferences settings.

  3. Select the VST instrument of choice and start playing music by moving your hands in thin air.

It can be played in three modes to cater to instrumentalists with all skill levels from beginner to professional: Scales, chromatic and continuous pitch

  • Easy Mode plays inbuilt diatonic scales almost like an invisible xylophone.

  • Intermediate Mode plays in chromatic scale like an invisible piano

  • Advanced mode plays continuous frequencies like a violin or cello with an invisible neck

System Design

image 3 (1).png

Block Diagram

Screenshot 2023-01-15 at 2.44 1.png

Design Decision Matrix

Option 2 was selected to move ahead for detail design because of the following reasons:


  • Manufacture Cost < 50$. It is the cheapest to manufacture as it requires no battery, inbuilt speaker and Bluetooth module

  • Best playing experience because of no latency and the highest tone flexibility

  • The most hassle-free for users as it can be powered via USB and requires no batteries, hence no recurring cost for users. 

  • Small in Size because the battery and speakers are not included.Device Weight < 1kg.

Detail Design

#1 Choosing the right Arduino board

Arduino Leonardo Pro Micro was chosen because it is the cheapest and smallest. The ATmega32U4 is MIDI class compliant which means it has plug-and-play functionality with Windows and Mac.

Choosing this board eliminates the Hairless MIDI and Loop MIDI software patches. Also Arduino Pro Micro makes the device compatible with the Ventura 13.1 Mac OS and via Loop midi on Windows.

Group 10833 (2).png
Screenshot 2023-01-12 at 1.55 1 (1).png

#2 Choosing the right Input Sensors
 

  • IR sensor was chosen for analog input in the prototype because they produce the best results with hands and provide a linear mapping of notes to distance to notes.

  • Ultrasonic Sensors were rejected because they work better with solid surfaces but pass through human flesh hence giving unstable notes when played with.

Group 10834.png

Prototype 

​Information Flow for Proof of Concept
 

  • ​Button/Potentiometer/Input Sensor: Digital/Analog in
    Arduino Uno: ATMEGA328P microcontroller produces Serial output.

  • Hairless MIDI: Serial to MIDI conversion

  • Loop MIDI: Create a virtual midi port

  • DAW: Read virtual MIDI port and take input from MIDI

  • VST Plugins: Audio plug-in software interface to simulate software instruments and recording hardware in DAW

  • Studio Monitors: Speaker out / Digital out

​Proof of Concept​

image 7.png

MIDI controller POC with Analog and Digital Inputs using Arduino Uno

Screenshot 2023-01-15 at 7.00 1.png

The Thermuino Schematic made on Fritzing

IMG_7904 1.png

The Thermuino Prototype

Test

​Component Testing

​

  1. Test hardware digital inputs in serial monitor: Buttons,

  2. Test hardware analog inputs in serial monitor: Potentiometers and IR Sensors

  3. Test other components: LED, Display, and IR/Ultrasonic Sensor sensors.

  4. Test MIDI functions: MIDI Volume Messages, MIDI Pitch Messages, Scales

  5. Test Pitchbend function of the midiUSB library

  6. Test Code: linear distance to notes/frequency mapping for Am pentatonic scale (5 notes 4 octaves)

Integration Testing

​

1. Test Digital inputs from buttons and IR Sensors in the Serial Monitor of Arduino IDE.
2. Plug the device into a laptop running DAW.
3. Create a virtual MIDI port through the software loopmidi software 
(Only for Windows machines, ignore this step for a mac machine)
4. DAW setup: Select input from the Arduino Leonardo on the DAW.

  • Add a midi instrument and select the correct MIDI Channel

  • Add a VST instrument of choice.

  • Select the correct speaker output on DAW.

5.  Test Scale mode and Pitch bend mode.
6. Test edge cases:

  • What happens if the hand is too close or too far from the sensor?

  • Test Sensor input in case of 2 simultaneous MIDI messages. One for Pitch and other customisable parameters like volume, LowCutoff, Modulation

User Interface Design

Buttons: Scale Select

  • Button 1 : Scale Type +

  • Button 2: Scale Type -

  • Button 1 + Button 2: Toggle Mode: Scale / Pitch Bend
     

Buttons: Octave Select

  • Button 3: Octave +

  • Button 4: Octave -
     

Buttons: MIDI Channel Select

  • Button 5: MIDI Channel +

  • Button 6: MIDI Channel -

  • Button 5 + Button 6: Select Key. Button 1 Key+ and Button 2 Key-
     

Display: Key, Octave, MIDI Channel, Mode: Scale / Pitch Bend

IR sensors: Left Sensor controls Volume, Right Sensor controls pitch

Next steps

Some initial testing with users highlighted a major pain point. Users require some form of feedback to understand the changing notes. It could be addressed by actuator LEDs or by displaying notes on the OLED display.

Currently, I am testing the prototype with more professional and hobbyist musicians to further optimise the interactions and understand the challenges faced by users. The next steps for the project would be

  • Usability testing and iterations

  • Design a casing for the MIDI controller

  • Create a user onboarding document

  • Adding more programmable buttons and knobs

IMG_7897 2 1.png
bottom of page