About Gesture Synth
Last updated: August 17, 2026
Gesture Synth is a free, browser-based musical instrument that turns your webcam into a chord pad and theremin. This page explains who built it and how it works.
What it is
Gesture Synth uses MediaPipe Hand Landmarker — a Google open-source machine learning model — to read your hand gestures in real time, then maps those gestures to chords, pitch, volume, and tone through the Web Audio API. Everything runs locally in your browser. No install, no account, no upload.
Who built it
Gesture Synth is a re-skinned version of the original open-source tool by Ethan and Eric. The current site adds internationalization (10 locales), modern SEO metadata, a Share dialog for recordings, and a recorder that exports WebM video.
How to play
Open the Play page, allow webcam access, and click "Enable audio". Raise your left hand and extend fingers to pick chord degrees from I-vi-IV-V-ii. Your right hand controls volume (height), chord style (fingers), tone (wrist tilt), and octave (thumb down). Switch to Theremin mode for continuous pitch from hand height. See the Guide page for the full mapping.
Technology
Built with Next.js 16 (App Router), React 19, MediaPipe Tasks Vision, Web Audio API, and MediaRecorder for export. Hosted on Vercel, served via Cloudflare. Source attribution: original open-source tool by Ethan and Eric.
Contact
Questions or feedback? See the Contact page.