Our perception of music defies technology. Music triggers our emotions, and evokes the full range of our senses. Take the artist Kandinsky, who associated musical qualities with particular colors and shapes. Meanwhile, a computer processes music as numbers and data which can be incredibly powerful. How can we map this data onto meaningful visuals that enhance our experience of music?
This session is for anyone who would like to explore music, visuals and creative coding for the web. We'll demonstrate types of data we can get from digital signal processing using interactive sketches in p5.js and the p5.sound library that builds upon the Web Audio API. We'll focus on Amplitude, Frequency Spectrum, and maybe some DIY Beat Detection. We'll share ideas and collaborate in the creation of online music visualizations.
Participants may use whatever tools they wish, but the demos in this repo use the following libraries:
p5.js is a JavaScript library that starts with the original goal of Processing, to make coding accessible for artists, designers, educators, and beginners, and reinterprets this for today’s web.
p5.sound.js is an addon library that brings the Processing approach to the Web Audio API.
- Download this github repo, and build off of the empty example sketch in the template folder. It links to the libraries.
- Running p5.js on a local server
- You'll need a text editor. Some options:
A few more complex p5 examples:
p5.AudioIn - microphone! documentation | source code
p5.SoundFile - load and play .mp3 / .ogg files. documentation | source code
loadsound()creates a SoundFile. Use duringpreload(), or with a callback.- To ensure browser compatability, you can provide both mp3 and ogg options.
- Convert to ogg here, and specify multiple filetypes with the soundFormats() method.
.getPeaks()- an array of peak amplitudes over the course of the entire sound file. demo | source
p5.Amplitude - Analyze volume. documentation | source code
.getLevel()returns a Root Mean Square (RMS) amplitude reading, between 0.0 and 1.0, usually peaking at 0.5.smooth()
p5.FFT - Analyze amplitude over time or frequency. documentation | source code
.analyze()returns amplitude readings from 0-255 across the frequency spectrum.waveform()returns amplitude readings from 0-255 across a brief snapshot of time. demo | source
Music included in the demos/repo:
- Yacht - Summer Song (Instrumental) - See Mystery Lights Instrumentals Creative Commons BY-NC-SA
- Broke For Free - As Colorful As Ever - Layers - Creative Commons BY-NC
- For more Creative Commons resources, check out the Free Music Archive's Guide to Online Audio Resources
- Pitch Detection - Web Audio Demo
- Another Approach to Beat Detection Using Web Audio API
- Making Audio Reactive Visuals w/ Web Audio API
- Marius Watz' Sound As Data workshop with Processing // blog post
- Echo Nest Remix API can get you beats, tatums, regular API has more data about music/artists/songs.
- p5.gibber Rapid music sequencing and synthesis. Also its own live coding environment.
- Tone.js is a JS library for composing interactive music.
- dancer.js is a JS library for audio visualizations.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent