bastibe / webgl-spectrogram

A smoothly animated spectrogram display in WebGL (FFT in Python/Tornado)

WebGL-Spectrogram

Get running in 5 minutes

First, check out WebGL-Spectrogram and launch it locally:

git clone https://github.com/bastibe/WebGL-Spectrogram.git
cd WebGL-Spectrogram

We have some installation commands that utilize brew, yum, and apt-get based on your system and package manager preference.

All Python packages are install using pip from the requirements.txt file. If you wish to install via another method i.e. conda, all the packages and versions can be found in requirements.txt.

The commands below install common packages for OSX or Linux and run the server.

Note: You need to use sudo if you are not working in a virtualenv.

If you are not using OSX, the apt-get manger is used by default.

To use yum to install packages instead, run make installdeps RPM=1.

make installdeps
make run

A webpage should now open and you can select the file example.wav to generate a sample spectrogram.

About

This is a small local web app that displays a spectrogram of an audio signal in the browser using WebGL. It is known to work with Firefox and Chrome, though performance is best in Firefox.

The spectrogram display can be zoomed and panned smoothly and has a configurable FFT length. The amplitude range can be adjusted on the fly as well. It can open any local .wav or .flac file.

server.py contains a small web server written in Python and Tornado that responds to messages on ws://localhost:XXXX/spectrogram, where XXXX is a random port in the local range. Currently, it supports two kinds of messages: One that requests a spectrogram from a file name, and another that requests a spectrogram from a file content attached to the message. It responds to these messages with a message containing a full spectrogram for the given audio file.

communications.js contains the Javascript implementation of the messaging protocol.

specsize.js contains a helper class for storing the extent of a spectro-temporal display.

spectrogram.js contains Javascript code that can load audio files and request spectrograms from the server, and draw those spectrograms using WebGL. It also contains a time and frequency scale and a small indicator that shows the cursor position in time/frequency coordinates.

main.html contains the website used to display the spectrogram.

The messaging protocol is JSON-based, easily extensible and supports transmission of textual or binary data.

webgl-spectrogram's Issues

Shader documentation

Can a bit of documentation be provided for the shader logic? Specifically I'm interested in the logical behind some of the constants in physicalColor of main.html and how I could modify them if I wanted.

filename = ffi.new('char[]', name.encode())

I've been getting the following error... any help? thank you, thomas

ERROR:tornado.application:Uncaught exception in /spectrogram
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/websocket.py", line 369, in _run_callback
callback(_args, *_kwargs)
File "server.py", line 101, in on_message
self.receive_message(header['type'], header['content'], data)
File "server.py", line 154, in receive_message
self.on_data_spectrogram(data, **content)
File "server.py", line 196, in on_data_spectrogram
file = SoundFile(io.BytesIO(data))
File "/usr/local/lib/python2.7/dist-packages/pysoundfile.py", line 293, in init
filename = ffi.new('char[]', name.encode())
AttributeError: '_io.BytesIO' object has no attribute 'encode'
Exception AttributeError: AttributeError('SoundFile has no attribute _file',) in <bound method SoundFile.del of <pysoundfile.SoundFile object at 0x7f2b5ccdf590>> ignored
WebSocket closed

Allow multiple canvases?

Would you even need/want multiple canvases for audio analysis? I wrapped the spectrogram logic into an object to remove the global use of the gl context to allow for multiple canvases. If you think it would be useful I can submit a PR from https://github.com/joshblum/eeg-spectrogram/tree/multiple-canvases (after removing the EEG logic etc). Let me know what you think :)

Problem when installing dependency pysoundfile.

Hey:
I've met a problem when installing dependency of this program:
File "path-topysoundfile/pysoundfile.py", line 240, in
_snd = _ffi.dlopen('sndfile')
OSError: library not found: 'sndfile'

The version of python is 2.7.3.
Any ideas?