spencerudnick / synth.kitchen

modular synth playground

Synth Kitchen - synth.kitchen

In-browser modular synthesis with Web Audio and Web MIDI.

Plans

Drag-And-Drop Canvas

The GUI should be inviting and feel natural.

Done:

• don't spam requestAnimationFrame
• stop the viewport from growing for no apparent reason
• support undo/redo of patch state changes (modules, connections, name)
• newly-added modules' names are based on the type of module
• adjust the viewport size to something reasonable during drag-continue and drag-end events

TODO:

• newly-added modules are positioned "meaningfully" (try to avoid overlaps, aim for the center of the viewport)
• modify the existing selection properly when using shift+drag
• scroll when dragging a module past the edge of the viewport
• infinite-scroll zoomable module canvas

Keyboard Nav

Done:

• Some way to move selection from module to module using only the keyboard
• modules can be deleted with the keyboard
• Some way to go in-to/out-of a module's controls

TODO:

• A way to navigate structurally, based on existing connections
• A way to navigate independent of structure/tab-order (search?)

Module UIs

Done:

• modules have names
• K-rate audio params have a relevant input
• A-rate audio params have a connector, text-input, and slider
• signal-flow is represented as left-to-right

TODO:

• modules can be renamed
• modules look nice
• sequencer is cool
• modules have colors and can be re-colored

Connections

Done:

• a way to connect/disconnect modules with the mouse
• a way to connect/disconnect modules with the keyboard

TODO:

• undo/redo takes connections into account
• connector inputs have some kind of gain control
• connection ports/cables give some visual feedback of what's passing through them

Saving/Loading

Done:

• export/import files containing JSON-stringified IState.modules

TODO:

• export/import files including connection state
• catch and handle parse errors
• catch and handle a potentially-malformed state
• ability define custom modules based on sub-patches

MIDI

TODO:

• MidiDevice module
• MidiOscillator module
• MidiEnvelope module
• MidiSequencer module
• MidiSequencer module can output to external devices (latency compensation?)

Global Sync

Done:

• some way™️ to sync sequencers, envelopes, etc.

TODO:

• sync with external MIDI clocks
• Ableton Link support

synth.kitchen's Issues

Infinite scroll feels "natural"

Design Chore: Do something about Patch UI/UX

Let's brainstorm (maybe "borrow" some ideas) about how we can improve the look and behavior of Patches. This includes evaluating how we render connections, how we use space, how we display modules, and how we improve the process of adding-to and editing a Patch.

Module insertion improvements

So far, modules have only been able to be added to the end of Racks. The next version of the synth kitchen UI should allow modules to be added anywhere: at the start and end of Racks, and between other Modules. The UI for inserting Modules should be the same in all positions.

When inserting a Module in a position where there is already a Module (start or between), all Modules to the right of that position should shift to the right.

Shift+Arrow Keys extends the selection as one would expect

Module UI - Navigation

Some way to go in-to/out-of a module's controls...

Module control improvements

New components for:

• knobs
• sliders
• radio buttons
• selects
• connectors

Module UI - Colors

Modules have colors and can be re-colored...

Modules have editable titles

Module UI - Connectors

Buttons, just focus on how they look for now.

There should be one button per A-rate param, one button per input, and one button per output.

Development Chore: Clean up debt in existing Modules

Modules as they currently exist are not uniform. Ideal path:

• Limit height of modules
• Work out a divide between UI-concerned Module components and synth-concerned Module classes
  -- Module components are tasked with rendering their current state and converting user input to state-changes and update-calls into the Module classes
  -- Module classes are the "business-logic" classes which define the Web Audio-specific behavior of a module
• It would be nice if Module classes would contain their own definitions of how they should be rendered, as this would enable the Module component to be defined once (for now), simplifying the responsibility of the React side of the code and emphasizing a standardized approach to defining modules
  -- This has the added benefit of illuminating a way for "Nested Modules" to contain their own definitions for how an internal Patch should be rendered

Module UI - Radios

Should have a nice look, with one set of radios per K-rate param.

Module spacing

Modules should be draggable to positions which are not directly adjacent to other modules.

[#] - a module
[ ] - an empty space

Given the rack:

[0] [1] [2]

It should be possible to drag any one of the Modules further to the right, leaving empty spaces between:

[0] [1] [2]
     |      ^
      \    /
        >>

Resulting in the following arrangement:

[0] [ ] [2] [1]

Another example:

[0] [1] [2]
     |         ^
      \       /
        >>>>>

Resulting in the following arrangement:

[0] [ ] [2] [ ] [1]

Module - MIDI Clock

Possibility to run standalone or sync to an external clock

Rack Drag-N-Drop

Add a button to add new Racks, and make the Racks drag-n-droppable.

Patch Improvements

Various aspects of the Patch/Rack/Module/Cable interfaces stand to be improved:

• Racks and Modules will be made drag-and-droppable
• Modules will be made draggable into spaces which are further apart
• Shared Module controls (knobs, radios, etc.) will be improved
• Cables will be rendered as SVGs instead of using an HTMLCanvas

This issue is to track these improvements. There is a branch new-patch which will contain the work done.

Sub-issues will be opened for each improvement above, and each will be accompanied by its own branch.

Development Chore: Do something about patch state

To simplify the eventual work of implementing import/export, centralize a patch's state in redux.

Module - MIDI Out

Patch UI - Connections

• a way to connect/disconnect modules with the mouse
• a way to connect/disconnect modules with the keyboard
• connector inputs have some kind of gain control
• connection ports/cables give some visual feedback of what's passing through them

All data should be included in saved patch files.

Module Drag-N-Drop

Modules should be drag-n-droppable within a Rack.

For the sake of simplicity, Modules are boxes with no controls. Make the NewPatch component show a single Rack, and have a button to add a new Module.

Module - MIDI Sequencer

Module - MIDI In > Pitch

Shift+Drag modifies the selection as one would expect

Module UI - Sliders

Should have a nice look, and be interactive with a mouse and keyboard, with one slider per A-rate param.

Module navigation order is defined by position

When an IO button is clicked, an IO Context menu should appear around that button.

Options:

• Close
• Connect
• Disconnect
• Clear
• Settings

Actions:
Close

• Clicking closes the IO Context
  Connect
• Valid connections are highlighted, while invalid connections are made translucent.
• Clicking a valid connection connects the two nodes.
• Clicking the Connect button again clears the Connect context and the user returns to the IO Context menu.
• Clicking the Close button closes the IO Context.
  Disconnect
• All existing connected nodes are highlighted.
• Clicking a connected node disconnects the two nodes.
• Clicking the Connect button again clears the Disconnect context and the user returns to the IO Context menu.
• Clicking the Close button closes the IO Context.
  Clear
• A prompt appears - "Are you sure?" [ok] [cancel]
• If ok: All existing connected nodes are disconnected.
• Clicking [cancel] clears the Clear context and the user returns to the IO Context menu.
  Settings
• Opens a settings menu of options, such as min, max, slider function (is control linear, logarithmic, etc.).
• Clicking [save] updates the settings of the node.
• Clicking [cancel] clears the Settings context and the user returns to the IO Context menu.

New modules get placed somewhere sensible

Module - MIDI In > Gate