soundio / sequence-json

A way of creating music sequence data in JSON

Sequence JSON

A simple format for representing sequences of timed events in JSON.

This format defines a minimal set of events that get music working, with the objective of supporting applications of the WebAudio and WebMIDI APIs. However, it is designed to be extensible. Consumers of Sequence JSON are expected to silently ignore unsupported event types so that users may also sequence their own data.

Concepts

The Sequence format defines two data structures, a Sequence object and an Event array.

SequenceJSON describes all times and durations in beats. Beat values are arbitrary, and depend on the rate of playback of a sequence. A sequence playing back at a rate of 1 is running at 1 beat per second, so it is following absolute time.

Example JSON

Here are the first two bars of Dolphin Dance represented as a sequence in JSON:

{
    "name": "Dolphin Dance",
    "events": [
        [0,   "meter", 4, 1],
        [0,   "rate", 2, "step"],
        [2,   "note", 76, 0.8, 0.5],
        [2.5, "note", 77, 0.6, 0.5],
        [3,   "note", 79, 1, 0.5],
        [3.5, "note", 74, 1, 3.5],
        [10,  "note", 76, 1, 0.5],
        [0,   "chord", "C", "∆", 4],
        [4,   "chord", "G", "-", 4]
    ]
}

Sequence

{
    "events": [...]
}

events – ARRAY
An array of events.

An events property is the only requirement for a top-level sequence. Sequences may also have the properties:

{
    "id": "0",
    "name": "My Sequence",
    "author": {...},
    "url":  "https://...",
    "sequences": [...]
}

id – STRING
in any array of sequences it must be unique, and is used to identify the sequence for playback. A top level sequence does not require an id.

name – STRING, optional
An arbitrary string.

author – OBJECT, optional
An object containing details of the author.

url – URL STRING, optional
Points to a resource where this sequence can be fetched as JSON.

sequences – ARRAY, optional
An array of sequence objects. Sequences are played back by "sequence" events in the events array. If there are no "sequence" events in the events array, the property sequences is not required.

---

Event

[beat, type, ...]

beat – FLOAT
Describes a point in time from the start of the sequence in beats.

type – STRING
The event type, determines the length of the event array and the structure of the rest of its values:

beat	type	2	3	4	5
beat	"note"	pitch	dynamic	duration
beat	"param"	name	value	curve	duration
beat	"rate"	number
beat	"meter"	duration	divisor
beat	"chord"	root	mode	duration
beat	"key"	name
beat	"sequence"	id	target	duration
beat	"start"	reserved
beat	"stop"	reserved

---

"note"

[beat, "note", name, dynamic, duration]

name – FLOAT [0-127] or STRING
Represents the pitch of a note. If name is a number, it is a MIDI note number, but may be a float and so can represent any frequency. MIDI note number 69 is 440Hz. If name is a string it is an arbitrary pitch name. Implementations must accept at least the 128 pitch names 'C0' - 'G9', and the use of both the hash # and the unicode sharp ♯, and both the small letter b and the unicode flat ♭ in their spellings, but output only the unicode spellings in any Sequence data output.

dynamic – FLOAT [0-1]
Represents the force of the note's attack. A dynamic larger than 1 is permissible, but negative dynamic is invalid.

duration – FLOAT [0-n]
Represents the duration of the note in beats.

---

"param"

[beat, "param", name, value]
[beat, "param", name, value, curve]
[beat, "param", name, value, "target", duration]

name – STRING
The name of the param to control. In a WebAudio context this would typically map to an AudioParam on the instrument being targetted.

value – FLOAT
The destination value of the param.

curve – STRING "step", "linear", "exponential" or "target"
The ramp to use for transition to value. This parameter is optional. If it is not present the event describes a "step" curve. If curve is "target" the event has a fifth parameter:

duration – FLOAT
The decay time of the "target" curve.

---

"rate"

[beat, "rate", rate]
[beat, "rate", rate, curve]
[beat, "rate", rate, "target", duration]

rate – FLOAT
Rate of playback of a sequence. Nominally in beats per second, but sequences may be played from other sequences, and rates are accumulative. If sequence A is playing at rate 2 and contains sequence B playing at rate 1.5, sequence B is playing at an absolute rate of 3, or 3 beats per second, a tempo of 90bpm.

curve – STRING, optional
One of "step", "linear", "exponential" or "target". Represents the type of ramp to use to transition to the new rate. Where curve is "target" a duration is required:

duration – STRING, optional
Where curve is "target", represents the target duration of the target curve.

Where there is no "rate" event at beat 0, consumers should assume a playback rate of 2, as if there were an initial [0, "rate", 2] event in the data. A rate of 2 is a tempo of 120bpm.

---

"meter"

[beat, "meter", duration, division]

duration – INT
The duration of a bar, in beats.

division – INT
The duration of a division, in beats.

Here are some common time signatures as meter events:

time	event
4/4	[0, "meter", 4, 1]
3/4	[0, "meter", 3, 1]
6/8	[0, "meter", 3, 0.5]
2/4	[0, "meter", 2, 1]
7/8	[0, "meter", 3.5, 0.5]

A meter event MUST occur at a beat that is a full bar from a previous meter event. The second event here is valid:

[0, "meter", 4, 1]
[4, "meter", 3, 1]

Where here it is invalid:

[0, "meter", 4, 1]
[2, "meter", 3, 1]

Consumers may choose to recover from invalid "meter" events by pushing them to the start of the following bar, or choose to throw an error.

Meter events have no effect on the rate of the beat clock (although they may have an effect on a metronome or rhythm generator). Where no "meter" event is defined at beat 0 consumers should assume a default meter of 4/4 - ie, [0, "meter", 4, 1].

---

"chord"

[beat, "chord", root, mode, duration]

root – INT [0-11] or STRING, represents the root of a chord. Where root is a number it represents root note as a MIDI number 0-11. Where root is a string it must be a root note name, ie. C, C♯, D ... A, B♭, B.

mode – STRING, represents the mode of a chord. The mode identifier may be arbitrary, but these mode names have fixed meanings:

Symbol	Meaning
"∆♯11"	4th mode of the major scale (lydian)
"∆"	1st mode of the major scale (ionian)
"7"	5th mode of the major scale (myxolydian)
"-7"	2nd mode of the major scale (dorian)
"-♭6"	6th mode of the major scale (aoelian)
"7sus♭9"	3rd mode of the major scale (phrygian)
"ø"	7th mode of the major scale (locrian)
"7♯11"	4th mode of melodic minor
"-∆"	1st mode of melodic minor
"∆♭6"	5th mode of melodic minor
"-♭9"	2nd mode of melodic minor
"ø7"	6th mode of melodic minor
"∆♯5"	3rd mode of melodic minor
"7alt"	7th mode of melodic minor
"°"	Diminished whole tone / half tone
"7♭9"	Diminished half tone / whole tone
"+7"	Whole tone

duration – FLOAT, the duration of the chord in beats.

A chord event provides information about the root and mode of the music. A chord event can be interpreted by a music generator, or used by a notation renderer to display chord symbols.

---

"key"

[beat, "key", name]

name – STRING
The name of a major key, a capital letter followed optionally by an accidental, as in "A♭", "A", "A♯", "B♭", "B", "C", and so on up to "G♯".

Represents a visual key change in written notation only. Does not affect how a performance sounds.

Harmonic generators should use `"chord"` events to generate sound based on modes.

---

"sequence"

[beat, "sequence", sequenceId, targetId, duration]

sequenceId – STRING
The id of a sequence found in the sequences array

targetId – STRING
The id of an instrument to play the sequence through.

duration – FLOAT
The duration in beats to play the sequence.

Renders a sequence from the sequences array. For example, this event plays the sequence "my-sequence" at beat 0.5 for 3 beats:

[0.5, "sequence", "my-sequence", 3]

TBD. It is not clear exactly how to spec targetId to select a target instrument in an interoperable manner. In Soundstage, it refers to the id of a node in the `nodes` array, where nodes are WebAudio nodes in the Soundstage graph.

---

"start"

[beat, "start", identifier, value]

Reserved event name. Basically, both "note" and "sequence" events, which have duration, may be decomposed into matching "start" and "stop" events without duration in internal implementations. Neither should appear in exported SequenceJSON.

---

"stop"

[beat, "stop", identifier, value]

Reserved event name. See "start".

---

Implementations

• Soundstage is a WebAudio wrapper and playback sequencer that consumes and outputs Sequence JSON.
• Scribe is a music notation interpreter and HTML renderer that consumes and outputs Sequence JSON. It also parses ABC notation to Sequence JSON, as well as a sequence text format that is basically Sequence JSON without any of the JSON syntax or quote marks, just spaces between values.

References

• OSC spec: http://opensseqoundcontrol.org/spec-1_0
• OSC example messages: http://opensoundcontrol.org/files/OSC-Demo.pdf
• Music XML: http://www.musicxml.com/for-developers/
• VexFlow: http://www.vexflow.com/

sequence-json's Issues

How to change time signature and key signature in the middle of a song

I'd like to check if it's possible to change time signature and key signature in the middle of a song? How? Thanks.

Nice proposal, but it needs work

First of all it is great to see that there are more people thinking about this. What is clear however is that a few fundamental aspects are missing from your proposal. To me this is because you seem to have based your work on OSC and MIDI which both are rather poor representations of music, because in their essence these protocols are controller streams.

My suggestion would be to read up with the work of Peter Desain and Henkjan Honing, most specifically their work on representing music in "Lisp as a second Language: Functional Aspects", as well as have a very thorough look at how Lilypond (http://lilypond.org) implements this system.
Together with a partner I am also developing a JSON protocol for music, which I will be using to replace the current model (which also doesn't take the work of Desain and Honing into account) in my canvas based notation system Presto (http://mauritslamers.github.io/presto).

What is the purpose of "pitch"?

Maybe this was in the publication from awhile back, but I don't see it documented in the README. I don't know what would be the use of this event--could someone enlighten me?

duration as part of event signature

Hello,

I've been following the progress of this for while. Good work! I wish something like this to get more or less standarized across different libraries. I'm interested in a declarative API to generate music and I've made a first attempt with scorejs and know I'm planning to rewrite a second version and I think it would be nice to follow your specification.

One problem that bothers me is that in scorejs the duration play an important role, the same as time, in order to perform events transformations without knowing the type of the events, and in your specification, the duration is not always present, so this kind of functionallity is harder to implement.

Something like this would made the things much easier:

event = [ time, dur, type, data... ] // dur === null means no duration

... but I'm open to any solution... what do you think?

Transforms

@danigb

So, can I start a discussion about possible transforms? Do you have an idea of the transforms you would like to see? Some obvious transforms that spring to mind ...

'transpose', n

Where n is a positive or negative float that describes the number of semitones to transpose by.

'diatonic-transpose', mode, n

Where mode describes a scale and key, and n the number of steps to transpose by. It is not clear to me what to do with notes that lie between notes in the defined mode in a way that makes musical sense.

'invert', note

Where note defines a pitch that a note or sequence of notes is inverted around.

'diatonic-invert', mode, note

Where mode describes the scale and key and note defines a pitch that a note or sequence of notes is inverted around diatonically. Again, it is not clear to me what to do with notes that lie between notes in the defined mode.

'translate', time

Where time is a positive or negative float that describing a shift in time.

'quantize', ref, strength

Where ref references another sequence of events, and strength defines how much time shift to apply to align the current event or events with the events in the referenced sequence.

'velocity-curve', ...

...

Sequence Event--suggestion for generic "override" instead of rate

A Sequence as an Event is currently specified as follows:

[time, "sequence", data, rate]

Parameter rate seems to belong to interpretation.
In the case that data is a Sequence object, this can go into interpretation.
In the case that data is a reference to a sequence dependency, there may be other parameters about the sequence one wishes to override, in which case, we could change the rate param into an override param or something of sorts.