narcode / codeklavier

Programming with the piano as interface

CodeKlavier

Live coding with the piano as interface.

An open source project by Felipe Ignacio Noriega and Anne Veinberg.

CodeKlavier development started in April 2017 and currently has five modules (newest on top):

• ARquatic (A CodeKlavier Augmented Reality experience!)
• Ckalcuλator (Lambda-calculus with the piano!)
• Hybrid (switching between the first two prototypes and includes a custom code display via udp sockets)
• Motippets (coding via pianistic gestures/motifs)
• hello world (proof of concept prototype, with 1-1 mapping of piano keys to alphanumeric characters)

More information about the specific prototypes can be found in their respective directories.

Latest

CodeKlavier made two videos for this year's HashiDays event. The first is ARquatic and the second uses the Motippets module to code in Mercury. With thanks to Simon Kelaita and Patrick Borgeat for the audio and video editing respectively.

CodeKlavier in action

Checkout our videos page to see the Codeklavier in action or visit the Activities page for info on past and future performances.

Goals

The dream:

🐍🎹 An intuitive programming language with the piano as interface.

The starting-to-be-graspable dream:

👩🏼‍💻 A programming language for the piano as interface

The long term goal:

🎼🎧 a live coding music-domain programming language for the piano as interface

The mid term goal:

💾📌 strategy-specific releases of mini-language approaches for live coding through the piano

The short term goal:

📆📈 research-release-evaluation cycles of prototypes which tackle specific aspects of the system

Equipment

1. An acoustic-MIDI piano such as a Disklavier, any piano fitted with a silent system or an acoustic intrument in combination with a MOOG piano bar or other MIDI keyscanner. We use a MIDI KeyScanner developed by Andrew McPherson and his team at Queen Mary University of London. If an acoustic-MIDI piano is not available, any 88-key MIDI keyboard will suffice providing this instrument is of suitable sensitivity for the pianist.

2. MIDI interface (if not integrated in the piano)

Libraries

Install with pip3

1. python-rtmidi
2. pynput
3. sphinx
4. numpy
5. python-osc (if you want to use the AR module and Caffeine Extension)
6. websockets (idem)

Run pip3 install -r requirements.txt to install the required libraries.

Modules

For this version please add the CodeKlavier directory to your sys.path so the CK modules are recognized

You can run the setPythonPath.sh every time you start a new shell (type . setPythonPath.sh - not the dot-space before the command), or put the following lines in your ~/.bash_profile or ~/.bashrc or equivalent add the following lines (make sure you have the correct path!):

# Modules for the CodeKlavier PYTHONPATH="/path/to/your/codeklavier/project/folder:$PYTHONPATH" export PYTHONPATH

As an example, the path to the Codeklavier in my system looks like this:

PYTHONPATH="/Users/narcodeb/Development/Repos/codeklavier-python/CodeKlavier:$PYTHONPATH"

Extensions

There are a number of different code output extensions for the CodeKlavier. The loading ini file for these can be found in the CodeKlavier-Extensions repository[https://github.com/codeklavier/codeklavier-extensions]. You will need to download/clone this directory in order to run CodeKlavier.

If you are interested in collaborating on an extension for the CodeKlavier, don't hesitate to get in touch.

Currently, outpute extensions exist for SuperCollider, Mercury, JavaScript, Augmented Reality and Hydra.

Test run

After installing the libraries, plug in your midi device and run the miditest by python3 miditest.py. After you complete the setup and configuration, you will see the midi messages on the screen. After that run codeklavier.py -p hybrid -i base.ini (be sure to change the directory too match your setup Try playing an upward C major 3 note arpeggio starting on middle C. If everything works, your test is complete and you are ready to explore CK!

Watch

Codeklavier Videos

Publications

Publications page

Support

This project was made possible by the Creative Industries Fund NL, our angel sponsor and the festivals and venues that book us for paid concerts. However, our initial funding has come to an end so if you believe in the project and would like to support it, please consider "buying us a coffee". Every little bit counts and we greatly appreciate your support!

codeklavier's Issues

CK development summer and further plans

Link to doc on google drive https://docs.google.com/document/d/1R8HLGy5L7mVDTv24spDOyfY-K6r1ocYNvORk_Kb_ucg/edit

CKAR - Ckalc crash

Dear CK Mastermind,

Here's an error that came up during my playing. It came up when i wasn't paying attention to the code and was more jamming, consequently making non-sensical 'equations'. It's difficult for me to reproduce but periodically stuff like this does happen so i think it's worth looking into. Of course we do this in Karlsruhe.

WIth many thanks from the ever grateful CK user and tester.
CKALc crash CKAR.pdf

tremolos over printing

The tremello numbers keep printing even if there are no changes

functions retaining value in stack

Functions are retailing the stack so anne cannot make free numbers.

GO GO Go Felipe!

CKAR enhancement

I would like to see which shape Im in when i change trees. Now i see the new or selected tree, but not its shape.

The next shape function should also have a previous shape function.

THANKS CK MASTERMIND

Weird behaviour

A few weird things that happened:

1. Whilst I was playing my prelude before launching snippet1, I somehow quit the process. It happened twice but I suppose it's possible I quit it accidentally myself in transitioning from terminal to atom.
2. The snippet1 won't restart after the conditional has been evaluated.
3. The tremolo's are over sensative and particular in L can give a value even when I've played it many notes apart.
4. The tremolo's played before the mapping of a minimotif already gives the value for that minimotif.

Display improvements

Titles need to stay

The shift should say "wrong note" and then the shift. This should also be (eventually) replaced - perhaps with next result.

Proposal: reorder structure

At the moment we are using the very ugly:

currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)

To fix the python search path. Perhaps we should move these inside the CodeKlavier folder.

Perhaps best after fixing issuer #40

Dev ?

What happend to the development branch? I cannot submit a pull request. Or did I miss a big change?

hybrid musical adjustments in python

I think the snippet 1 should last much longer after the Gomb! Also, I miss the Huygens.end in this context. Would be cool if snippet 2 had a fade out. These are my current feelings about it. It's super elementary I know but I can't find the file to make these adjustments. Sorry

releases need corresponding SC docs

Dear CodeKlavier master,

Looking at the latest release, I realised there is no link to the corresponding SC directory. I think this is very useful otherwise it is difficult for users to come back to these versions.

With many thanks,

Your trusty mush brains CK user

Release?

Are we ready for a new release and merge into the master branch? What is missing to make this happen?

delta time thresholds in defaultsetup.ini

Dear CodeKlavier guru, will you move the delta time thresholds to the ini file? For quick adjustment

codeklavier.py: use default piece

running ./codeklavier.py -c default.ini just exits. Perhaps we should print a message that the user pick a piece to play, or run interactively.

Motippets- Snippets volume

Can you balance out the volume of the 2 snippets. Snippet 1 appears to be much louder than Snippet 2.

createConfig is used twice as variable in codeklavier.py @0.2.1

change createConfig to createConfigFile here and here

countdown for ALL timers associated to conditionals

Countdown for timer in the conditionals in terminal would be handy

Scrutinize Chordal Recognition function

Sometimes chordal motifs are recognized when not really being played.

Include delta times in the function and analyze the cases where the motif is being recognized but its not.

Update README

The readme from [hello world](codeklavier/hello world/README.md) has some old references. It refers to the 'rtp' version and the 'v4' version. Both files do not exist anymore.

conditionals

Discussion conditional implementation Motippets.

1. Have a note clicker that when pianist has played 2000 notes, there is a bang and killl all to SC (like we did for Bilbao).

Divide range conditional in 2 independent conditionals

Divide conditional 2 in:

conditional 2: range > n

conditional 3: range < n

volume control for snippet 1

I think it would be nice to have volume control for snippet 1. Can we add an extra variable that controls the volume?

CPU 99% when runnin CK motippets

Investigate why and compare to JS version and v0.1 (hello world). I noticed because of the fan noise and i hadn't noticed this before. Might have something to do with latest additions??

Quiting Process with ctrl+c

Clean the quitting process. Now its 2 steps (at least in Presenter version)

display improvement

It would be great if the display could open on the secondary screen rather than the primary and take the appropriate dimensions. In performance, it is much better to keep the primary as the computer and only show the display.

Thanks

Dual display Motippets

The prototype motippets needs to be able to send the code from the LH ✋ to one screen/window and the RH ✋ to another one for the sake of clarity and readability in the code.

any ideas @borrob ?

random crash (list index out of range) when tremolos

crash again although I got abit further... here's the error Press Control-C to exit.
Bye-Bye :(
Bye bye from CodeKlavier setup :(
Traceback (most recent call last):
File "motippets.py", line 154, in
tremoloHi.parse_midi(msg, 'tremoloHi')
File "/Users/Anne/codeklavier-python/CodeKlavier/motippets_classes.py", line 155, in parse_midi
[self._memory[2], self._memory[3]], 'hi',
IndexError: list index out of range
anne:motippets Anne$

ini improvements 2

Hey,
we discussed a different setup of the motifs in motippets so that it is possible to add variables and motifs in one place, rather than changing it in a bunch of places.

This is useful for the addition of new motifs for variables, conditionals, results etc.

Won't boot- keeps crashing upon note input for device

Can't get it to start. When I give note for device ID everything crashes. It says it's line 144

File "motippets.py", line 144, in
if message[2] > 0 and message[0]:
IndexError: list index out of range

Hope it's an easy fix

toggle on ini file

Make it possible to change the toggle key via the default setup ini

Hybrid issues

1. All the hello world text is printing in the middle column of the 3 display. I think it's best when it doesn't print anything

2. When switching back and forth, the conditionals can go whack and I made it crash once but i think this is because i was trying to break it going back and forth. We can probably ignore this for now (see attached)
   

3. At one point Motippets stopped responding...I think it happens when I use the counter conditional- it crashes but not the other threads.
   

system crash

error Terminal Saved Output.pdf
attached the error

Masterscript

Create a masterscript that:

• waits for user input
• user can selection options:
  • quit
  • run any of the CodeKlavier works (at the moment: HelloWorld and Mottipets).

This is probably easiest to realise when #39 is done.

Clean README's in prototypes folders

yeah, make sure they are written good. @wooloogooloo can help???

conditional 3 not working properly

Conditional 3 doesn't handle results 1 and 2 properly. it just prints the "less than an 8ve. Nothing happens" even when the condition is met. This is only true for results 1 and 2 (stop snippets). The gongs work perfectly fine. Conditional 2 also works correctly.

ConfigParser

Every time you start a CodeKlavier session, you have put in the midi-instrument and the settings and stuff. Let's use configparser for that!

Suggested workflow:

• starting CodeKlavier will ask you to give the filename for the configfile.
• leave blank to use the default
• add a 'configFileCreator' to write an appropriate configfile and save it

Logo Proporsal

Hey, I'm graphic designer and my aim is supporting open source projects and make my portfolio better. So, I can contribute on your project with making logo design. I am waiting your feedback! Have a nice day! :)
@narcode

Best Regards,
TJ Ulises
Visual Communication Designer

1. Have a note counter that when pianist has played 2000 notes, there is a bang to close the piece (like we did for Bilbao)

This is not really a conditional. It's more like a self destruct button

printing of variable values

Perhaps when I map a mini -motif, the current value of that variable can be printed in post window?

Note counter in "hello world" mode

The note counter isn't counting when switching to "hello world". I actually think we had already fixed this at one point but can't remember.

fix crash that happened during Huygens festival

2 versions crashed:

• text typer
• cklaculator

Conditional Posting in Terminal

When conditionals post it should look

Cn - Rn - Time to go - Loop time

length huyg stuk s

Perhaps it's wise to put a limiter on the duration these stuks can grow to since I had a run when they went much longer than the designated finishing sample. I dont think however the gomb should kill the other mini- samples- but they should just be sure to not get too long

CKAR crashes (ckalc)

Frequent sever disconnecting leads to CKALC crash.
Terminal output attached
CKALC error.txt

createconfig not working @0.2.1

codeklavier.py doesn't write a configfile
helloworld.py does let you select the port, but strands on the device_id

Repository structure

I propose to move to a clear git-repository structure. The master branch should be the one of the latest release of a stable version. A dev branch is where all the magic happens and we'll use feature branches whenever necessary. NVIE wrote a beautiful blog about a good workflow.

change the original variable values

I THink it would be nice if the snippet2 could go to the earlier values for the variables

Ckalculator display is printing twice

Note: On Ckalc branch. The lambda calculus is fine, it's just a display issue.

parse tremolos only after snippets are playing

The tremolo's played before the mapping of a minimotif already gives the value for that minimotif.

gomb not working (randomly)

Sorry- i cant get the Kaboom to happen. It doesnt register result 4...