sbambach / folk-rnn

folk music modelling with LSTM

Folk music style modelling using LSTMs

See the following websites:

3. Let's Have Another Gan Ainm (music album)
4. folkrnn.org
5. The Machine Folk Session

This code was used for the following published works:

1. Sturm and Ben-Tal, "Let’s Have Another Gan Ainm: An experimental album of Irish traditional music and computer-generated tunes"

2. Sturm, Ben-Tal, Monaghan, Collins, Herremans, Chew, Hadjeres, Deruty and Pachet, “Machine learning research that matters for music creation: A case study,” J. New Music Research (in press), 2018.

3. Sturm, “What do these 5,599,881 parameters mean? An analysis of a specific LSTM music transcription model, starting with the 70,281 parameters of its softmax layer,” in Proc. Music Metacreation workshop of ICCC, 2018.

4. Sturm and Ben-Tal, "Taking the Models back to Music Practice: Evaluating Generative Transcription Models built using Deep Learning,” J. Creative Music Systems, Vol. 2, No. 1, Sep. 2017.

5. Sturm, Santos, Ben-Tal and Korshunova, "Music transcription modelling and composition using deep learning", in Proc. 1st Conf. Computer Simulation of Musical Creativity, Huddersfield, UK, July 2016.

6. Sturm, Santos and Korshunova, "Folk Music Style Modelling by Recurrent Neural Networks with Long Short Term Memory Units", Late-breaking demo at the 2015 Int. Symposium on Music Information Retrieval

7. The folk-rnn v1, v2 and v3 Session Books https://highnoongmt.wordpress.com/2018/01/05/volumes-1-20-of-folk-rnn-v1-transcriptions/

8. 47,000+ tunes at The Endless folk-rnn Traditional Music Session http://www.eecs.qmul.ac.uk/~sturm/research/RNNIrishTrad/index.html

Music compositions resulting from versions of this code:

1. "Bastard Tunes" by Oded Ben-Tal + folk-rnn (v2) (2017) https://www.youtube.com/playlist?list=PLdTpPwVfxuXpQ03F398HH463SAE0vR2X8

2. "Safe Houses" by Úna Monaghan + folk-rnn (v2) (for concertina and tape, 2017) https://youtu.be/x6LS9MbQj7Y

3. "Interpretations of Computer-Generated Traditional Music" by John Hughes + folk-rnn (v2) (for double bass, 2017) https://youtu.be/GmwYtNgHW4g

4. "Dialogues with folk-rnn" by Luca Tuchet + folk-rnn (v2) (for smart mandolin, 2017) https://youtu.be/pkf3VqPieoo; at NIME 2018 https://youtu.be/VmJdLqejb-E

5. "The Fortootuise Pollo" by Bob L. Sturm + folk-rnn (v1) (2017) https://soundcloud.com/sturmen-1/the-fortootuise-pollo-1

6. "March to the Mainframe" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/TLzBcMvl15M?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/march-to-the-mainframe-by-bob-l-sturm-folk-rnn-v2 Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

7. "Interlude" by Bob L. Sturm + folk-rnn (v2) (2017) Performed by Ensemble x.y: https://youtu.be/NZ08dDdYh3U?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/interlude-by-bob-l-sturm-folk-rnn-v2 (synthesized version: https://soundcloud.com/sturmen-1/interlude-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

8. "The Humours of Time Pigeon" by Bob L. Sturm + folk-rnn (v1) (2017) Performed by Ensemble x.y: https://youtu.be/1xBisQK8-3E?list=PLdTpPwVfxuXrdOyjtwfokrpzfpIlnJc5o Performed by Ensemble Volans: https://soundcloud.com/sturmen-1/the-humours-of-time-pigeon-by-bob-l-sturm-folk-rnn-v1 (synthesized version: https://soundcloud.com/sturmen-1/the-humours-time-pigeon-synthesised) Score is here: https://highnoongmt.files.wordpress.com/2017/12/twoshortpieceswithaninterlude.pdf

9. "Chicken Bits and Bits and Bobs" by Bob L. Sturm + folk-rnn (v1) (2017) https://youtu.be/n-avS-ozrqU Score is here: https://highnoongmt.files.wordpress.com/2017/04/sturm_chicken.pdf

10. "The Ranston Cassock" by Bob L. Sturm + folk-rnn (v1) (2016) https://youtu.be/JZ-47IavYAU (Version for viola and tape: https://highnoongmt.wordpress.com/2017/06/18/the-ranston-cassock-take-2/)

11. Tunes by folk-rnn harmonised by DeepBach (2017)

    1. "The Glas Herry Comment" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/y9xJl-ljOuA
    2. "The Drunken Pint" by folk-rnn (v1) + DeepBach (2017) https://youtu.be/xJyp7vBNVA0
    3. X:633 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/BUIrbZS5eXc
    4. X:7153 by folk-rnn (v2) + DeepBach (2017) https://youtu.be/tdKCzAyynu4

12. Tunes from folk-rnn v1 session volumes

    1. "The Cunning Storm" adapted and performed by Bob L. Sturm and Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/07/the-cunning-storm-a-folk-rnn-v1-original/
    2. "The Irish Show" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/23/folk-rnn-v1-tune-the-irish-show/
    3. "Sean No Cottifall" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/11/11/folk-rnn-v1-tune-sean-no-cottifall/
    4. "Optoly Louden" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/07/01/optoly-louden-a-folk-rnn-original/
    5. "Bonny An Ade Nullway" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/18/bobby-an-ade-nullway-a-folk-rnn-v1-tune/
    6. "The Drunken Pint" adapted and performed by Bob L. Sturm (2017) https://youtu.be/omHhyVD3PD8; performed by EECSers (2017) https://youtu.be/0gosLln8Org
    7. "The Glas Herry Comment" adapted and performed by Bob L. Sturm (2017) https://youtu.be/QZh0WSjFFDs; performed by EECSers (2017) https://youtu.be/NiUAZBLh2t0
    8. "The Mal's Copporim" adapted and performed by Bob L. Sturm (2016) https://youtu.be/YMbWwU2JdLg; performed by EECSers (2017) https://youtu.be/HOPz71Bx714
    9. "The Castle Star" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/12/deep-learning-for-assisting-the-process-of-music-composition-part-2/
    10. "The Doutlace" adapted and performed by Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/08/11/deep-learning-for-assisting-the-process-of-music-composition-part-1/

13. Tunes from the folk-rnn v2 session volumes

    1. Transcriptions 1469, 1470 & 1472 performed by Torbjorn Hultmark (2016) https://youtu.be/4kLxvJ-rXDs
    2. X:488 performed by Bob L. Sturm (2017) https://youtu.be/QWvlnOqlSes; performed by EECSers (2017) https://youtu.be/QWvlnOqlSes
    3. X:4542 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/06/folk-rnn-v2-tune-4542/
    4. X:2857 adapted and performed by Bob L. Sturm and Carla Sturm (2017) https://highnoongmt.wordpress.com/2017/12/02/folk-rnn-v2-tune-2857/

14. Tunes from the folk-rnn v3 session volumes

    1. "The 2714 Polka" adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/10/the-2714-polka-a-folk-rnn-original/
    2. X:1166 Performed by Weltsauerstoff (2018) https://soundcloud.com/weltsauerstoff/1166-conquest-of-brittany
    3. X:1166 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/16/folk-rnn-v3-tune-1166/, https://youtu.be/avxXRNJvUMk (2018)
    4. X:1650 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/17/folk-rnn-v3-tune-1650/
    5. X:6197 adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/09/25/folk-rnn-v3-tune-6197/
    6. X:8589 (A Derp Deep Learning Ditty) adapted and performed by Bob L. Sturm (2017) https://highnoongmt.wordpress.com/2017/10/08/a-derp-deep-learning-ditty/

15. "Why are you and your 6 million parameters so hard to understand?" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/21/making-sense-of-the-folk-rnn-v2-model-part-8/

16. "Two Burner Brew No. 1" by folk-rnn (v2) and Bob L. Sturm and Carla Sturm (2018) https://soundcloud.com/sturmen-1/two-burner-brew-no-1

17. "A Windy Canal" by folk-rnn (v2) and Bob L. Sturm (2018) https://highnoongmt.wordpress.com/2018/01/11/making-sense-of-the-folk-rnn-v2-model-part-6/

18. "Swing Swang Swung" by folk-rnn (v2) and Bob L. Sturm (2018) https://youtu.be/Y_DBRg6SK7E (analysis here: https://highnoongmt.wordpress.com/2018/01/03/making-sense-of-the-folk-rnn-v2-model-part-5/ )

19. "Experimental lobotomy of a deep network with subsequent stimulation (2)" by folk-rnn (v2 with lobotomy) and Bob L. Sturm, Carla Sturm (2018) https://highnoongmt.wordpress.com/2018/01/13/making-sense-of-the-folk-rnn-v2-model-part-7/

20. "The Millennial Whoop Reel" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/29/millennial-whoop-with-derp-learning-a-reel/

21. "The Millennial Whoop Jig" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/08/28/millennial-whoop-with-derp-learning/

22. "Eight Short Outputs ..." by folk-rnn (v1) + Bob L. Sturm (2015) https://highnoongmt.wordpress.com/2015/12/20/eight-short-outputs-now-on-youtube/

23. "Carol of the Cells" by Bob L. Sturm + folk-rnn (v2) (2017) https://highnoongmt.wordpress.com/2017/12/16/carol-of-the-cells-from-the-ai-to-the-orchestra/

24. "It came out from a pretrained net" by Bob L. Sturm + folk-rnn (v2) (2016) https://highnoongmt.wordpress.com/2016/12/24/taking-a-christmas-carol-toward-the-dodecaphonic-by-derp-learning/

25. “We three layers o’ hidd’n units are” by Bob L. Sturm + folk-rnn (v2) (2015) https://highnoongmt.wordpress.com/2015/12/16/tis-the-season-for-some-deep-carols/

26. "The March of Deep Learning" by Bob L. Sturm + folk-rnn (v1) (2015) https://highnoongmt.wordpress.com/2015/08/15/deep-learning-for-assisting-the-process-of-music-composition-part-4/

Media

1. March 31 2017, ‘Machine folk’ music composed by AI shows technology’s creative side, The Conversation (https://theconversation.com/machine-folk-music-composed-by-ai-shows-technologys-creative-side-74708)
2. Sep. 2018, AI created more than 100,000 pieces of music after analyzing Irish and English folk tunes, KTH research news
3. Nov. 2 2018, Så har en artificiell intelligens skapat 100.000 folkmusiklåtar, Dagens Nyheter, https://www.dn.se/kultur-noje/sa-har-en-artificiell-intelligens-skapat-100000-folkmusiklatar/
4. Sep. 2018, Do computers have musical intelligence?, Science Node
5. Sep. 2018, AI skapar 100 000 folkmusiklåtar, Musikindustrin (Sweden)
6. May 2018, "No humans required? If computers can be taught to compose creatively, what does that mean for the future of music?", BBC Music Magazine, Alex Marshal
7. Feb. 28 2018, "Is music about to have its first AI No.1?" https://www.bbc.co.uk/music/articles/0c3dc8f7-4853-4379-b0d5-62175d33d557
8. Dec. 23 2017 "AI Has Been Creating Music and the Results Are...Weird" PC Mag (http://uk.pcmag.com/news/92577/ai-has-been-creating-music-and-the-results-areweird)
9. Nov. 18, 2017 Le Tube avec Stéphane Bern et Laurence Bloch, France https://www.youtube.com/watch?v=LQQER9479Xk
10. June 3 2017, "An A.I. in London is Writing Its Own Music and It Sounds Heavenly" https://www.inverse.com/article/32276-folk-music-ai-folk-rnn-musician-s-best-friend
11. June 8 2017, "Computer program created to write Irish trad tunes" http://www.irishtimes.com/business/technology/computer-program-created-to-write-irish-trad-tunes-1.3112238
12. June 19 2017 "Folk-RNN is the Loquantur Rhythm artist of June" (providing music for phone call waits) https://zc1.campaign-view.com/ua/SharedView?od=11287eca6b3187&cno=11a2b0b20c9c037&cd=12a539b2f47976f3&m=4 (Here is a sample: https://highnoongmt.wordpress.com/2017/06/17/deep-learning-on-hold/)
13. June 18 2017, "Real Musicians Evaluate Music Made by Artificial Intelligence" https://motherboard.vice.com/en_us/article/irish-folk-music-ai
14. June 1 2017, "Can an AI Machine Hold Copyright Protection Over Its Work?" https://artlawjournal.com/ai-machine-copyright/
15. May 26 2017 The Daily Mail named our project "Bot Dylan" (http://www.dailymail.co.uk/sciencetech/article-4544400/Researchers-create-computer-writes-folk-music.html), and then didn't even link to this page. Plus the video they edited has no computer-generated music in it. Well done!
16. April 13 2017 “Eine Maschine meistert traditionelle Folk-Music” http://www.srf.ch/kultur/netzwelt/eine-maschine-meistert-traditionelle-folk-music