bearpelican / musicautobot Goto Github PK
View Code? Open in Web Editor NEWUsing deep learning to generate music in MIDI format.
License: MIT License
Using deep learning to generate music in MIDI format.
License: MIT License
I'm curious what the purpose of lines 85 and 86 is in the vocab.py file. Why are you padding the tokens with dummy tokens?
Dear Bearpelican,
What do you mean by "SCRIPT=run_gpt.py bash run_multi.sh" command?
If I use this@midi_generator-master dir,
$ SCRIPT=run_gpt.py bash run_multi.sh --path data/midi/midi_transcribe_v3_shortdur/ --batch_size 8 --lr .0001 --epochs 5 --save gpt/clc/v3ep50
/home/changhyk/anaconda3/envs/midi/bin/python: can't open file 'run_gpt.py': [Errno 2] No such file or directory
Traceback (most recent call last):
File "/home/changhyk/anaconda3/envs/midi/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/changhyk/anaconda3/envs/midi/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/changhyk/anaconda3/envs/midi/lib/python3.7/site-packages/torch/distributed/launch.py", line 235, in
main()
File "/home/changhyk/anaconda3/envs/midi/lib/python3.7/site-packages/torch/distributed/launch.py", line 231, in main
cmd=process.args)
subprocess.CalledProcessError: Command '['/home/changhyk/anaconda3/envs/midi/bin/python', '-u', 'run_gpt.py', '--local_rank=0', '--path', 'data/midi/midi_transcribe_v3_shortdur/', '--batch_size', '8', '--lr', '.0001', '--epochs', '5', '--save', 'gpt/clc/v3ep50']' returned non-zero exit status 2.
Any comment ?
Changhyun
Hi, I'm trying to use this project do something like what Hooktheory demonstrate in this video, and I'm not sure how to proceed. Basically, I want to take a MIDI file of a melody with an accompanying chord progression, and regenerate the pitches of the melody, while leaving the durations and the chord progression as they are--hopefully using the chord progression as a seed (if that's the right concept) for those new pitches.
Thanks a lot for this repo.
when ever i put source activate musicautobot it just says "'source' is not recognized as an internal or external command,
operable program or batch file." so is there anyway to fix this? also how do you even run it even if it is installed correctly?
Hi Andrew, thanks for the great code. In my own code for MIDI generation, at inference time, I tried your repeat penalty formula and it worked great. Do you have any references about this formula? (I mean if someone else came up with it before and published it). Thanks in advance.
Thanks for the repo, this is really neat !
Any chances you release pretrained weights to download ?
Thanks !
Issue #4 mentioned training solely on chord progressions, and I've wondered about doing the same with melodies. To that end I'm linking a project I've just found that uses random forest to isolate melodies, harmonies, and bass parts in MIDI files: https://github.com/ruiguo-bio/midi-miner.
When I was trying to use the code provided in the guide on my own dataset (the Reddit one), I ran into a problem:
music21.midi.MidiException: badly formated midi bytes, got: b'<html>\r\n<head>\r\n<tit'
which is definitely an error in the midi files because the examples provided in the repository works.
Thus, I'm just curious if there is a way to skip these files - to make the code work even if there are less samples available.
Here is my code:
`midi_path = Path('data/midi/')
data_path = Path('data/numpy')
data_save_name = 'musicitem_data_save.pkl'
midi_files = get_files(midi_path, '.mid', recurse=True)
processors = [Midi2ItemProcessor()]
data = MusicDataBunch.from_files(midi_files, data_path, processors=[Midi2ItemProcessor()], bs=16, bptt=512,
encode_position=True, dl_tfms=[batch_position_tfm])
data.save(data_save_name)`
Thank you for your repo, it is brilliant!
I am new to this area and I have got several questions
1 I notitced the CLI scripts both for multitask training and music transformer training are runing run_multitask.py, so did you just discard run_music_transformer.py? If not, when I try to run this file, it pops several errors, are you gonna to fix that?
2 Is about the notebook scripts, I noticed that during the generate procedure, you still need to load the pretrained data pkl file , is that mean we need the pretrain data as well as the pretrained model to help us make prediction?
3. When I run the multitask_transformer/Generate.ipynb and to the line of:
pred_nw = learn.predict_nw(seed, n_words=200)
sometimes it shows it has been interrupted but if I try several times, finally it will work, any reason for that?
4 I try to use my own dataset to train, is there any restriction for the format, like minimum bas numbers? Does it has to be 2 tracks?
5 I try to train with your example midi files, sometime the prediction result will be empty, any suggestion for that?
Truly thankful for your reply!
When trying to preprocess my midi files (using the DataPreProcessing notebook), it keeps on returning None when
try:
# if duet_only and num_piano_tracks(input_path) not in [1, 2]: return None
input_file = compress_midi_file(input_path, min_variation=min_variation, cutoff=cutoff) # remove non note tracks and standardize instruments
in the transform_midi function.
This leads to no conversion to npy's of my midi files
Hi, this line of code seems redundant since bar_pos
is already < self.max_bar_len
by the line above.
Also I am a little bit confused about the default value of beat_len=32
and max_bar_len=1024
. beat_len=4
makes more sense in my opinion since the value will loop every quarter note. Could you please give me more details?
Can someone please implement this project in my local host? I can easily pay you.
Hi,
Is there any chance you can add comment about the vocab token. Tokens like BOS, PAD are easy to understand, but Tokens like CSEQ, MSEQ, NSCLS are quite confusing.
By the way, why you decide to use tokens inside of using pure digital number. In another way to ask this question, why not just using idxenc to feed the model but using the "textify token" to feed the model.
Cheers!
I'm wondering if there's an example of training from .txt files? I have my own representation, from which I'm generating .txt files from MIDI. I just need a bit of guidance on how I'd go about formatting these for the multitask transformer.
Any advice appreciated!
Thanks for the repo, this is very impressive!
Can I use this code at work?
Can I use use the pre-trained model at work?
I would be happy if you could answer.
Thank you.
Hi
I have run into an error trying to replicate the train.ipynb notebook for the music transformer
I have installed the library using the instructions in the repo and tried to run the the notebook. the error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
<ipython-input-13-495233eaf2b4> in <module>
----> 1 learn.fit_one_cycle(4)
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, final_div, wd, callbacks, tot_epochs, start_epoch)
21 callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor, pct_start=pct_start,
22 final_div=final_div, tot_epochs=tot_epochs, start_epoch=start_epoch))
---> 23 learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
24
25 def fit_fc(learn:Learner, tot_epochs:int=1, lr:float=defaults.lr, moms:Tuple[float,float]=(0.95,0.85), start_pct:float=0.72,
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
198 else: self.opt.lr,self.opt.wd = lr,wd
199 callbacks = [cb(self) for cb in self.callback_fns + listify(defaults.extra_callback_fns)] + listify(callbacks)
--> 200 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
201
202 def create_opt(self, lr:Floats, wd:Floats=0.)->None:
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, learn, callbacks, metrics)
99 for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
100 xb, yb = cb_handler.on_batch_begin(xb, yb)
--> 101 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
102 if cb_handler.on_batch_end(loss): break
103
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/fastai/basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
24 if not is_listy(xb): xb = [xb]
25 if not is_listy(yb): yb = [yb]
---> 26 out = model(*xb)
27 out = cb_handler.on_loss_begin(out)
28
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input)
98 def forward(self, input):
99 for module in self:
--> 100 input = module(input)
101 return input
102
~/anaconda3/envs/musicautobot/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
530 result = self._slow_forward(*input, **kwargs)
531 else:
--> 532 result = self.forward(*input, **kwargs)
533 for hook in self._forward_hooks.values():
534 hook_result = hook(self, input, result)
/GW/Health-Corpus/work/nn/musicautobot/musicautobot/music_transformer/model.py in forward(self, x)
33 seq_len = m_len + x_len
34
---> 35 mask = rand_window_mask(x_len, m_len, inp.device, max_size=self.mask_steps, is_eval=not self.training) if self.mask else None
36 if m_len == 0: mask[...,0,0] = 0
37 #[None,:,:None] for einsum implementation of attention
/GW/Health-Corpus/work/nn/musicautobot/musicautobot/utils/attention_mask.py in rand_window_mask(x_len, m_len, device, max_size, p, is_eval)
15 win_size,k = (1,1)
16 else: win_size,k = (np.random.randint(0,max_size)+1,0)
---> 17 return window_mask(x_len, device, m_len, size=(win_size,k))
18
19 def lm_mask(x_len, device):
/GW/Health-Corpus/work/nn/musicautobot/musicautobot/utils/attention_mask.py in window_mask(x_len, device, m_len, size)
6 mem_mask = torch.zeros((x_len,m_len), device=device)
7 tri_mask = torch.triu(torch.ones((x_len//win_size+1,x_len//win_size+1), device=device),diagonal=k)
----> 8 window_mask = tri_mask.repeat_interleave(win_size,dim=0).repeat_interleave(win_size,dim=1)[:x_len,:x_len]
9 if x_len: window_mask[...,0] = 0 # Always allowing first index to see. Otherwise you'll get NaN loss
10 mask = torch.cat((mem_mask, window_mask), dim=1)[None,None]
RuntimeError: CUDA error: device-side assert triggered
Hi, I was wondering if someone might have some insight.
I've followed the "Train" notebook, but when I try to run it, I get the titular error. I am using the .mid files from "/data/midi/examples" as training data or a different set.The "Index, seed #" number changes on different runs.
Additionally, I cannot run the Generate notebook with the example.pth file generated either, it gives the following:
\learner.py", line 18, in music_model_learner
if config is None: config = state['config']
KeyError: 'config'
When I run run_multitask.py in Windows10, I met a path error:
NotImplementedError: cannot instantiate 'PosixPath' on your system
in the following code:
data = load_data(args.path, Path('piano_duet')/args.data_file, bs=args.batch_size, bptt=args.bptt, transpose_range=transpose_range, dl_tfms=mlm_tfm, num_workers=args.num_workers)
while it is fine in Linux.
can you help me to solve this problem? Thanks!
when ever i put "conda env update -f environment.yml" it just says "'conda' is not recognized as an internal or external command,
operable program or batch file." and if i put "source activate musicautobot" it just says "'source' is not recognized as an internal or external command, operable program or batch file." so is there anyways to fix this?
Why while training the accuracy doesn't change?
Even in the example train notebook it stays on 0.397959..
In multitask_transformer, some strange things happened.
In both DataPreprocessing-Reddit ipynb (/music_transformer/) and DataPreprocessing-Reddit-Seq2seq ipynb (/multitask_transformer/),
when I executed 'processed = process_all(...)', it just continued running and never stopped, even clicking interrupt cell could not stop it.
In Train ipynb (/multitask_transformer/), RuntimeError likes Train ipynb in music_transformer when executing learn.fit.one_cycle(4) happened.
I got the same RunTimeError when I tried to execute learn.predict in both "Generate" and "Train" ipynb. All of the cells above the "predict" cells worked just fine. Besides, I got ValueError in "Train-Simple" ipynb. All of the cells above the "learn.fit_one_cycle(4)" cells worked just fine. I wonder is it only me that encountered these problems.
In /notebooks/music_transformer/Generate.ipynb
pred, full = learn.predict(seed_item, n_words=400, temperatures=(1.1,0.4), min_bars=12, top_k=24, top_p=0.7)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding)
In /notebooks/music_transformer/Train.ipynb
pred, full = learn.predict(item, n_words=100)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding)
In /notebooks/music_transformer/Train-Simple.ipynb
learn.fit_one_cycle(4)
ValueError: too many values to unpack (expected 2)
I am trying to start the Flask server. This results in the following error message:
(base) felix@LAPTOP:~/Project/musicautobot/serve$ python run.py
Traceback (most recent call last):
File "/home/felix/Project/musicautobot/serve/run.py", line 2, in <module>
from api import app
File "/home/felix/Project/musicautobot/serve/api/__init__.py", line 38, in <module>
from .predict_multitask import *
File "/home/felix/Project/musicautobot/serve/api/predict_multitask.py", line 16, in <module>
data = load_data(app.config['DATA_PATH'], app.config['DATA_SAVE_NAME'], num_workers=1)
File "/home/felix/anaconda3/lib/python3.9/site-packages/fastai/basic_data.py", line 281, in load_data
ll = torch.load(source, map_location='cpu') if defaults.device == torch.device('cpu') else torch.load(source)
File "/home/felix/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 699, in load
with _open_file_like(f, 'rb') as opened_file:
File "/home/felix/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/felix/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 211, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: '/home/felix/Project/musicautobot/data/numpy/musicitem_data_save.pkl'
Since I didn't find a matching file in the repository, I just created an empty file, which doesn't do much either:
> (base) felix@LAPTOP:~/Project/musicautobot/serve$ python run.py
Traceback (most recent call last):
File "/home/felix/Project/musicautobot/serve/run.py", line 2, in <module>
from api import app
File "/home/felix/Project/musicautobot/serve/api/__init__.py", line 38, in <module>
from .predict_multitask import *
File "/home/felix/Project/musicautobot/serve/api/predict_multitask.py", line 16, in <module>
data = load_data(app.config['DATA_PATH'], app.config['DATA_SAVE_NAME'], num_workers=1)
File "/home/felix/anaconda3/lib/python3.9/site-packages/fastai/basic_data.py", line 281, in load_data
ll = torch.load(source, map_location='cpu') if defaults.device == torch.device('cpu') else torch.load(source)
File "/home/felix/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 713, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/felix/anaconda3/lib/python3.9/site-packages/torch/serialization.py", line 920, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input
I only edited the api.cfg
file as follows, nothing more:
# Input bucket name only. Not the whole s3 URL
S3_BUCKET_NAME = 'MultitaskLarge'
How can I solve the problem? I followed the instructions in the readme file during the installation. Where is the error?
So I want to use the large seq2seq model to create some music, but then I found out that it is extremely memory consuming and I can only input a mere 16 bars into the network.
However, I have this idea that if I can input the next 32 bars into the decoder as the context, it might perform better since we are working with the Transformer-XL. But I've tried many ways to add it in and its either totally not working or at best yielding the exact same behaviour as before. So, is there a simple way to add this?
This is such an interesting work. I hope I could finetune the model to generate particular type of genre and I want to listen to your suggestions. For example, I prepared a small set of jazz songs and attempt to train a little before I infer a new jazz song, where would you recommend to extend for the code?
Appreciate your help
Best,
Hi Andrew, I was running the Midi2Tensor notebook and there was a problem when importing music_transformer. I think the issue came from the append method in transform.py - I haven't tested it, but I think it might be fixed if you add a vocab argument.
Thanks for checking it out!
In #Show Data block, when execute
data.train_dl.on_epoch_begin()
x, y = data.one_batch();
x, y
the output format is like this:
(tensor( [ [ [ 0, 0],
[ 1, 0],
[77, 0], ...)
seems like load position info, which is different from the declared output:
(tensor([[ 0, 1, 76, 138, 64, 140, 59, 140, 57, 140, 8, 138],
[139, 75, 139, 8, 139, 73, 141, 8, 141, 70, 138, 8]]),
and lead to learn.fit_one_cycle(4) to crash:
in music_transformer/model.py
def forward:
...
bs, x_len = x.size() to many value to unpack
how should I fix this bug?
big thx!
Hi, I tried installing this package with
!git clone https://github.com/bearpelican/musicautobot.git
cd musicautobot
!wget https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh && bash Anaconda3-5.2.0-Linux-x86_64.sh -bfp /usr/local
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages')
!conda env update -f environment.yml
!sudo apt-get install musescore
!source activate musicautobot
on colab, and run
%reload_ext autoreload
%autoreload 2
%matplotlib inline
import os
os.chdir('../../')
everything is fine, but importing musicautobot got me an error
from musicautobot.numpy_encode import *
from musicautobot.config import *
from musicautobot.music_transformer import *
from musicautobot.utils.midifile import *
from musicautobot.utils.file_processing import process_all
File "/content/musicautobot/musicautobot/music_transformer/transform.py", line 1
from future import annotations
^
SyntaxError: future feature annotations is not defined
Everything seems to change to 4/4 time regardless of the original time signature, which I suppose is because 4/4 MIDIs are so much more numerous. Would training separately on other time signatures be the fix for this? (I haven't done any training myself.)
I don't have a GPU and can't figure out whether any of the prediction (not training) tasks use a GPU, or only the CPU. Would running it on a GPU speed things up considerably?
I was wondering about the possibility of triplets. Musicautobot has a default beat grid of 16 steps in a 4/4 measure, or 4 pulses per quarter note. In order to encode eighth-note triplets (probably the shortest duration of triplet that would be useful), the beat grid would need to be 96 PPQN, i.e. 24 times as fine a resolution. Is that worth attempting, or would the much larger number of durations require too large a vocabulary?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.