Midi event transformer for music generation
License: Apache License 2.0
Generative models
Game development
Check out my demos on huggingface skytnt
Hello SkyTNT,
I tried to clone the GIT project and open ti through VSCode, installed required modules but I got an error while running App.py, can you kindly look and advise it? Thanks in advance.
python app.py
Traceback (most recent call last):
File "C:\midi-model\app.py", line 7, in
import torch
File "C:\Users\test\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch_init_.py", line 141, in
raise err
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\test\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\torch\lib\shm.dll" or one of its dependencies.
Hey,
I'm working on a fine-tuning of your model to build a Pokémon diamond music generator.
For now, I'm trying to understand your code and run it. When I try to run the train.py (data is my folder with midi file) :
python .\train.py --data ".\data\"
I encounter this error :
Traceback (most recent call last):
File "D:\Fac\ESIR3\IA\TP\midi-model\train.py", line 336, in <module>
model = TrainMIDIModel(tokenizer, flash=True, lr=opt.lr, weight_decay=opt.weight_decay,
File "D:\Fac\ESIR3\IA\TP\midi-model\train.py", line 101, in __init__
super(TrainMIDIModel, self).__init__(tokenizer=tokenizer, n_layer=n_layer, n_head=n_head, n_embd=n_embd,
File "D:\Fac\ESIR3\IA\TP\midi-model\midi_model.py", line 30, in __init__
self.net = self.net.to_bettertransformer()
File "D:\Fac\ESIR3\IA\TP\midi-model\venv\lib\site-packages\transformers\modeling_utils.py", line 4314, in to_bettertransformer
return BetterTransformer.transform(self)
File "C:\Users\diman\AppData\Local\Programs\Python\Python39\lib\contextlib.py", line 79, in inner
return func(*args, **kwds)
File "D:\Fac\ESIR3\IA\TP\midi-model\venv\lib\site-packages\optimum\bettertransformer\transformation.py", line 211, in transform
raise ValueError(
ValueError: Transformers now supports natively BetterTransformer optimizations (torch.nn.functional.scaled_dot_product_attention) for the model type llama. Please upgrade to transformers>=4.36 and torch>=2.1.1 to use it. Details: https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-and-memory-efficient-attention-through-pytorchs-scaleddotproductattention
I'm using torch==2.1.2+cu121 and transformers==4.36.2
Any ideas ?
你好，感谢你的开源这个项目。
我目的想把扩大”max seq length“到8192，由于我的GPU只有20G的限制，考虑到之前使用过HF的QLora-For-Llama推理大模型的经验。
我想把PL的模型转成HF模型，再使用QLora量化来减少VRAM。
我尝试了一个下午好像把模型转过去了，但是Tokenizer无从下手（没有找到好的参考模式）。
模型的训练方式也要改。（我对HF底层原理不是很熟悉）
参考 https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model
想请教一下有转HF模型的想法吗？
谢谢！
Hello, thanks for open source this project.
My intention was to expand the "max seq length" to 8192, since my GPU is only limited to 20G, given my previous experience using HF's QLora-For-Llama inference large model.
I want to convert the PL model to the HF model and use QLora quantization to reduce VRAM.
I tried for an afternoon and seemed to turn the model around, but Tokenizer couldn't get started (didn't find a good reference pattern).
The training method of the model should also be changed. (I'm not very familiar with the underlying principle of HF)
Refer to https://huggingface.co/docs/transformers/custom_models#writing-a-custom-model
Do you have an idea of switching to the HF model?
Thank you!
I keep encountering this same error every time I try running the training script:
Traceback (most recent call last): File "C:\Users\MainUser\Downloads\midi-generator-app-gpu\train.py", line 316, in <module> train_dataloader = DataLoader( File "C:\Python\lib\site-packages\torch\utils\data\dataloader.py", line 351, in __init__ sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type] File "C:\Python\lib\site-packages\torch\utils\data\sampler.py", line 107, in __init__ raise ValueError("num_samples should be a positive integer " ValueError: num_samples should be a positive integer value, but got num_samples=0
This is the command I used:
.\train.py --data "C:\Users\Administrator\Downloads\midi-generator-app-gpu\dataset\" --resume model_token.onnx --ckpt model_base.onnx --batch-size-train 29 --batch-size-val 29 --accelerator gpu
I don't know exactly what is returning a value of 0 that shouldn't be. The dataset folder I pointed it two has 29 midi files in it (hence the batch size commands), so it shouldn't be that.
Hi! thank you for making this, I've been looking for something exactly like it for a while and I think it will suit me perfectly. I had no problems installing and inferencing with the pretrained model but when i try to train i get a strange error
seed=0, lr=2e-05, weight_decay=0.01, warmup_step=1000.0, max_step=60000, grad_clip=1.0, batch_size_train=2, batch_size_val=2, workers_train=8, workers_val=8, acc_grad=2, accelerator='gpu', devices=-1, fp32=False, disable_benchmark=False, log_step=1, val_step=3200)
Seed set to 0
---load dataset---
train: 41681 val: 2560
Traceback (most recent call last):
File "H:\AI\MIDI\midi-model\train.py", line 335, in <module>
model = TrainMIDIModel(tokenizer, flash=True, lr=opt.lr, weight_decay=opt.weight_decay,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "H:\AI\MIDI\midi-model\train.py", line 101, in __init__
super(TrainMIDIModel, self).__init__(tokenizer=tokenizer, n_layer=n_layer, n_head=n_head, n_embd=n_embd,
File "H:\AI\MIDI\midi-model\midi_model.py", line 26, in __init__
self.net = self.net.to_bettertransformer()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\Miniconda3\envs\midi\Lib\site-packages\transformers\modeling_utils.py", line 4314, in to_bettertransformer
return BetterTransformer.transform(self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\Miniconda3\envs\midi\Lib\contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "C:\ProgramData\Miniconda3\envs\midi\Lib\site-packages\optimum\bettertransformer\transformation.py", line 211, in transform
raise ValueError(
ValueError: Transformers now supports natively BetterTransformer optimizations (torch.nn.functional.scaled_dot_product_attention) for the model type llama. Please upgrade to transformers>=4.36 and torch>=2.1.1 to use it. Details: https://huggingface.co/docs/transformers/perf_infer_gpu_one#flashattention-and-memory-efficient-attention-through-pytorchs-scaleddotproductattention
I'm certain that i'm running these versions of both transformers and torch and unsure why model type is showing as llama?
I couldn't find anyone else encountering this and would be grateful for any assistance. thanks. i'm using conda venv and my pip list is below.
aiofiles 23.2.1
aiohttp 3.9.1
aiosignal 1.3.1
altair 5.2.0
annotated-types 0.6.0
anyio 4.2.0
attrs 23.2.0
Brotli 1.0.9
certifi 2023.11.17
cffi 1.16.0
charset-normalizer 2.0.4
click 8.1.7
colorama 0.4.6
coloredlogs 15.0.1
contourpy 1.2.0
cryptography 41.0.7
cycler 0.12.1
datasets 2.14.4
dill 0.3.7
fastapi 0.108.0
ffmpy 0.3.1
filelock 3.13.1
fonttools 4.47.0
frozenlist 1.4.1
fsspec 2023.12.2
gmpy2 2.1.2
gradio 3.41.2
gradio_client 0.5.0
h11 0.14.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.20.1
humanfriendly 10.0
idna 3.4
importlib-resources 6.1.1
Jinja2 3.1.2
jsonschema 4.20.0
jsonschema-specifications 2023.12.1
kiwisolver 1.4.5
lightning-utilities 0.10.0
MarkupSafe 2.1.1
matplotlib 3.8.2
mkl-fft 1.3.8
mkl-random 1.2.4
mkl-service 2.4.0
mpmath 1.3.0
multidict 6.0.4
multiprocess 0.70.15
networkx 3.1
numpy 1.26.2
optimum 1.16.1
orjson 3.9.10
packaging 23.2
pandas 2.1.4
Pillow 10.0.1
pip 23.3.1
protobuf 4.25.1
pyarrow 14.0.2
pycparser 2.21
pydantic 2.5.3
pydantic_core 2.14.6
pydub 0.25.1
pyFluidSynth 1.3.2
pyOpenSSL 23.2.0
pyparsing 3.1.1
pyreadline3 3.4.1
PySocks 1.7.1
python-dateutil 2.8.2
python-multipart 0.0.6
pytorch-lightning 2.1.3
pytz 2023.3.post1
PyYAML 6.0.1
referencing 0.32.0
regex 2023.12.25
requests 2.31.0
rpds-py 0.16.2
safetensors 0.4.1
semantic-version 2.10.0
sentencepiece 0.1.99
setuptools 68.2.2
six 1.16.0
sniffio 1.3.0
starlette 0.32.0.post1
sympy 1.12
tokenizers 0.15.0
toolz 0.12.0
torch 2.1.2+cu118
torchaudio 2.1.2+cu118
torchmetrics 1.2.1
torchvision 0.16.2+cu118
tqdm 4.66.1
transformers 4.36.2
typing_extensions 4.9.0
tzdata 2023.4
urllib3 1.26.18
uvicorn 0.25.0
websockets 11.0.3
wheel 0.41.2
win-inet-pton 1.1.0
xxhash 3.4.1
yarl 1.9.4
Hey @SkyTNT
First of all, thank you for using my MIDI dataset and thank you for creating Hugging Face app/implementation. It is absolutely great and I love it :)
I was wondering if you want to collaborate and expand this project to add additional features like custom MIDI and inpainting?
I can help you with that if you are interested.
Let me know please. :)
And thank you again for this great project :)
Alex
First of all, thank you for creating the model. It's an exemplary and straightforward implementation.
I'm working on a university project where we want to do a style transfer for music generation. Please help us by providing a notebook with the code necessary for fine-tuning or pointing me in the right direction. I'm new to the field and struggling a bit with navigating your code.
Thank you so much for your help
我更新了代码下来，生成的时候大概率会碰到这个错误。
Scores 会传入nan的情况
CUDA Version: 11.8
torch==2.0.1cu118
transformers==4.33.1
pytorch-lightning==2.0.7
打印sample_top_p_k参数，model.sample_top_p_k(scores, top_p, top_k)
def sample_top_p_k(self, probs, p, k):
# Debugging code
print("probs:", probs)
print("Contains inf:", torch.isinf(probs).any())
print("Contains nan:", torch.isnan(probs).any())
print("Contains negative:", (probs < 0).any())
probs: tensor([[[nan, nan, nan, ..., nan, nan, nan]]], device='cuda:0')
Contains inf: tensor(False, device='cuda:0')
Contains nan: tensor(True, device='cuda:0')
Contains negative: tensor(False, device='cuda:0')
generating: 77%|███████▋ | 1149/1493 [01:55<00:34, 9.94it/s]
Traceback (most recent call last):
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\routes.py", line 427, in run_predict
output = await app.get_blocks().process_api(
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1323, in process_api
result = await self.call_function(
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py", line 1067, in call_function
prediction = await utils.async_iteration(iterator)
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 336, in async_iteration
return await iterator.anext()
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 329, in anext
return await anyio.to_thread.run_sync(
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py", line 312, in run_sync_iterator_async
return next(iterator)
File "D:\Data\Git Repository\midi-model\app.py", line 128, in run
for i, token_seq in enumerate(generator):
File "C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py", line 56, in generator_context
response = gen.send(request)
File "D:\Data\Git Repository\midi-model\app.py", line 62, in generate
sample = model.sample_top_p_k(scores, top_p, top_k)
File "D:\Data\Git Repository\midi-model\midi_model.py", line 72, in sample_top_p_k
next_token = torch.multinomial(probs_sort.reshape(-1, shape[-1]), num_samples=1).reshape(*shape[:-1], 1)
RuntimeError: probability tensor contains either inf, nan or element < 0
Traceback (most recent call last):
File "C:\Users\admin\Downloads\midi-model-main\midi-model-main\train.py", line 320, in
warmup=opt.warmup_step, max_step=opt.max_step)
File "C:\Users\admin\Downloads\midi-model-main\midi-model-main\train.py", line 87, in init
n_inner=n_inner, flash=flash)
File "C:\Users\admin\Downloads\midi-model-main\midi-model-main\midi_model.py", line 26, in init
self.net = self.net.to_bettertransformer()
File "C:\Users\admin\python\lib\site-packages\transformers\modeling_utils.py", line 3422, in to_bettertransformer
from optimum.bettertransformer import BetterTransformer
File "C:\Users\admin\python\lib\site-packages\optimum\bettertransformer_init_.py", line 14, in
from .models import BetterTransformerManager
File "C:\Users\admin\python\lib\site-packages\optimum\bettertransformer\models_init_.py", line 16, in
from .attention import _llama_prepare_decoder_attention_mask
File "C:\Users\admin\python\lib\site-packages\optimum\bettertransformer\models\attention.py", line 20, in
from transformers.models.llama.modeling_llama import apply_rotary_pos_emb, repeat_kv
ImportError: cannot import name 'repeat_kv' from 'transformers.models.llama.modeling_llama' (C:\Users\admin\python\lib\site-packages\transformers\models\llama\modeling_llama.py)
Hey @SkyTNT
I just wanted to thank you again for your great work/code! :)
Please check out my project which is based on your code/implementation and on one of my models:
https://huggingface.co/spaces/asigalov61/Allegro-Music-Transformer
This project was only possible thanks to you and your code :)
Thank you and I would love to hear any thoughts or feedback from you :)
Sincerely,
Alex.
I put my midi files into the samples folder, but it still turned out wrong
Error
Expecting value: line 1 column 1 (char 0)
I just used "generate" and it turned out wrong.
Hi! Thanks for sharing your code! I'm really enjoying playing with you Huggingface Space.
When I went to try the Colab notebook, the line
!python app.py --max-gen 4096 --share
```
would fail with this error mesage:
```python
Traceback (most recent call last):
File "/content/midi-composer/midi-composer/app.py", line 191, in <module>
class JSMsgReceiver(gr.HTML):
File "/usr/local/lib/python3.10/dist-packages/gradio/component_meta.py", line 191, in __new__
create_or_modify_pyi(component_class, name, events)
File "/usr/local/lib/python3.10/dist-packages/gradio/component_meta.py", line 110, in create_or_modify_pyi
current_interface, _ = extract_class_source_code(pyi_file.read_text(), class_name)
File "/usr/local/lib/python3.10/dist-packages/gradio/component_meta.py", line 72, in extract_class_source_code
class_ast = ast.parse(code)
File "/usr/lib/python3.10/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 192
from gradio.events import Dependency
^
IndentationError: expected an indented block after class definition on line 191
```
I know that `gradio` has seen a lot of updates recently, perhaps something broke relative to your code. What version of `gradio` have you been using to build this?
Thanks!
关于编码问题请教一下，能不能把现有的midi多轨道同时编码(多个轨道同时编码)，变成一轨完成再接着另一轨，一轨只有一个乐器，
比如:
编完完整的钢琴轨后，在钢琴轨后面继续编吉他或者其他乐器，再重复这样。
（解码时再同步各个乐器），
我觉得这样做的好处是容易单独调整单声部。
或许容易根据钢琴续写🎸或者鼓组或者其它。
或者根据旋律续写和声。
我感觉拓展性强，估计实现比较复杂。（我还没时间仔细研究Tokenizer）
不知道这样实现是否可行？
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.