Custom MIDI option ? about midi-model

[SkyTNT]

@asigalov61 OK. Thank you. Maybe I'll try to improve my model to do that later.

[asigalov61]

@SkyTNT Nm, I see you have a custom MIDI option...Sorry I did not noticed it before....

What about inpainting and better models? I can help with that.

Let me know please :)

[SkyTNT]

Nice to collaborate with you. inpainting is a very good idea. But it seems that only the diffusion model can do it. Do you have any ideas?

[asigalov61]

@SkyTNT It is possible to do inpainting with autoregressive models. There are also other things that can be done like melody/chords generation and harmonization.

Check out one of my demos here:

https://colab.research.google.com/github/asigalov61/Lars-Ulrich-Transformer/blob/main/Lars_Ulrich_Transformer.ipynb

Your model uses notes_on/notes_off events which is why it is limited to what it can do. A good idea to switch it score format (think score format of MIDI.py). Then you will be able to do different things with it.

Anyway, you seem to have better programming skills than I do, so if you can tell me what you need help with, I will try to do it and help you however I can. :)

What do you need help with for this project?

Alex.

[SkyTNT]

@asigalov61 You didn't see clearly that my model already uses score format. I'm a bit confused about your code. Can you help me implement inpainting?

[asigalov61]

@SkyTNT Sorry, my bad :) I have not slept yesterday so I did not read your code properly.

Yes, of'course I can help with inpainting. Let me start by making a colab for you and then if you like it we can integrate it to your codebase.

[asigalov61]

@SkyTNT Here is a very rough draft colab for instrument pitches inpainting. Check it out and let me know what you think, please :)

I am also attaching a sample output MIDI.

Let me know.

Alex.
Pitches_Inpainting_MIDI_Composer.zip

[SkyTNT]

@asigalov61 Thank you so much. I understand how to implement inpainting. However I feel this approach is limited by the position of the events in the sequence. It would be even better if you can inpaint a track without restrictions (any time, any number of notes).

[asigalov61]

@SkyTNT You are welcome :)

Yes, this approach is limited in what it can do.

There is a seq2seq approach developed by Stanford: https://github.com/jthickstun/anticipation
This one can do a better job, but it is also somewhat limited and does not always produce good results either.

Let me know what you think.

Alex.

[SkyTNT]

@asigalov61 Cool! This is exactly what I want. Can it be implemented on my model?

[asigalov61]

@SkyTNT AFAIK unfortunately no because they used seq2seq implementation, while your model is autoregressive.

However, your MIDI processor/tokenizer and your Gradio app can be adopted for their architecture AFAIK.

I can't help with their implementation because I mostly deal with autregressive models. But you are welcome to reach out to them for collaboration/help.

Hope this is helpful.

And if you need any more help with autoregressive stuff, feel free to let me know :)

Alex