Comments (22)
Well, we have the plan to support second. As you can see, compared to other liraries, we defined ttype
(time unit type) for supporting tick
, quarter
and second
But to support second, I need to dive into prettymidi's parsing algorithm, because I‘m not quite familiar with such conversion.
And now, our highest-priority goal is to merge the librarify branch. In this branch:
- split the
Score.hpp
into multiple .cpp and .h files, making it more readable and easier to be maintained and developed - optimize the performance significantly (most changes is small, so I copy them manually to the main branch yesterday)
- switch from
pybind11
tonanobind
. This could largely reduce the overhead of accessing small attributes, likeNote.pitch
- Add bindings for pinao roll and note array. Pianoroll's interface is still under desining #16 . And note array is a
struct of array
while track.notes is aarray of struct
, which means, note_arr.pitch would be a numpy array. This is great for preprocessing in python utilizing numpy.
As you can see, we just get lots of things to do. I will work on the support for second
after all the above things.
from symusic.
@Yikai-Liao no pressure, focus on the things you are already working on first.
Just to add on the performance, the update from v0.2.3 to 0.2.5 made the read operation on my test set of 10,000 MIDIs 20% faster!
from symusic.
@Yikai-Liao no pressure, focus on the things you are already working on first.
Just to add on the performance, the update from v0.2.3 to 0.2.5 made the read operation on my test set of 10,000 MIDIs 20% faster!
What kind of test are you doing? Maybe midi parsing is now not the bottle neck for your test, becuase in our own test, midi parsing part got a nearly 3 times speed up (the time cost here includes reading file from disk).
from symusic.
Just reading 10,000 different piano-only single-track MIDIs in a for loop:
for midi_path in midi_paths:
score = Score(midi_path)
Parsing times:
- v0.2.3: 13.78 s
- v0.2.4: 14.90 s
- v0.2.5: 10.52 s
Also tested on mahler.mid using timeit
:
- v0.2.3: 13.2 ms ± 102 µs
- v0.2.4: 14 ms ± 81.3 µs
- v0.2.5: 4.15 ms ± 38.3 µs
Which is indeed 3 times faster. The platform is Linux.
from symusic.
I get the case.
I presume that most of your test examples are small files (we basically haven't found any files larger than mahler.mid, which is an order of magnitude larger than the average midi). The performance bottleneck should be in disk reads for small files (this is actually pretty close to 4k random reads and writes)
One possible way to further speed up this kind of situation is to use pickle
to create caches. We use zpp_bits as pickle's backend. There are generaly 2 kinds choice.
- Loading data into memory at same time: dump all scores in a python list or dist using pickle, and just load the whole things. Sequential reads and writes of large files are much faster, and parsing the binary files created by zpp_bits is also much faster.
- Loading data seperately: you could use some kv database (like leveldb) or other kinds of no-sql database to store the bytes you get from pickle.dumps. This is also faster to access lots of small files from disk.
Note that the file size would be a bit larger than midi, here is size the list of 1000 mahler (641KB per midi file)
Also, it occurs to me that I didn't add a constructor in python that could accept python bytes which exists in our c++ code. So it would be also possible to let the database direclty manage your raw midi data.
But in my opinion, we just don't get that large midi dataset. lmd_full.tar.gz is only 1.65GB, which means it would be easy for a server to load all data in memory.
BTW, the support for pickle also makes it possible to let symusic
support python's multiprocessing. All the objects defined in symusic is picklable, which means they can be exchanged among threads and processes. We also make sure that symusic
is a thread-safe
library.
from symusic.
@ilya16 I have started to implement second conversion. Although I met some bugs, I think things would be done soon.
from symusic.
@ilya16 I have fixed the bugs I met. Now loading midi with second as time unit is support in the librarify branch. (Convert them back to tick is not implemented now).
You could try it by the following steps:
git clone --recursive https://github.com/Yikai-Liao/symusic --branch librarify
pip install ./symusic
make sure you have a cmake env.
from symusic.
All the 3 kinds time unit is not fully supported in librarify branch! 🎉
from symusic.
@Yikai-Liao thanks! trying it out right now
from symusic.
@Yikai-Liao found an inconsistency with times returned by miditoolkit
's midi.get_tick_to_time_mapping()
Can't catch the full case, but it seems that for some notes the start time in seconds is calculated using the tempo that comes after the note's start tick.
Here are the plotted differences between the note seconds from miditoolkit
and symusic
on one MIDI:
The tempo seconds are all the same in both cases.
from symusic.
Also, here's a numpy
-based time mapper I wrote to convert between ticks and seconds, adapted to symusic
's Score
. It returns the same times as miditoolkit
's midi.get_tick_to_time_mapping()
.
import numpy as np
from typing import Union
from symusic import Score
class ScoreTimeMapper:
def __init__(self, score: Score):
self.score = score
self.tempos = self.compute_tempo_times(score)
@staticmethod
def compute_tempo_times(score: Score) -> np.ndarray:
prev_tempo_tick, prev_tempo_time = 0, 0
scale_factor = 60 / float(score.ticks_per_quarter)
seconds_per_tick = scale_factor / 120.
tempo_data = []
for tempo in score.tempos:
tempo_time = prev_tempo_time + seconds_per_tick * (tempo.time - prev_tempo_tick)
seconds_per_tick = scale_factor / tempo.qpm
tempo_data.append([tempo.qpm, tempo.time, tempo_time, seconds_per_tick])
prev_tempo_tick, prev_tempo_time = tempo.time, tempo_time
tempo_data = np.stack(tempo_data, axis=1)
return tempo_data
def t2s(self, ticks: Union[int, np.ndarray]) -> Union[float, np.ndarray]:
ids = np.searchsorted(self.tempos[1], ticks, side="right") - 1
tempo_ticks, tempo_times, seconds_per_tick = self.tempos[1:4, ids]
return tempo_times + seconds_per_tick * (ticks - tempo_ticks)
def s2t(self, seconds: Union[float, np.ndarray]) -> Union[int, np.ndarray]:
ids = np.searchsorted(self.tempos[2], seconds, side="right") - 1
tempo_ticks, tempo_times, seconds_per_tick = self.tempos[1:4, ids]
return (tempo_ticks + (seconds - tempo_times) / seconds_per_tick).astype(int)
score = Score(...)
stm = ScoreTimeMapper(score)
score_note_times = stm.t2s([n.time for n in score.tracks[0].notes])
from symusic.
Thanks, I will test it and try to find the bug.
After fixing it, we will get the 0.3.0 version
from symusic.
Ok, things are fixed. The bug is that I didn't take into account the case where the two tempo's are close to each other.
Changing an if condition in the code to a while loop results in the correct answer.
There's also some precision loss from floating point, but I think that's acceptable
[5.60742188e-06 1.78144531e-05 1.78144531e-05 5.60742188e-06
1.17109375e-05 1.70078125e-05 7.97070311e-06 2.17343750e-05
1.26972656e-05 1.06718750e-05 5.45507811e-06 4.69843744e-05
8.03828120e-05 2.49140619e-05 9.15156245e-05 5.98359370e-05
9.62421871e-05 2.09296870e-05 6.76874995e-05 3.04687433e-06
4.68906244e-05 1.33437493e-05]
from symusic.
#19 v0.3.0 is now released, and we add full support for second in this version.
If the bug for second conversion is fixed, I'll close this issue.
from symusic.
The times are now correct for the same tested MIDI, but yes, some error is accumulated:
from symusic.
Is it only occured on midi files with dynamic tempos? I think there should be nothing wrong to be accumulated in my code except for the time (in sescond, float32) of tempo events.
from symusic.
Yes, it is a MIDI file with tempo changes at beat boundaries. For a single tempo, the differences are of the order of 1e-5
from symusic.
Here is the core of the conversion code. I guess the bug could possibly be in 2 places:
- pivot_time is float 32, which could cause a precission loss (but this would be a random value, possitive or negtive)
- the calculate of order of
cur_factor
cause a precission loss
template<template<class> class T>
[[nodiscard]] vec<T<Second>> time_vec(const vec<T<Tick>> & data) const {
vec<T<Tick>> origin(data);
ops::sort_by_time(origin); // sort it
vec<T<Second>> ans;
ans.reserve(origin.size());
auto t_iter = tempos.begin() + 1;
i32 pivot_tick = 0;
f32 pivot_time = 0;
double cur_factor = static_cast<double>(tempos[0].mspq) / 1000000. / static_cast<double>(tpq);
for(const auto & event : origin) {
while(event.time > t_iter->time) {
pivot_time += static_cast<float>(cur_factor * (t_iter->time - pivot_tick));
pivot_tick = t_iter->time;
cur_factor = static_cast<double>(t_iter->mspq) / 1000000. / static_cast<double>(tpq);
++t_iter;
}
ans.emplace_back(
pivot_time + static_cast<float>(cur_factor * (event.time - pivot_tick)),
event
);
}
return ans;
}
from symusic.
Maybe it's the casting from double to float for `pivot_time' and its further accumulation. Anyway, the error is very small (less than ms) to be considered an overall conversion error.
from symusic.
I further checked the consistency of the conversion results with pretty midi, but I found them to be quite different.
The maximum error between symusic and ScoreTimeMapper's results is on the order of 1e-5 in all test examples.
But in the comparison with pretty midi I could observe even a difference of several seconds.
I don't know whose results are the GROUND TRUTH!
I use the midi files in symusic's repository.
BTW, don't use symusic with python 3.12. There are some strange bugs currently.
And here is the code:
import os
import pretty_midi as pm
import symusic as sm
import numpy as np
from symusic import Score
from typing import Union
class ScoreTimeMapper:
def __init__(self, score: Score):
self.score = score
self.tempos = self.compute_tempo_times(score)
@staticmethod
def compute_tempo_times(score: Score) -> np.ndarray:
prev_tempo_tick, prev_tempo_time = 0, 0
scale_factor = 60 / float(score.ticks_per_quarter)
seconds_per_tick = scale_factor / 120.
tempo_data = []
for tempo in score.tempos:
tempo_time = prev_tempo_time + seconds_per_tick * (tempo.time - prev_tempo_tick)
seconds_per_tick = scale_factor / tempo.qpm
tempo_data.append([tempo.qpm, tempo.time, tempo_time, seconds_per_tick])
prev_tempo_tick, prev_tempo_time = tempo.time, tempo_time
tempo_data = np.stack(tempo_data, axis=1)
return tempo_data
def t2s(self, ticks):
ids = np.searchsorted(self.tempos[1], ticks, side="right") - 1
tempo_ticks, tempo_times, seconds_per_tick = self.tempos[1:4, ids]
return tempo_times + seconds_per_tick * (ticks - tempo_ticks)
def s2t(self, seconds: Union[float, np.ndarray]) -> Union[int, np.ndarray]:
ids = np.searchsorted(self.tempos[2], seconds, side="right") - 1
tempo_ticks, tempo_times, seconds_per_tick = self.tempos[1:4, ids]
return (tempo_ticks + (seconds - tempo_times) / seconds_per_tick).astype(int)
def check(p: str):
score_t = sm.Score(p)
score_t.tracks.sort(key=lambda x: (x.program, x.name, x.note_num()), inplace=True)
score_s = score_t.to("second")
midi = pm.PrettyMIDI(p)
midi.instruments.sort(key=lambda x: (x.program, x.name, len(x.notes)))
assert len(score_t.tracks) == len(midi.instruments)
delta = []
m_delta = []
mapper = ScoreTimeMapper(score_t)
for s_track, t_track in zip(score_s.tracks, score_t.tracks):
m_delta.append(s_track.notes.numpy()['time'] - mapper.t2s(t_track.notes.numpy()["time"]))
m_ans = np.concatenate(m_delta)
for i, (s_track, p_track) in enumerate(zip(score_s.tracks, midi.instruments)):
assert s_track.note_num() == len(p_track.notes), f"{p} track {i} note equal"
for j, (s_note, p_note) in enumerate(zip(s_track.notes, p_track.notes)):
delta.append(s_note.start - p_note.start)
delta = np.array(delta)
print(p)
print(f"Tempos Num: {len(score_t.tempos)}")
print("Pretty Midi Delta", delta[delta > 1e-5])
print("TimeMapper", m_ans[m_ans > 1e-5])
if __name__ == "__main__":
root = "C:/Users/nhy/code/symusic-libra/tests/testcases/One_track_MIDIs"
for name in os.listdir(root):
if name.endswith(".mid"):
check(os.path.join(root, name))
from symusic.
Oh sorry, it's my fault. I forgot to sort the notes. After sorting them, pretty midi could get the same answer as ScoreTimeMapper
from symusic.
Changing f32 to f64 works well. If I use f64, there will be no difference with ScoreTimeMapper.
But I don't think it's quite worth it. This change would make the size of a Note grow from 12 bytes to 24 bytes。
- using f32: (time: 4, duration: 4, pitch: 1, velocity: 1, memory alignment: 2)
- using f64: (time: 8, duration: 8, pitch: 1, velocity: 1, memory alignment: 6)
from symusic.
Related Issues (20)
- [Question] Performance improvement compared to mido HOT 4
- Incorrect/inconsistent treatment of program change in dump_midi HOT 9
- Bug when multithreading with macOS since 0.3.4 HOT 14
- Unable to build on linux HOT 10
- Adding a function to easily modify the event times at a score level (like `midi.adjust_times` in PrettyMIDI) HOT 35
- Question and Request: Sorting after conversion to second HOT 5
- Inconsistency between the interface of vector bindings and python list & potential wild pointer problems. HOT 55
- merging tracks HOT 3
- Segmentation Fault when reading Score from MIDI files HOT 3
- Make `Score.clip` include the ongoing tempo/time signature/key signature at `start` HOT 7
- Inconsistency with Score when writing to Audio HOT 11
- MIDI with negative beginning tick HOT 5
- MIDI files with simultaneous note on/off ticks result in incorrect note durations HOT 3
- nanobind v2.0.0 breaks building compatibility HOT 3
- nanobind problem HOT 7
- Suppression of Nanobind leak messages HOT 6
- Set-Tempo events not applied correctly to timestamps in multi-track files HOT 1
- MIDI message level interface design HOT 3
- Note durations in a midi file increase after loading and dumping via symusic.Score HOT 3
- Nanobind Issue HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from symusic.