jhaco / smdatatools Goto Github PK

Current

Main binary only uses --script as a flag and doesn't support passing in the input/output locations when script runs (handle by config.ini file instead).

Desired

Add parsing of --input and --output flags in main binary to use in both script types. Afterwards, the config used can be specified by another flag --config. All flags can be defined and read from the config file instead of the half and half approach currently done.

Current:

• When parsing data from an SM file, the tool bakes in offset time into the rest of the note data. This was needed to work with StepCOVNet, since it doesn't consider offset timings using current methods.
• When parsing an SM file, and rewriting back to a different SM file, it's highly possible that the note colors have changed due to them being placed differently with the offset baked in.

Path Forward:

• Could be tricky, but I could probably add in a flag to not bake in offset time and preserve original data while extracting note timings. Depends on whether this is needed with how current convolution methods only consider timings...

TODO:

• Use librosa.load and sf.write to convert audio to 16000hz, mono-channel wav format that StepCOVNet requires
• Add commandline flags to control input/output dir, whether or not to resample, whether or not to ignore needing to parse data first
• Could use some refactoring to split this whole --copyaudio code block up...it's gotten quite ugly after adding the resampling logic

Doing it this way should resolve a case where a few simfiles that had differing .sm and .ogg/.mp3/.wav names. Both will have the same name when copying audio.

Have been exploring the uses of having all the note data into one csv file. Will write notes here as I mull over whether or not this is needed now...

Currently, I'm writing to individual .txt files that contain the following data:

-- title
-- bpm
-- note_data
--- difficulties
---- note:timing data for each difficulty

so the output looks something like:

-- title
-- bpm
-- note_data
--- easy
---- note + timing data for easy
--- medium
---- note + timing data for medium
--- challenge
---- note + timing data for challenge

and the user will need to manage audio files seperately.

The .csv format I am considering is:

title | bpm | audio_filename | difficulty | note_data | timing_data

As each song has multiple difficulty levels, I want it to be represented something like:

title | bpm | audio_filename | easy | easy_note_data | easy_timing_data
title | bpm | audio_filename | medium | medium_note_data | medium_timing_data
title | bpm | audio_filename | challenge | challenge_note_data | challenge_timing_data

Not sure if csv can handle writing an array of data into a single cell though, that might be tricky. Could have the whole array as one ugly string, but will need to make sure it's being parsed correctly...

Pros:

• In-house python csv module to directly parse/write CSV files to dictionaries
• Easier bookkeeping; no more managing individual .txt files for each song

Cons:

• Not needed atm since current implementation is working fine with StepCOVNet
• Possible file bloat storing note:timing data for a large number of songs
• Will need to redo all the data processing logic since the dict being passed by csv might be different than the one being used by my DataHandler
• Too lazy if I have to do another refactor

If you're running this on a directory where some of the songs don't have static BPMs it will stop the entire batch. I see that there's a script that nukes files that don't have static BPMs but it might be better to just... skip it and keep going.