spleeter-pytorch

A small implementation of the Spleeter stem separation model in PyTorch. Using this model, audio files can be demixed into vocals, instrumentation etc.

Example

Install the package using pip3 install ., then run

spleeter-pytorch audio-example.mp3

to separate the example file. The output will be located in output/stems.

Conversion to Core ML

The non-FFT parts of the Spleeter model can be converted to Core ML, for efficient inference on macOS/iOS devices. To perform the conversion, run

./convert-to-coreml

The .mlpackage will be located under output/coreml.

Note: The converted model corresponds to the Separator module and still requires the consumer of the model to manually perform the STFT conversion as performed in the Estimator. This is due to Core ML not supporting FFT operations yet.

Note

Currently this is only tested with the 2stems model. Feel free to get one of the other models and test it on them.
There might be some bugs, the quality of output isn't as good as the original. If someone found the reason, please open a pull request. Thanks.

Reference

License

MIT.

PyTorch convert function for op 'index_add_' not implemented.

I tried using convert tools at branch manual-istft-plus-coremltools-7.0, but i got some errors:

Traceback (most recent call last):
File "/Users/x/Downloads/spleeter-pytorch-manual-istft-plus-coremltools-7.0/convert-to-coreml", line 59, in
main()
File "/Users/x/Downloads/spleeter-pytorch-manual-istft-plus-coremltools-7.0/convert-to-coreml", line 43, in main
mlmodel = ct.convert(
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/_converters_entry.py", line 551, in convert
mlmodel = mil_convert(
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 188, in mil_convert
return _mil_convert(model, convert_from, convert_to, ConverterRegistry, MLModel, compute_units, **kwargs)
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 212, in _mil_convert
proto, mil_program = mil_convert_to_proto(
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 286, in mil_convert_to_proto
prog = frontend_converter(model, **kwargs)
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/converter.py", line 108, in call
return load(*args, **kwargs)
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 75, in load
return _perform_torch_convert(converter, debug)
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 122, in _perform_torch_convert
raise e
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 114, in perform_torch_convert
prog = converter.convert()
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 484, in convert
convert_nodes(self.context, self.graph)
File "/Users/x/Library/Python/3.9/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 88, in convert_nodes
raise RuntimeError(
RuntimeError: PyTorch convert function for op 'index_add' not implemented.

fwcd / spleeter-pytorch Goto Github PK