adefossez / julius Goto Github PK

Hey @adefossez I just stumbled across this courtesy of @mpariente's GitHub activity feed. I added it to this comparison of resample implementations: https://gist.github.com/jonashaag/2cde4daa72cebfb20641373d8dceeaae

Just FYI :)

Hi
I’m getting a RuntimeError: cuFFT error: CUFFT_EXEC_FAILED, when I try to use the bandpass_filter with fft=True (a single GPU)
The last function called is new_fft.irfft
Any idea what could be the root cause?
Thanks

Small issue in SplitBands.

Steps to reproduce:

Run the following in a Jupyter notebook:

import julius

split_bands = julius.SplitBands(1, 16)
print(split_bands)

split_bands = julius.SplitBands(1, cutoffs=[0.1, 0.25])
print(split_bands)

split_bands = julius.SplitBands(1, cutoffs=np.array([0.1, 0.25]))
print(split_bands)

This has the following output:

SplitBands(sample_rate=1,n_bands=16)
SplitBands(sample_rate=1,cutoffs=[0.1, 0.25])
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-93-5b31698bfbfc> in <module>
      8 
      9 split_bands = julius.SplitBands(1, cutoffs=np.array([0.1, 0.25]))
---> 10 print(split_bands)

/home/admin/miniconda3/envs/venv/lib/python3.8/site-packages/julius/bands.py in __repr__(self)
    100 
    101     def __repr__(self):
--> 102         return simple_repr(self, overrides={"cutoffs": self._cutoffs})
    103 
    104 

/home/admin/miniconda3/envs/venv/lib/python3.8/site-packages/julius/utils.py in simple_repr(obj, attrs, overrides)
     30         if attr in params:
     31             param = params[attr]
---> 32             if param.default is inspect._empty or value != param.default:
     33                 display = True
     34         else:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Probably just need a tolist if it's a numpy array in simple_repr.

Anyway, great project! I've enjoyed using it.

Hi @adefossez ,
First of all, thanks for your wonderful implementation! I have started to use it a lot recently, it really saves my time.
Also, your ResampleFrac module support torch.jit which is another great thing to admire.

Although the speed of resampling is excellent, I have noticed that the quality of julius isn't as good as other famous packages (like resampy). From @jonashaag's notebook, we can see that julius frequency response main lobe is wider than others, which I think is a result from the window function.

Current julius window function is fixed to use hann window:

If we add an option of window functions, user can have more choice to control the quality besides raising number of zeros.

BTW: As a side note, can I ask why not using fft_conv in ResampleFrac? Sinc interpolation use very large kernel, and I think it suites perfectly to use FFT to accelerate sinc interpolation.

Nice work.
But it seems there're already resample implemented by conv1d in pytorch's official torchaudio
Is there any necessary to implement it again?

If I want to install julius, what is the minimum version of torch I should use? Julius seems to be incompatible with torch 1.5.1.

As you say in the code, a high pass filter can be implemented by using a low pass filter and doing some subtraction. A convenience function that does this would be nice :) This makes it easier for developers with less DSP knowledge to apply high pass filtering

Do we have bandpass filter also available ?

def apply_bandpass(x, lf=20, hf=512, order=8, sr=2048):
    sos = signal.butter(order, [lf, hf], btype="bandpass", output="sos", fs=sr)
    normalization = np.sqrt((hf - lf) / (sr / 2))
    return signal.sosfiltfilt(sos, x) / normalization

First, let me say THANK YOU for reducing the time spent resampling ~ 30x in my particular application, compared to using resampy with the kaiser_fast mode. And thereby also reducing total application runtime ~ 50%! <3

After swapping resampy for julius my application spent most of its time in _init_kernels because I was using the resample_frac interface. I've worked around this by caching ResampleFrac instances:

def resample(wav_arr, from_sr, to_sr, _resamplers={}):
    try:
        resampler = _resamplers[(from_sr, to_sr)]
    except KeyError:
        resampler = _resamplers[(from_sr, to_sr)] = julius.resample.ResampleFrac(from_sr, to_sr)
    with torch.no_grad():
        return resampler(torch.from_numpy(wav_arr)).numpy()

I wonder what you think about caching those instances (or maybe only the kernels) by default, OR adding a hint to the documentation that you might want to use the ResampleFrac interface if you're repeatedly resampling between the same sample rates.

Hi Alexandre :)

Consider this scenario:

We have a machine learning model that takes in and outputs audio at a high sample rate. For some reasons (amongst other things execution time) the model uses an internal sample rate that is lower than the input-output. The output shape is expected to be the same as the input shape.

class ResampleWrapper(nn.Module):
    """
    This class downsamples audio before it's passed to the audio denoiser model,
    and upsamples the audio back to the original sample rate before returning.
    """

    def __init__(
        self,
        model: nn.Module,
        input_sample_rate: int = 48_000,
        internal_sample_rate: int = 32_000,
    ):
        super().__init__()
        self.model = model
        self.input_sample_rate = input_sample_rate
        self.internal_sample_rate = internal_sample_rate

        self.downsampler = ResampleFrac(
            self.input_sample_rate, self.internal_sample_rate
        )
        self.upsampler = ResampleFrac(self.internal_sample_rate, self.input_sample_rate)

    def forward(self, x):
        """
        :param x: tensor with shape (batch_size, num_channels, num_samples)
        :return: tensor with shape (batch_size, num_channels, num_samples)
        """
        x = self.downsampler(x)
        x = self.model(x)
        x = self.upsampler(x)
        return x

Depending on the exact length of the input, downsampling and then upsampling will often give an output that is off by one sample in length.

In librosa this kind of issue can be solved by setting the fix parameter or by using fix_length manually.

I imagine that julius.ResampleFrac could provide a parameter called something like expected_length in its forward function that customizes the slice end offset in the end so that the result has the given length 😄

Would you like a pull request that adds this feature?

Julius works great, but I notice that the memory usage grows super-linearly with filter length. Are there any settings to change or improvements to be made that could reduce gpu memory consumption?