siggigue / sigfeat

Feature Extraction from Signals e.g. for Audio Feature Extraction and Processing.

SigFeat: A Signal Feature Extraction Framework

Build Status:

Test Coverage:

BCH compliance:

Documentation:

This library is developed with focus on audio signals but it's base functionality is generalized to all kinds of (time)-signals.

The key advantages of this library are:

• SigFeat provides an appropriate and simple abstraction layer for the feature extraction problem:

  

• SigFeat minimizes computational cost by avoiding repeated computation of (interim) results. (For instance if many features depend on a result of another feature, this feature result is only computed once. Simple example: all Spectral features use one FFT output.)

• SigFeat has a low memory footprint due to generators (except your own defined features blow up memory or your sources load all data at once...).

See the examples folder, the feature, source and sink subpackage to check the intended usage of the library.

Some parts of the library (the parameters module) are inspired by luigi.

Simple Example Usage

from pylab import plt, np

from sigfeat import Extractor
from sigfeat.feature import SpectralFlux, SpectralCentroid, SpectralFlatness
from sigfeat.source.soundfile import SoundFileSource
from sigfeat.preprocess import MeanMix
from sigfeat.sink import DefaultDictSink


extractor = Extractor(
    SpectralFlux(),
    SpectralCentroid(),
    SpectralFlatness(),
)


source = MeanMix(SoundFileSource('bonsai.wav'))

sink = extractor.extract(source, DefaultDictSink())


plt.figure()

for label, result in sink['results'].items():
    plt.plot(result), label=str(label))

plt.legend()
plt.show()

Structure

The main base classes are: Source, Feature, Extractor and Sink.

Seven principal classes build up the whole framework:

1. Feature
2. Source
3. Sink
4. Extractor
5. Preprocess
6. Parameter
7. Result

Additionally to the basic framework, some commonly known features, useful sources and sinks are implemented. But it is not the focus of this project to provide a complete feature collection. It is up to the user to define own features, sources and sinks. This framework allows many different usage scenarios... be creative.

Requirements

The base framework only depends on six (sigfeat.base and sigfeat.extractor).

The namespaces sigfeat.feature .source and .preprocess depend on the scipy stack, soundfile and pyfilterbank. It depends on which submodules are used.

sigfeat's Issues

Correct SpectralFlux

Something like this (multichannels not supported in this example)

class SpectralFlux(Feature):
    axis = Parameter(0)
    fmin = Parameter()
    fmax = Parameter()
    
    def requires(self):
        yield AbsRfft

    def on_start(self, source, featureset, sink):
        nfft = featureset['AbsRfft'].nfft
        freqs = featureset['AbsRfft'].frequencies
        fmin, fmax = self.fmin, self.fmax
        if fmin and fmax:
            self._index = np.logical_and(freqs >=fmin, freqs < fmax)
            nfft = np.sum(self._index)
        else: 
            self._index = self._lastspec > 0
        
        if source.channels > 1:
            self._lastspec = np.ones((nfft, source.channels))
        else:
            self._lastspec = np.ones(nfft)

        if not any(self._index):
            raise ValueError('Could not find frequency bins between {} and {} Hz'.format(fmin, fmax))
                    
    def process(self, data, featuredata):
        curspec = featuredata['AbsRfft'][self._index]
        ds = curspec - self._lastspec
        specflux = np.sum((ds + np.abs(ds)) * 0.5)
        self._lastspec = curspec
        return specflux
    
    @classmethod
    def gen_by_edge_frequencies(cls, edgefreqs, prefixname='SpectralFlux'):
        for le, ue in zip(edgefreqs[:-1], edgefreqs[1:]):
            yield cls(name='{}{}to{}Hz'.format(cls.__name__, le, ue), fmin=le, fmax=ue)