qinxiaoyi / heymann-nn-gev-bf

A Pytorch implementation of 'Neural network based spectral mask estimation for acoustic beamforming'

A refactoring of heymann's 'Neural network based spectral mask estimation for acoustic beamforming'

his ppt on icassp 2016

references, tools

• paper
• author's git repo
• A Well-Conditioned and Sparse Estimation of Covariance and Inverse Covariance Matrices Using a Joint Penalty
• Xueliang's paper
• xiong's paper
• google's paper
• higuchi's paper

requirments

• python3
• pytorch
• kaldi
• matlab

tools

• AudioLabs's rir generator
• (https://github.com/ehabets/RIR-Generator/blob/master/rir_generator.pdf)
• reverb challenge 2014 generator
• CHiME3's simulation code
• gnu parallel
• sox
• https://kr.mathworks.com/matlabcentral/fileexchange/23573-csvimport

to do

• print (avg, max, median) of L2 norm
• gev BF
• add timer(of
• MVDR BF
• PESQ
• SNR
• F-score for each bin

Installation

cd your_kaldi/egs
git clone [email protected]:gogyzzz/heymann-nn-gev-bf.git
cd heymann-nn-gev-bf/s5

Data preparation

mylocal/prepare_noise.sh

mylocal/prepare_wsjcam0.sh
mylocal/prepare_mixed_wsjcam0.sh

matlab -nodesktop -nosplash -r \
 "mix('ext/mixed/wsjcam0/si_dt/mixed.csv', 1024000); exit;"
 
mylocal/prepare_chime3.sh

Data preparation 2

# for pytorch dataset, dataloader

def wav_to_ibm(clean, noisy, channel=-1):
    return (y_psd, x_psd, n_psd, x_mask, n_mask)

# psd normalization needed? -> no. just use batchnorm

Exp. results