jeremybboy / compressed-drumgan

Compressing Neural Networks with Distillation, Pruning and Quantization

The complete research thesis can be read if you click on the following pdf :

Visualization and Listening

To generate samples, we take a latent vector of size 131 and pass it to the teacher and the student. For each model, we take 2 same latent spaces and obtain 2 generations. The student generation should sound like his corresponding teacher generation, as it is the same latent vector used.

Model 1 : Only MSE Generation teacher, Generation student

Teacher

Your browser does not support the audio element. Your browser does not support the audio element. 

Student reconstruction

Your browser does not support the audio element. Your browser does not support the audio element.

Model 2 : Add feature loss

Teacher

Your browser does not support the audio element. Your browser does not support the audio element. 

Student reconstruction

Your browser does not support the audio element. Your browser does not support the audio element.

Model 3: Feature Loss + Classif Loss

A. Feature Loss + Classif Loss with Fake Data

Teacher

Your browser does not support the audio element. Your browser does not support the audio element. 

Student reconstruction

Your browser does not support the audio element. Your browser does not support the audio element.

B. Feature Loss + Classif Loss with Real Data

Teacher

Your browser does not support the audio element. Your browser does not support the audio element. 

Student reconstruction

Your browser does not support the audio element. Your browser does not support the audio element.

Model 4: Feature Loss + Classif Loss + change in Leaky Relu Last layer

Teacher

Your browser does not support the audio element. Your browser does not support the audio element. 

Student reconstruction

Your browser does not support the audio element. Your browser does not support the audio element.