irapha / replayed_distillation Goto Github PK
View Code? Open in Web Editor NEWImplementation of Data-free Knowledge Distillation for Deep Neural Networks (on arxiv!)
Implementation of Data-free Knowledge Distillation for Deep Neural Networks (on arxiv!)
Hi, @irapha
I got a question:
For now,
I've followed the steps according to what you mentioned in README file,
and have finished training and compute-stats for hinton1200 model on mnist dataset.
It will generate new optimized dataset, and will be stored in many numpy files in /experiments/data/data_optimized_top_layer_experiment0:90:299.npy
But, for the next step: distill, which requires the optimized dataset got from the previous steps,
according to your command:
python main.py --run_name=experiment --model=hinton1200 \
--dataset=summaries/experiment/data/data_optimized_top_layer_experiment.npy \
--procedure=distill \
--model_meta=summaries/experiment/train/checkpoint/hinton1200-8000.meta \
--model_checkpoint=summaries/experiment/train/checkpoint/hinton1200-8000 \
--eval_dataset=mnist --student_model=hinton800 --epochs=30 --lr=0.00001
We need to input a data_optimized_top_layer_experiment.npy dataset, however,
we got several and separated npy optimized dataset at the previous step.
Do I need to do any further command or surgery on those optimized datasets,
to make it integrated as one dataset (i.e. summaries/experiment/data/data_optimized_top_layer_experiment.npy )?
Hi, @irapha:
Here I come again, hahah... ๐ฏโโ๏ธ
For the second command for visualization per-class and per-pixel means,
the documentation suggests us to have cmd like:
python viz/print_stats.py \
--dataset="summaries/experiment/data/data_optimized_all_layers_dropout_experiment_<clas>_<batch>.npy"
But, there's an error to do this, as shown in following picture:
After reading the code you've upload in "replayed_distillation/viz" folder,
I found there's no dataset as input argumentation in print_stats.py.
Should we use pixel_intensities.py
or pixel_intensities_batch.py
instead of print_stats.py
?
If we do have to use these two files, how to use these two files?
And what's the difference to use these two files?
Dear @irapha:
When create student model,
this line
https://github.com/iRapha/replayed_distillation/blob/943169e3b37f5453b386ae652af81db935efc7bd/procedures/distill.py#L28
shouldn't it be like:
outputs, _, feed_dicts = m.get(f.student_model).create_model(inputs_reshape, output_size)
Fix to come by @StefanoFenu
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.