irapha / replayed_distillation Goto Github PK

Hi, @irapha
I got a question:
For now,
I've followed the steps according to what you mentioned in README file,
and have finished training and compute-stats for hinton1200 model on mnist dataset.
It will generate new optimized dataset, and will be stored in many numpy files in /experiments/data/data_optimized_top_layer_experiment0:90:299.npy

But, for the next step: distill, which requires the optimized dataset got from the previous steps,
according to your command:

python main.py --run_name=experiment --model=hinton1200 \
    --dataset=summaries/experiment/data/data_optimized_top_layer_experiment.npy \
    --procedure=distill \
    --model_meta=summaries/experiment/train/checkpoint/hinton1200-8000.meta \
    --model_checkpoint=summaries/experiment/train/checkpoint/hinton1200-8000 \
    --eval_dataset=mnist --student_model=hinton800 --epochs=30 --lr=0.00001

We need to input a data_optimized_top_layer_experiment.npy dataset, however,
we got several and separated npy optimized dataset at the previous step.

Do I need to do any further command or surgery on those optimized datasets,
to make it integrated as one dataset (i.e. summaries/experiment/data/data_optimized_top_layer_experiment.npy )?

Hi, @irapha:
Here I come again, hahah... 👯‍♂️
For the second command for visualization per-class and per-pixel means,
the documentation suggests us to have cmd like:

python viz/print_stats.py \
    --dataset="summaries/experiment/data/data_optimized_all_layers_dropout_experiment_<clas>_<batch>.npy"

But, there's an error to do this, as shown in following picture:

After reading the code you've upload in "replayed_distillation/viz" folder,
I found there's no dataset as input argumentation in print_stats.py.
Should we use pixel_intensities.py or pixel_intensities_batch.py instead of print_stats.py?

If we do have to use these two files, how to use these two files?
And what's the difference to use these two files?

Dear @irapha:
When create student model,
this line
https://github.com/iRapha/replayed_distillation/blob/943169e3b37f5453b386ae652af81db935efc7bd/procedures/distill.py#L28

shouldn't it be like:
outputs, _, feed_dicts = m.get(f.student_model).create_model(inputs_reshape, output_size)

Fix to come by @StefanoFenu