bam/run_classifier.py model always load distill_outputs from _train_predictions_1.pkl about google-research

[clarkkev]

I don't see a bug. Line 131 loads from {$BAM_DIR}/models/{teachers[task]}/output/{task}_train_predictions_1.pkl. Therefore it should work as long as you have the files

$BAM_DIR/models/mrpc-model/outputs/mrpc_train_predictions_1.pkl
$BAM_DIR/models/rte-model/outputs/rte_train_predictions_1.pkl

Based on the error message, it looks like the predictions file is present but incorrect in some way. Perhaps you trained the single-task models with {"debug": True} and are now training in non-debug-mode? Debug mode uses a tiny train set, so the written-out predictions won't cover the full train set.

[SanPianWang]

Thanks for your reply, after another trying, it fixed.
But i wonder that when writting the test outputs in run_classifier.py Line 269, the distill_input perhaps should be loaded from the corresponding files like: {$BAM_DIR}/models/{teachers[task]}/output/{task}_test_predictions_1.pkl? Due to the testing examples are pickling from the test.tsv
Is there any mismatch between the testing examples and distill inputs during the prediction?

[clarkkev]

Distillation is only used during training, not at test time. So although we write out the test predictions because they might be useful for evaluation or error analysis, they are never needed for training a BAM model.

[SanPianWang]

Thanks for the explanation

[Mahhos]

Thanks for your reply, after another trying, it fixed.
But i wonder that when writting the test outputs in run_classifier.py Line 269, the distill_input perhaps should be loaded from the corresponding files like: {$BAM_DIR}/models/{teachers[task]}/output/{task}_test_predictions_1.pkl? Due to the testing examples are pickling from the test.tsv
Is there any mismatch between the testing examples and distill inputs during the prediction?

May I know how you fix this? I am running BAM, and on one of my datasets, I am getting key_error and I am not sure why.