keitakurita / contextual_embedding_bias_measure Goto Github PK

Hello,

I went through the notebooks for NIPS method replication, however, I couldn't find some files.
Are the files:

• gender_occ_pos_w_probs_train.txt
• gender_occ_neg_w_probs_train.txt
• gender_emo_w_probs_train.txt
• gender_occ_pos_w_probs_val.txt
• gender_occ_neg_w_probs_val.txt
• gender_emo_w_probs_val.txt
  available anywhere?

In

weat_result_replication_permute_targets.ipynb

There seems to be problem with get_index() called in function in get_mask_fill_logits.

print(np.log(get_mask_fill_logits("[MASK] are beautiful.", ["clover"])['clover']))
print(np.log(get_mask_fill_logits("[MASK] are [MASK].", ["clover"], use_last_mask=True)['clover']))

results in

gender_comes_first=False

Tokens for ptgt : [[MASK], are, beautiful, .]
0 [MASK]
return maskid from get_index() function : 1
response from get_mask_fill_logits : -18.595278

gender_comes_first=True

Tokens for prior : [[MASK], are, [, [UNK]]
0 [UNK]
1 [
2 are
3 [MASK]
maskid : 0
response from get_mask_fill_logits : -10.874125

gender_comes_first=True
Expected response:

Tokens for prior : [[MASK], are, [MASK], .]
0 [MASK]
1 are
2 [MASK]
3 .
maskid : 3
response from get_mask_fill_logits : -28.40166