About orig_to_token_index padding problem about pytorch_neural_crf

[allanj]

For code[2], it is practically same if we use some other index instead of 0.

Because in CRF, we will take the subsequence representation instead of the full representation.

Particularly in this step (https://github.com/allanj/pytorch_neural_crf/blob/master/src/model/module/linear_crf_inferencer.py#L102),
if the word_seq_len is only 8, only the max length is 10, we only take the scores up to 8. The rest will not be considered.

[EternalEep]

I have understood what you have meant. So you use mask in CRF layer. Thank you for your explanation for me.
It seems that I should read the CRF carefully about the CRF code, thank you, Allan.