Error running bert pretraining example about megatron-lm

[jaredcasper]

With only two sentences in your data set there isn't enough for a train/valid/test set, so you need to add more data or change the --split argument to something like 99,1,0 or something? You'll want to look at the " > dataset split:" part of the output, make sure there is data in the training and eval set.

[Lavenderjiang]

With only two sentences in your data set there isn't enough for a train/valid/test set, so you need to add more data or change the --split argument to something like 99,1,0 or something? You'll want to look at the " > dataset split:" part of the output, make sure there is data in the training and eval set.

I tried with a dataset with ~900 samples, but still encountered the error. Here's my output

dataset split:
    train:
     document indices in [0, 726) total of 726 documents
     sentence indices in [0, 726) total of 726 sentences
    validation:
     document indices in [726, 817) total of 91 documents
     sentence indices in [726, 817) total of 91 sentences
    test:
     document indices in [817, 908) total of 91 documents
     sentence indices in [817, 908) total of 91 sentences
 loading indexed mapping from my-bert_text_sentence_train_indexmap_160mns_509msl_0.10ssp_1234s.npy
    loaded indexed file in 0.001 seconds
    total number of samples: 0
  loading indexed mapping from my-bert_text_sentence_valid_indexmap_240mns_509msl_0.10ssp_1234s.npy
    loaded indexed file in 0.001 seconds
    total number of samples: 0
  loading indexed mapping from my-bert_text_sentence_test_indexmap_80mns_509msl_0.10ssp_1234s.npy
    loaded indexed file in 0.000 seconds
    total number of samples: 0
finished creating BERT datasets ...

[github-actions]

Marking as stale. No activity in 60 days. Remove stale label or comment or this will be closed in 7 days.

[github-actions]

Marking as stale. No activity in 60 days.