XTransformer model can't be trained it always shows cuda out of memory about pecos

[jiong-zhang]

You can use smaller batch size or smaller transformer encoder to save GPU memory. To train the models defined in examples/xr-transformer-neurips21/ please use AWS p3.16xlarge instance or larger.

[Pallavi3759]

predictions are accurate but scores are very low , is there any way to increase the prediction scores

[jiong-zhang]

could you provide an example of predictions are accurate but scores are very low?

[Pallavi3759]

@jiong-zhang Input text: guard services
Predicted label: Psc_S206
Predicted score: 1.0
Predicted label: Naics_523110
Predicted score: 1.0
Predicted label: Naics_523120
Predicted score: 1.0
Predicted label: Naics_523910
Predicted score: 1.0
Predicted label: Naics_523991
Predicted score: 1.0
Predicted label: Naics_561612
Predicted score: 1.0
Predicted label: Naics_561613
Predicted score: 1.0
Predicted label: Naics_922150
Predicted score: 1.0
Predicted label: Naics_922190
Predicted score: 1.0
Predicted label: Naics_928110
Predicted score: 1.0
Predicted label: Naics_922120
Predicted score: 1.0
Predicted label: Psc_R430
Predicted score: 0.006939241662621498
Predicted label: Psc_V002
Predicted score: 0.004626440815627575
Predicted label: Naics_485111
Predicted score: 0.004626440815627575
Predicted label: Naics_488410
Predicted score: 0.004626440815627575
Predicted label: Naics_711212
Predicted score: 0.004626440815627575
Predicted label: Psc_U004
Predicted score: 0.004298868589103222
Predicted label: Psc_F110
Predicted score: 0.004275548737496138
Predicted label: Psc_U014
Predicted score: 0.003936069086194038 all of the codes are relatable to the input but only few of them having good prediction score

[Pallavi3759]

@jiong-zhang is there any parameter we can use for incremental learning in xtransformer customxtf
??

[jiong-zhang]

XR-Transformer supports training from you own pre-trained model with init_model_dir, for details please see our tutorial.

As for the evaluation, please provide minimal example with PECOS evaluation functionalities so we can reproduce.

[Pallavi3759]

@jiong-zhang do we apply incremental learning in xtransformer for our custom data set???

[Pallavi3759]

@jiong-zhang we already use pretrained model in init_model_dir but can't recieve desired results we have huge data and our use case is extreme multilable txt classification but prediction score are not desired is there any way to improve them for ex- milk and dairy for nd sd and ne.
0 milk and dairy items for customers in utah and nevada area.
0 milk and dairy products.
0 milk and dairy requirements.
0 milk and ice cream delivery servicesasheville durham fayetteville and salisbury nc.
0 milk and ice cream products. we have these kind of corpus and output label file but can't recieve desired score

[jiong-zhang]

We don't have built-in support for data sharding in single node training (if that's what you mean by 'incremental training'). If the result is not desirable you should maybe consider using some other libraries.