hannibal046 / xrag Goto Github PK

Thanks for the great work! As the paper fix the parameter of retrieved chunks to 1, I wonder does the performance still good if the num of documents retrieved is large than 1, which is more similar to the real world RAG situation. Does some performance score has been test?

The stopping criteria only works for batch size = 1. I don't get an error, but no stopping criteria is applied at all for batch size > 1

Thanks for the great work. From your paper, I understood that you are using EM for the evaluation of the open QA datasets. From the evaluation code, it seems like you are using the Match metric. Could you please clarify?

作者您好！xrag这篇工作非常惊艳，我们在复现它的结果，请问下能否提供处理好的训练和测试用的数据？非常感谢！

As titled. Do we need to train retriever?
tokenizer = RetrieverTokenizer.from_pretrained(args.base_model,additional_special_tokens=["[Q]","[D]"])

It seems that PwC_train dataset is not mentioned in readme

Throughout the repo I find different values for max_new_tokens,

• 30:

  xRAG/README.md

  Line 79 in 121fa41

  python -m src.language_modeling.profiler --instruction_length 54 --generation_length 30 --dataset triviaqa --use_xrag

• 100:

  xRAG/src/eval/run_eval.py

  Line 227 in 121fa41

  max_new_tokens=100,

• 20: https://github.com/Hannibal046/xRAG/blob/121fa4180a8c1fa0ec1af5901d879452e5c9ce89/tutorial.ipynb

can you please indicate which number should be used?

Congratulations for the great work.

I have the following doubt: The reference document/representation retrieved is only based on the chunks from wikipedia or does it also include the document references from each of the datasets? For example, hotpotqa could benefit more from retrieving the relevant documents from their dataset compared to wikipedia dump. Am I missing something. ?

Thanks for your time.

The xrag-7b already knew about Motel 6 when I try tutorial.ipynb.

Was the model updated?

This is the response without RAG or xRAG.

Motel 6. Motel 6 is a budget motel chain in the United States and

But thanks for providing a helpful tutorial.