raphaelsty / neural-cherche

Neural Search

Hello 👋

₁ Passionate about Language Models, Information Retrieval and Knowledge Graphs (PhD).

₂ Love modelizing things using ML. Still haven't found the killer feature yet.

₃ Love sharing models using APIs.

₄ Strong interest for databases and SQL.

₅ Strong interest for the retrieval augmented generation (RAG) paradigm -> cherche, neural-cherche, neural-tree

₆ My personal Knowledge Base is available here.

neural-cherche's Issues

could you please explain the rational behind applying clip on ranking loss

neural-cherche/neural_cherche/losses/ranking.py

Line 74 in 97ba18d

return torch.clip(loss, min=0.0, max=1.0)

How did you train SparseEmbed?

First, awesome project!

How did you train your model at https://huggingface.co/raphaelsty/neural-cherche-sparse-embed? Did you train it from scratch? I found an old copy of your sparsembed library. Was that library used or was this repository? What data did your train on exactly?

I am surveying various sparse embedding models and SparseEmbed while interesting has very little code or docs beyond the original google paper. Any assistance would be appreciated. Thanks!

How to train ColBERT model on a lot of data?

How do i use the trainer to train on a lot of data on limited RAM on colab. Im using the V100 16GB GPU and have close to 6.9M training examples/triplets

What is licence of the splade/sparsembed models hosted in huggingface?

Hi @raphaelsty, could you please let me know what is the licence of the below splade/sparsembed models hosted in huggingface?

https://huggingface.co/raphaelsty/splade-max
https://huggingface.co/raphaelsty/distilbert-splade
https://huggingface.co/raphaelsty/sparsembed-max
https://huggingface.co/raphaelsty/distilbert-sparsembed

Load the triplets input incrementally

Hi, I have triplets whose size does not fit my memory. Is there a way to avoid OOM, say, by loading "X" incrementally from a file instead?

DeepImpact implementation

I would love to see DeepImpact (https://arxiv.org/pdf/2104.12016.pdf) here.

License

Hello, thank you for the fantastic work!
Could you please tell me the license of this repository?

Mismatch in documents embeddings size

Hi,

Thanks for the great work on the repo.

While attempting to encode documents using my custom dataset, I've encountered a discrepancy in the number of input docs and the number of embedding produced. For eg.

ranker_documents_embeddings = ranker.encode_documents(
    documents=documents, # total: 30452
    batch_size=batch_size,
)
print(len(ranker_documents_embeddings), len(documents)) #prints 30427 30452

Do you have any insight into what might be causing this issue?

Thanks

[Feature request] ColBERT V2

Hi @raphaelsty,

first of all, thanks a lot for your this project. I really appreciate its simplicity and effectiveness.

Question: do you have any plans to implement ColBERT V2?

Best wishes.

add_duplicates() got an unexpected keyword argument 'results'

Hey there !

I'm trying to run the "evaluate" demo code written in the documentation but I run into this error :

scores = add_duplicates(queries=queries, results=scores)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: add_duplicates() got an unexpected keyword argument 'results'

And when looking in the library code, at the neural_cherche/utils/evaluate.py path, we can see that the add_duplicates function has 2 params : queries and scores but not 'results'.

Otherwise, thanks for the library, excellent work 👍 !

Questions about model evaluation

When I used the pre-trained model 'raphaelsty/neural-cherche-sparse-embed' to evaluate the dataset, specifically, the arguana dataset, with a retrieval k value of 100, the result was very poor
{'map': 0.033567943638956016,
'ndcg@10': 0.042417859280348115,
'ndcg@100': 0.08691780846498275,
'recall@10': 0.09815078236130868,
'recall@100': 0.32147937411095306}
As shown above, ndcg is only 4.2%