firstbatchxyz / firstbatch-sdk Goto Github PK

Description:

It would be beneficial to integrate Qdrant as one of the supported VectorStores. Qdrant is a powerful vector search engine, and widely used by AI community

Implement a new class Qdrant that inherits from the VectorStore abstract class.

Title: Compatibility Issues with the Latest Pydantic Version and Other Libraries

Issue Body:

I noticed that several other libraries in our project are not compatible with this new version of Pydantic. Due to wider support concerns, I'm considering downgrading to Pydantic version 1.10.13.

Details:

Compatibility Issues:
The latest version of Pydantic seems to have introduced changes that are not compatible with some of the libraries we heavily rely on.

Affected Libraries:

Semantic Kernel

Error Traceback (from Library A as an example):

[ERROR] Runtime.ImportModuleError: Unable to import module 'app': No module named 'pydantic_core._pydantic_core' Traceback (most recent call last):

Title: asyncio Requirement Causes AWS Lambda to Use Outdated Version

Issue Body:

I've encountered an issue with our library's integration with AWS Lambda. It appears that when asyncio is set as a requirement for our library, AWS Lambda defaults to using a non-maintained version of the runtime. I believe we should consider removing asyncio from our requirements to ensure compatibility with AWS Lambda and its users.

Details:

Issue:
When deploying our library to AWS Lambda, the inclusion of asyncio as a requirement causes AWS Lambda to use an outdated runtime version. And it leads to runtime errors in Python3.7+.

Title: Enhancement: Update Signal Function to Accept Vector Input

Issue Body:

Been working with the signal function in our library and the current version of it only accepts content IDs as input. For our use case, and potentially for others, it would be immensely beneficial if this function could be enhanced to accept vector inputs directly.

Desired Enhancement:

Update the signal function to accept vector inputs directly.
Update the documentation to reflect this change and provide examples of both use cases.

Our SDK is capable of storing content ids in a history, and make sure if a content exists in history it is not returned, and a new one is given instead. This makes sense for a number of use-cases, but a relaxed version of this should also be possible.

In my case, the user is solving a set of questions; say A B C D. The following happens:

• She solves A, and does not want to see this question again in a batch. The SDK moves away from questions like A as well.
• She solves B but signals that she would like to see it again in a future batch. The SDK stays around similar questions for personalization.
• She cant solve C and definitely wants to try more related questions.
• She didn't attempt D at all, so it should be returned again in a future batch.

The described scenario is similar to flash-card training programs.

Problem Description:
When creating a Like button, I can manage the state according but not sure we could remove it after passing to the vectordb.

Possible Solution:
It would be good to have a way to "dislike" (for the LIKE Signal) or remove signals in general.
Like .remove_signal or similar, the same way we have a .add_signal method.

ChromaDB integration, Where clause error

Where clause can't be accesed with current imports.

File "/home/runner/Bounty-FirstBatch/venv/lib/python3.10/site-packages/firstbatch/async_core.py", line 97, in add_vdb
    results = await vs.a_multi_search(batch)
  File "/home/runner/Bounty-FirstBatch/venv/lib/python3.10/site-packages/firstbatch/vector_store/chroma.py", line 167, in a_multi_search
    results = await asyncio.gather(*coroutines)
  File "/home/runner/Bounty-FirstBatch/venv/lib/python3.10/site-packages/firstbatch/vector_store/chroma.py", line 130, in asearch
    return await asyncio.get_event_loop().run_in_executor(None, func)
  File "/nix/store/hd4cc9rh83j291r5539hkf6qd8lgiikb-python3-3.10.8/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/runner/Bounty-FirstBatch/venv/lib/python3.10/site-packages/firstbatch/vector_store/chroma.py", line 98, in search
    where=cast(Optional[Where], query.filter.filter),
NameError: name 'Where' is not defined

Add history filter for ChromaDB

Description:

History Mode ID field should be fed from metadata/payload not the id itself. This is due to the fact that vector stores implement id differently.

• Typesene does not allow "id" field inside metadata. So we use "_id"
• Qdrant does not accept "string" ids so object id and id inside payload is different causing bugs.

Make sure VectorStore has history_field attribute which looks for unique fields to filter out existing results.