[Install issue]: Packaging and Deployment Issue with chromadb-client in AWS Lambda about chroma

[tazarov]

@marichkazb, thanks for reaching out. Let me start by saying that your approach to Lambda is correct and is how many Chroma users are deploying/using Chroma in AWS.

Your original error does not seem to be an actual Chroma issue. From the trace, it appears to be related to pydantic models in the OpenAI package.

Regarding the second error, this appears to be some library version conflicts, which is a frequent thing in the fast-moving GenAI ecosystem. What are your system dependencies e.g. packages you have installed - chromadb-client and openai library alone?

Regarding your more specific question on the AWS Lambda. While I'll admit I am not expert in AWS stack, my personal preference would be a docker image over zipped dependencies. Have a look here for an example (https://github.com/erenyasarkurt/OpenAI-AWS-Lambda-Layer/blob/main/build/build.sh).

I understand that you can easily bake a docker image, upload it to ECR, and use it as the basis for your Lambda. If you're interested, I'll happily provide you with a more detailed example.

[marichkazb]

Note: also getting the following message when trying to install chromadb-client in the linux env. although when installing those manually, system responds that the requirement is already satisfied

pip3 install -t ./python/ chromadb-client

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. botocore 1.34.61 requires urllib3<1.27,>=1.25.4; python_version < "3.10", but you have urllib3 2.2.1 which is incompatible. aws-sam-cli 1.112.0 requires requests~=2.31.0, but you have requests 2.32.1 which is incompatible.

[marichkazb]

@tazarov thank you for your time!! I’ve created a docker image and currently use it as a basis for the Lambda function, it indeed resolved all dependency conflicts, thank you! 🙌🏻

Also, I was wondering if chroma uses any temporary files when quering the collection?

I’m using the following function get_results to get the context for the system prompt for openAI. Although it seems like within the scope of this function it attempts to write files, resulting in an error: "error": "[Errno 30] Read-only file system: '/home/sbx_user1051’”. On AWS only /tmp folder is a writable directory, so any other attempt fails.

I tried setting the home environment to /tmp in the Dockerfile using ENV HOME=/tmp, but it didn’t help. If you have any ideas on how to possibly fix this, I'd really appreciate it!

def get_results(message):
    chroma_client = chromadb.HttpClient(host='11.11.111.11’, port=8000)
    from chromadb.utils.embedding_functions import SentenceTransformerEmbeddingFunction

    embedding_function = SentenceTransformerEmbeddingFunction()
    chroma_collection = chroma_client.get_collection("knowledge", embedding_function=embedding_function)

    results = chroma_collection.query(query_texts=[message], n_results=5)
    retrieved_documents = results['documents'][0]
    concatenated_string = ""
    for document in retrieved_documents:
        concatenated_string += str(document)
    return concatenated_string