AWS Bedrock Embedding doesn't work about weaviate

[antas-marcin]

Hello, I'm working on fixing this issue, I think that I will be able to fix this by Thursday (maybe even sooner), once this is resolved I will write about this here

[antas-marcin]

I have a PR ready that fixes Bedrock support. I hope to merge it tomorrow and hopefully make a patch release on Friday.

[antas-marcin]

@chrisk314 we have switched to AWS SDK and now the Bedrock service URL is being generated internally by SDK, we don't control it anymore. And yes, you are right, we don't log the requests to Bedrock.

btw we have released v1.24.12 which contains a fix for this issue.

[chrisk314]

@antas-marcin ok thanks for confirming on the logs and for the patch release.

[kobrinartem]

tested on us-west-2:

weaviate-aws-weaviate-1 | {"action":"requests_total","api":"rest","class_name":"Article","error":"update vector: send POST request: send POST request: Post "https://bedrock-runtime.us-west-2.amazonaws.com/model/cohere.embed-multilingual-v3/invoke\": context canceled","level":"error","msg":"unexpected error","query_type":"objects","time":"2023-12-22T12:55:40Z"}

[kobrinartem]

when I tried cloud version I got another error:

uuid = client.data_object.create(
    class_name="Article",
    data_object={
            "title": "test",
            "body": "test content",
        }
)

File .venv/lib/python3.11/site-packages/requests/models.py:975, in Response.json(self, **kwargs)
971 return complexjson.loads(self.text, **kwargs)
972 except JSONDecodeError as e:
973 # Catch JSON-related errors and raise as requests.JSONDecodeError
974 # This aliases json.JSONDecodeError and simplejson.JSONDecodeError
--> 975 raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: Expecting value: line 2 column 1 (char 1)

[sandeepchandra]

I encountered a similar challenge while attempting to generate embeddings using the AWS Bedrock model - amazon.titan-embed-text-v1. Utilizing a Docker setup with the image semitechnologies/weaviate:1.23.9, I aimed to add batch objects to the weaviate vector database. But ended up with read timeouts and invalid bedrock API URL (which doesn't have 'bedrock-runtime' in the domain) as above.

To resolve the issue, I conducted an investigation into the code of the Weaviate repository (master branch). It appears that within the codebase, there exists a conditional statement that distinguishes the URL creation for model IDs beginning with 'cohere' and 'amazon'. A snippet of this code is depicted below:

Having identified this condition, I proceeded to remove it, thereby allowing for the creation of the service name regardless of the model ID. Subsequently, I rebuilt the Docker image and applied it within my use-case, resulting in the desired outcomes.

[bbeaumatin]

Hello, bumping this a little bit. There is indeed a bug here preventing from using the titan embedding model, as the URL is supposed to start with "bedrock-runtime" and given this condition in the code shown by @sandeepchandra it is not possible to build the URL properly, then the vectorization doesn't work. I'm not much of a Go developer but if needed I can propose a PR. Let me know if it's needed or if I misunderstood the configuration to do to have the titan model working, considering the doc states it's compatible:

Thanks!

[IzioDev]

URL should be : https://bedrock-runtime.${REGION}.amazonaws.com/model/${MODEL_ID}/invoke for Bedrock service

[chrisk314]

We're also hitting this. Any update on timeline for the fix?

[antas-marcin]

If you want to test it here's a preview docker image:

semitechnologies/weaviate:preview-fix-text2vec-aws-module-bedrock-support-730b4c5

I have added support for AWS_SESSION_TOKEN in the module if one wants to pass using headers then X-Aws-Session-Token header should be used for that

[chrisk314]

@antas-marcin I can confirm this is working for us now using the preview docker image. Thanks for the fix! Looking forward to the patch release.

P.S. I could not see any evidence of the calls to Bedrock in the Weaviate logs. I know it's all working correctly because my integration test is passing; however, even with debug or trace level logging I can't see any reference to the calls to the Bedrock service in the logs. I see a lot of messages in the logs like the below example when I add an item to Weaviate. There is a path shown but no host. Would be nice if the logs actually showed the Bedrock service url.

{"action":"restapi_request","level":"debug","method":"GET","msg":"received HTTP request","time":"2024-05-09T09:51:31Z","url":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/nodes","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""}}

[antas-marcin]

Thank you @chrisk314! I will resolve this issue now