Azure OpenAI: Can't seem to get it to work about hoarder-app

[MohamedBassem]

Can you share the logs of the workers container? Also in the example you shared, do you have an extra slash after "https" or is that just a typo?

[markheaps]

Sorry was just a typo in my post - here is a screen shot of the actual config section for openai.

Logs:

Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.0.6.tgz.

@hoarder/[email protected] start:prod /app/apps/workers
tsx index.ts
2024-05-04T16:47:32.687Z info: Workers version: nightly
2024-05-04T16:47:32.720Z info: [Crawler] Connecting to existing browser instance: http://chrome:9222
(node:34) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
(Use node --trace-deprecation ... to show where the warning was created)
2024-05-04T16:47:32.737Z info: [Crawler] Successfully resolved IP address, new address: http://172.20.0.4:9222/
2024-05-04T16:47:35.101Z info: Starting crawler worker ...
2024-05-04T16:47:35.103Z info: Starting inference worker ...
2024-05-04T16:47:35.104Z info: Starting search indexing worker ...
2024-05-04T16:50:31.725Z info: [search][3] Attempting to index bookmark with id kmehwk3wl5wnitrdbi39kqhi ...
2024-05-04T16:50:33.222Z info: [search][3] Completed successfully
2024-05-04T16:50:40.081Z info: [Crawler][2] Will crawl "https://yahoo.com" for link with id "awilbmha1kouz9kml982wz0t"
2024-05-04T16:50:40.085Z info: [search][4] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:41.070Z error: [search][4] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:41.548Z info: [Crawler][2] Successfully navigated to "https://yahoo.com". Waiting for the page to load ...
2024-05-04T16:50:42.103Z info: [search][4] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:43.193Z error: [search][4] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:43.911Z info: [Crawler][2] Finished waiting for the page to load.
2024-05-04T16:50:44.366Z info: [Crawler][2] Finished capturing page content and a screenshot.
2024-05-04T16:50:44.380Z info: [Crawler][2] Will attempt to extract metadata from page ...
2024-05-04T16:50:47.140Z info: [Crawler][2] Will attempt to extract readable content ...
2024-05-04T16:50:48.774Z info: [Crawler][2] Done extracting readable content.
2024-05-04T16:50:48.830Z info: [search][4] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:48.837Z info: [Crawler][2] Stored the screenshot as assetId: 79d42659-cf97-4958-9f67-6e1298855a20
2024-05-04T16:50:48.943Z info: [Crawler][2] Done extracting metadata from the page.
2024-05-04T16:50:48.943Z info: [Crawler][2] Downloading image from "https://s.yimg.com/cv/apiv2/social/images/yahoo_default_logo.png"
2024-05-04T16:50:49.002Z info: [Crawler][2] Downloaded the image as assetId: beef3931-1ad2-46ef-bf3e-c2a0898a5576
2024-05-04T16:50:49.884Z info: [Crawler][2] Completed successfully
2024-05-04T16:50:50.575Z info: [inference][2] Starting an inference job for bookmark with id "awilbmha1kouz9kml982wz0t"
2024-05-04T16:50:50.587Z error: [search][4] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:50.588Z info: [search][5] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:50.908Z error: [inference][2] inference job failed: Error: 404 Resource not found
2024-05-04T16:50:51.574Z info: [inference][2] Starting an inference job for bookmark with id "awilbmha1kouz9kml982wz0t"
2024-05-04T16:50:51.581Z error: [search][5] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:51.635Z error: [inference][2] inference job failed: Error: 404 Resource not found
2024-05-04T16:50:52.630Z info: [search][5] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:52.732Z info: [inference][2] Starting an inference job for bookmark with id "awilbmha1kouz9kml982wz0t"
2024-05-04T16:50:52.794Z error: [inference][2] inference job failed: Error: 404 Resource not found
2024-05-04T16:50:53.784Z error: [search][5] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:54.637Z info: [search][4] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:55.565Z error: [search][4] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:50:55.840Z info: [search][5] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:50:57.083Z error: [search][5] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:51:01.156Z info: [search][5] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:51:01.718Z error: [search][5] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:51:03.663Z info: [search][4] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:51:04.605Z error: [search][4] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.
2024-05-04T16:51:09.780Z info: [search][5] Attempting to index bookmark with id awilbmha1kouz9kml982wz0t ...
2024-05-04T16:51:10.617Z error: [search][5] search job failed: Error: Search task failed: internal: MDB_KEYEXIST: Key/data pair already exists.

[MohamedBassem]

2024-05-04T16:50:52.794Z error: [inference][2] inference job failed: Error: 404 Resource not found

I'm not sure if this means that the url is incorrect or the model name is incorrect. It might also be that the azure api not compatible with the openai sdk. I'll need to do more research.

[markheaps]

In my pyhton code, you have to pass it the Deployment name, in my case GPT4 so that is what I did here, but here are the options in Azure, but I don't think it takes Model name or Model Version in your code, but I can try those as well:

[MohamedBassem]

Maybe try using this as your base url 'https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME'

and the version as the model name?

Just a guess from:

https://learn.microsoft.com/en-us/azure/ai-services/openai/reference

[markheaps]

Still not having luck yet. Could this be the reason? Not following your code so not sure what your expecting on the Azure OpenAI version of this:

[markheaps]

Also how are you making the call (REST?) You might want to look at this:

https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=command-line%2Cpython-new&pivots=rest-api