Docs feedback from tutorial user about chatgpt-your-files

[gregnr]

@louisguitton this is great feedback, thanks for taking the time to share 💚

• ON DELETE CASCADE on document_sections: Yep this is better 👍
• Transformers.js requires ONNX model: Definitely worth adding a note here. I also did a slightly more detailed explanation in this video, in case that helps.
• Fallback if embed edge function fails: This is a good point. Functions in the private schema should still be callable from the SQL editor, but in this case since it's specifically a trigger function, you won't be able to manually call it. Some potential options:
  • Add another function that can be triggered manually which checks for document_sections with missing embeddings and invokes the /embed edge function for them.
  • Create a scheduled function that periodically checks for document_sections with missing embeddings and re-generates them. We would likely need to add a locking mechanism (ie. via another column) to prevent the scheduled function from conflicting with the normal embed trigger.
• Using other LLMs: Yep, this will be a great addition. We are working on some Ollama content right now that will likely relate to this. Some other providers with OpenAI-compaitble API:
  • together.ai
  • fireworks.ai
  • perplexity.ai

Would you (or anybody else) be interested in opening a PR to help document these points?

[gregnr]

Awesome, thanks! 1 PR for all points is fine 👍

Let's modify the relevant sections directly and then just add a callout underneath that explains what was changed since the video. eg:

Since the video was published, on delete cascade was added here so that the lifecycle of document_sections is tied to their respective document.

For the LLM alternatives, maybe we just add a note/callout that briefly explains how you might swap in other models (without going into too much detail which would probably better suited for another tutorial). Something along the lines of just ensuring that whichever provider they choose offers an OpenAI compatible API, then modifying the baseURL accordingly.

Similar to what Together AI describes here:
https://docs.together.ai/docs/openai-api-compatibility#nodejs

Or Ollama here:
https://github.com/ollama/ollama/blob/main/docs/openai.md#openai-javascript-library

What do you think?

[louisguitton]

went out for lunch, came back with fresh eyes and remembered my docker networking basics 😂 .
For the supabase docker stack to reach Ollama, we need to specify host.docker.internal instead of localhost in the connection string. So this works 🎉 :

const openai = new OpenAI({
  apiKey: "ollama",
  baseURL: "http://host.docker.internal:11434/v1/",
});

[louisguitton]

I've opened #18 to close this

[louisguitton]

Thanks for the reply and added context @gregnr !
I'd gladly open a PR for this. A few questions:

• 1 PR addressing all points is fine or 4 small PRs preferred?
• for changes like the delete cascade that won't be mentioned in the video play-through, would you rather have them in context of the part? (E.g. in part 2 for part 2 changes) or at the end in an annexe section?
• I have the code for Ollama, unfortunately as a next.js edge function not a Supabase edge function (couldn't connect to Ollama, and didn't manage to debug enough in Deno). Is that fine or would you rather have a supabase edge function? (Ollama is anyways for local dev)

Thanks and best

[louisguitton]

That sounds great, I'll start the PR.

For using other LLMs though, I can't get a working code sample with Deno within Supabase functions. This line fails with this error:

event loop error: Error: undefined Connection error.
    at i.makeRequest (https://esm.sh/v135/[email protected]/esnext/core.js:4:5011)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async Object.handler (file:///home/deno/functions/chat/index.ts:103:26)
    at async handleHttp (ext:sb_core_main_js/js/http.js:35:25)
failed to send request to user worker: connection closed before message completed
InvalidWorkerResponse: user worker failed to respond
    at async Promise.all (index 1)
    at async UserWorker.fetch (ext:sb_user_workers/user_workers.js:64:19)
    at async Server.<anonymous> (file:///home/deno/main/index.ts:128:12)
    at async Server.#respond (https://deno.land/[email protected]/http/server.ts:220:18) {
  name: "InvalidWorkerResponse"
}

The same code works with a Next.js edge function. I'll add a separate PR with reproducible code for you to try out. Maybe you see something obvious that will point me to a working solution. I don't want to document "use Ollama as drop-in replacement" if I don't have the code working.

[louisguitton]

So, after "a bit" of debugging, I've got working code for Ollama + Supabase Deno function, but I had to ngrok my localhost Ollama for Supabase Deno functions to be able to see it.
I've pushed the code to https://github.com/louisguitton/chatgpt-your-files/tree/ollama
to reproduce:

npm run dev
npx supabase start
ollama serve
ngrok http 11434
npx supabase functions serve