[Feature]: Add a custom AI that can call rest API endpoint about k8sgpt

[AlexsJones]

Hi @lili-wan thanks for creating this issue.
If you're using an openAI like API you can configure an alternative endpoint e.g.

bash k8sgpt auth new --backend localai --model <model_name> --baseurl http://localhost:8080/v1

Would this work for you? Or do you have a completely different API?

[warjiang]

Hi @lili-wan thanks for creating this issue. If you're using an openAI like API you can configure an alternative endpoint e.g.

bash k8sgpt auth new --backend localai --model <model_name> --baseurl http://localhost:8080/v1

Would this work for you? Or do you have a completely different API?

Is it necessary to provide a localai sample project for user to make integration with their self hosted llm? Just a suggestion. @AlexsJones

[AlexsJones]

Hi @lili-wan thanks for creating this issue. If you're using an openAI like API you can configure an alternative endpoint e.g.

bash k8sgpt auth new --backend localai --model <model_name> --baseurl http://localhost:8080/v1

Would this work for you? Or do you have a completely different API?

Is it necessary to provide a localai sample project for user to make integration with their self hosted llm? Just a suggestion. @AlexsJones

There are lots of tutorials on how to use k8sgpt/localAI.
The question I have is regarding @lili-wan use case for a generic restful API.

[remmen-io]

I've tried to setup the localai to point to a local endpoint made with huggingface tgi.

 k8sgpt auth update localai --model tgi --baseurl https://deepseek.k8scluster.ch/v1

but I get a

➜ k8sgpt analyze -b localai --explain
   0% |                                                                                                              | (0/17, 0 it/hr) [0s:0s]
Error: failed while calling AI provider localai: error, status code: 422, message:

Running a test query although works fine

curl https://deepseek.k8scluster.ch/v1/chat/completions \
    -X POST \
    -d '{
  "model": "tgi",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "What is deep learning?"
    }
  ],
  "stream": true,
  "max_tokens": 100
}' \
    -H 'Content-Type: application/json'