0015 / chatgpt_client_for_arduino

Library for communication with ChatGPT. Now it supports Vision Question.

Hi there 👋

Welcome to That Project, a channel dedicated to showcasing exciting projects using microcontrollers, with a particular focus on the ESP32 series. Whether you're a hobbyist or a professional, you'll find plenty of inspiration here as I explore the world of IoT and demonstrate how to use microcontrollers to create a wide range of innovative projects.

Our videos cover a variety of topics, from using ChatGPT to enhance your microcontroller projects, to creating projects using ESP32-CAM and LVGL. I also offer detailed tutorials on programming, hardware, and other aspects of microcontroller development, so you can dive into the world of IoT with confidence.

Join our community of like-minded makers and innovators and discover the endless possibilities of microcontroller projects. Don't forget to subscribe to stay up-to-date with our latest videos and projects, and be sure to leave a comment if you have any questions or suggestions for future videos. My Channel

chatgpt_client_for_arduino's Issues

Getting a non Json formatted response

Hi,

Yesterday my integration with Arduino UNO R4 worked just fine, but today it seems like something is changed in the way the Chat GPT API framework handle the requests\response.
I was getting consistent and pretty neat JSON formatted response, but today this happened:

[ChatGPT] Response: HTTP/1.1 200 OK
Date: Wed, 17 Jul 2024 08:36:42 GMT
Content-Type: application/json
Content-Length: 523
Connection: close
openai-organization: user-[Redacted]
openai-processing-ms: 952
openai-version: 2020-10-01
strict-transport-security: max-age=15552000; includeSubDomains; preload
x-ratelimit-limit-requests: 10000
x-ratelimit-limit-tokens: 10000
x-ratelimit-remaining-requests: 9999
x-ratelimit-remaining-tokens: 9594
x-ratelimit-reset-requests: 8.64s
x-ratelimit-reset-tokens: 2.436s
x-request-id: req_b3096fbb7647b93f6e34613ac64d4e58
CF-Cache-Status: DYNAMIC
Set-Cookie: __cf_bm=3agTzRX30qFrJtYhEkMsZdNAFWzLyIHqp.j78cgrArc-1721205402-1.0.1.1-qGxb2cptvU1RayQk7l62kRkS0VR3.1xGYZllr6u1Ri4CmWVg8PiVllVGc.aPNa8cXn.l2vM4fQOEpgpTt_VoYA; path=/; expires=Wed, 17-Jul-24 09:06:42 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
X-Content-Type-Options: nosniff
Set-Cookie: _cfuvid=fb2.ZVLMHHsES1VBsSfrnJ4.rnvJWo42ThoWALIBsMg-1721205402822-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
Server: cloudflare
CF-RAY: 8a48e7dfae6abae2-MXP
alt-svc: h3=":443"; ma=86400

Already tried to tweak and change some parameters inside the API Playground \ Chat settings on Open AI side, but nothing is changed.

I only receive partial JSON responses when I intentionally send a bad parameter ( like changing the version of the AI with a wrong one )

EG:

{
"error": {
"message": "Cannot specify both model and engine",
"type": "invalid_request_error",
"param": null,
"code": null

BR

Fox

esp32 picture to base64 to chatGPT_Client.vision_question

I want to take picture with esp32 convert to base64 and send it to chatgpt

I have issue to convert the picture to base64, can you share your function

I came up with this function

const char* base64Image = Photo2Base64();
Serial.println(base64Image);

then pass base64Image to the function

if (chatGPT_Client.vision_question("gpt-4o", "user", "text", "What’s in this image?", "image_url", base64Image, "auto", 500, true, result)) {
    Serial.print("[ChatGPT] Response: ");
    Serial.println(result);
  } else {
    Serial.print("[ChatGPT] Error: ");
    Serial.println(result);
  }

if I paste this string into online tool, it does show my picture but there is error which could explain why chatgpt api does not take it, returns error.

• Check the repair tool and convert your value to a valid Base64 string.
• The specified string is a data URI that contains the Base64 value.

This is the function that take picture and convert to base64

const char* Photo2Base64() {
    camera_fb_t * fb = NULL;
    fb = esp_camera_fb_get();  
    if (!fb) {
      Serial.println("Camera capture failed");
      return NULL;
    }
  
    String base64String = "data:image/jpeg;base64,";
    char *input = (char *)fb->buf;
    char output[base64_enc_len(3)];
    for (int i = 0; i < fb->len; i += 3) {
      base64_encode(output, input, 3);
      input += 3;
      base64String += String(output);
    }

    esp_camera_fb_return(fb);
    
    // Allocate memory for the C-style string
    char *cString = (char*)malloc(base64String.length() + 1);
    if (cString == NULL) {
        Serial.println("Failed to allocate memory");
        return NULL;
    }
    
    // Copy the contents of the String object to the C-style string
    strcpy(cString, base64String.c_str());

    return cString;
}

next step for me is to try other base64 library until that online website does not detect any error

Getitng Connection Fail Error

I am trying to run sketch on Arduino Nano RP2040 Connect and getting 'Connection Faild Error'.

16:33:13.395 -> Connecting to Wi-Fi...Connecting to Wi-Fi...Connecting to Wi-Fi...Connecting to Wi-Fi... 16:35:51.138 -> Connected to Wi-Fi! 16:35:51.138 -> 16:35:51.138 -> [ChatGPT] Only print a content message 16:35:51.404 -> ===ERROR=== 16:35:51.404 -> [ERR] Connection failed!

Installed version 0.1.2
Sample Arduino Code

#include <ArduinoJson.h>
#include <WiFiNINA.h>
#include <ArduinoBearSSL.h>
#include <ChatGPT.hpp>

static const char *ssid = "xxxx";
static const char *password = "xxx";

WiFiClient client;
BearSSLClient sslClient(client);
ChatGPT<BearSSLClient> chat_gpt(&sslClient, "v1", "xxxxxx");

void setup() {
  Serial.begin(115200);

  // Wait for serial port to initialize
  while (!Serial) {
    delay(10);
  }

  // Connect to Wi-Fi network
  connectWiFi();

  // Ignore SSL certificate validation
//  client.setInsecure();

  String result;
  Serial.println("[ChatGPT] Only print a content message");
  if (chat_gpt.simple_message("gpt-3.5-turbo-0301", "user", "Planning a 3-day trip to San Diego", result)) {
    Serial.println("===OK===");
    Serial.println(result);
  } else {
    Serial.println("===ERROR===");
    Serial.println(result);
  }
}

void loop() {}

void connectWiFi() {
  // Connect to Wi-Fi network
}

Any suggestions would be helpful.

azure-openai compatible with azure open ai api?

Hi! Thanks for ur excellent work in implementing this great library for arduino and esp32 interacting with chatgpt. i hv met a question right now since i hv another free subscription of azure openai api, i wonder if this kind of key is compatible with ur library to use?

the most of requests are err

Using ESP8266

the most of requests are return error:
"[ERR] deserializeJson() failed: InvalidInput"

Interestingly, requests of the format: "trip for N days Somewhere" is always processed without errors :)

arduino unor4 wifi

hi i find that i even could not run the example

WARNING: library ArduinoECCX08 claims to run on samd, megaavr, mbed, mbed_nano, mbed_portenta, esp32 architecture(s) and may be incompatible with your current board which runs on renesas_uno architecture(s).
In file included from C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiS3.h:26:0,
                 from C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino:10:
C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiUdp.h:33:0: warning: "RX_BUFFER_DIM" redefined
 #define RX_BUFFER_DIM 1461
 
In file included from C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiS3.h:24:0,
                 from C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino:10:
C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiClient.h:33:0: note: this is the location of the previous definition
 #define RX_BUFFER_DIM 1024
 
C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino: In function 'void setup()':
C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino:45:53: warning: 'void BearSSLClient::setInsecure(BearSSLClient::SNI)' is deprecated: INSECURE. DO NOT USE IN PRODUCTION [-Wdeprecated-declarations]
   sslClient.setInsecure(BearSSLClient::SNI::Insecure);
                                                     ^
In file included from c:\Users\94837\OneDrive\Documents\Arduino\libraries\ArduinoBearSSL\src/ArduinoBearSSL.h:34:0,
                 from C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino:9:
c:\Users\94837\OneDrive\Documents\Arduino\libraries\ArduinoBearSSL\src/BearSSLClient.h:76:8: note: declared here
   void setInsecure(SNI insecure) __attribute__((deprecated("INSECURE. DO NOT USE IN PRODUCTION")));
        ^~~~~~~~~~~
In file included from C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFi.h:7:0,
                 from C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiS3.h:23,
                 from C:\Users\94837\AppData\Local\Temp\.arduinoIDE-unsaved2023613-25312-gs1els.4h57k\Arduino_BearSSLExample\Arduino_BearSSLExample.ino:10:
C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiCommands.h: At global scope:
C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiCommands.h:17:3: warning: 'file_op' defined but not used [-Wunused-variable]
 } file_op;
   ^~~~~~~
C:\Users\94837\AppData\Local\Arduino15\packages\arduino\hardware\renesas_uno\1.0.2\libraries\WiFiS3\src/WiFiCommands.h:10:3: warning: 'ip_type' defined but not used [-Wunused-variable]
 } ip_type;
   ^~~~~~~
c:/users/94837/appdata/local/arduino15/packages/arduino/tools/arm-none-eabi-gcc/7-2017q4/bin/../lib/gcc/arm-none-eabi/7.2.1/../../../../arm-none-eabi/bin/ld.exe: section .stack_dummy VMA [20007b00,20007eff] overlaps section .heap VMA [20005e68,20007e67]
collect2.exe: error: ld returned 1 exit status

exit status 1

Compilation error: exit status 1

Connection failed error

Hello thanks for the great lib,
since this morning I am getting an error when the request is made.
Do you think I should reinstall the library?

Initialisation done.
Connecting...
Connecting...
Connecting...
Connecting...
Connected!
[ChatGPT] Only print a content message
//this loads graphics
Successfully png file
image specs: (170 x 320), 8 bpp, pixel type: 2
57ms
===ERROR===
[ERR] Connection failed!

I have checked the internet connection, I will try to change my API key and report back

Too Many Requests

Hi, I get this output in Serial monitor. Any idea what should I do? I just opened an example ESP8266_WifiClientSecureBearSSL.ino. I aaded my API and name and password for wifi. I am using ESP8266 Wemos D1 Mini.

16:30:50.299 -> [String] 'POST /v1/c ...  ChatGPT
16:30:50.299 -> ': Reallocating large String(185 -> 217 bytes)
16:30:50.299 -> [String] 'POST /v1/c ... ion/json
16:30:50.343 -> ': Reallocating large String(217 -> 233 bytes)
16:30:50.343 -> [String] 'POST /v1/c ... ngth: 85
16:30:50.343 -> ': Reallocating large String(237 -> 256 bytes)
16:30:50.343 -> [String] 'POST /v1/c ...  close
16:30:50.343 -> 
16:30:50.343 -> ': Reallocating large String(258 -> 343 bytes)
16:30:50.527 -> ===ERROR===
16:30:50.527 -> HTTP/1.1 429 Too Many Requests

16:30:50.527 -> 
16:30:58.772 -> pm open,type:2 0

I cant run the example

`exit status 1

WifiCientSecure.h: no such file or directory`

Anyone can help me with this?

exit status 1

I am facing this error which I believe might relate to the mismatch version of Arduino_Json. Can you help me with this. Plenty of thank to your great work.

In file included from something\ChatGPT_Client\src/ChatGPT.hpp: In member function 'bool ChatGPT<T>::simple_message(const String&, const String&, const String&, String&)':
something\\Documents\Arduino\libraries\ChatGPT_Client\src/ChatGPT.hpp:30:7: error: 'DynamicJsonDocument' was not declared in this scope
       DynamicJsonDocument doc(result.length() + 200);
       ^~~~~~~~~~~~~~~~~~~
something\\Documents\Arduino\libraries\ChatGPT_Client\src/ChatGPT.hpp:31:7: error: 'DeserializationError' was not declared in this scope
       DeserializationError error = deserializeJson(doc, result.c_str());
       ^~~~~~~~~~~~~~~~~~~~
something\\Documents\Arduino\libraries\ChatGPT_Client\src/ChatGPT.hpp:32:11: error: 'error' was not declared in this scope
       if (error) {
           ^~~~~
something\\Documents\Arduino\libraries\ChatGPT_Client\src/ChatGPT.hpp:32:11: note: suggested alternative: 'perror'
       if (error) {
           ^~~~~
           Perror
something\\Documents\Arduino\libraries\ChatGPT_Client\src/ChatGPT.hpp:36:30: error: 'doc' was not declared in this scope
       const char* _content = doc["choices"][0]["message"]["content"];
                              ^~~

exit status 1
Compilation error: exit status 1