esp32 picture to base64 to chatGPT_Client.vision_question about chatgpt_client_for_arduino

[0015]

@robomaniac Memory consumption reaches its peak when doing HTTP Posts, including Base64 encoded strings.
Captured image buffer (JPEG) + base64 encoded string + HTTP Post Body... This is why I used QCIF (176x144) as the FrameSize when I created a demo application with ESP32CAM. Try setting FrameSize to minimum.

[robomaniac]

I did try to make smaller images but the issue is how I assemble/construct the base64 URL, how you attach data:image/jpeg;base64, to image taken, merge the 2/ Strycopy does not cut it.

This is json section in python that works. I need to reproduce this json format in C.

payload = {
        "model": "gpt-4o",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "Describe the objects or elements that are in the foreground of this image?"
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": f"data:image/jpeg;base64,{base64_image}"
                        }
                    }
                ]
            }
        ],

[0015]

It worked like this.

const char *base64Prefix = "data:image/jpeg;base64,";

auto inputLength = fb->len;  // Get the length of the framebuffer data
size_t prefixLength = strlen(base64Prefix);

// Calculate the total length for the buffer
size_t totalLength = prefixLength + base64::encodeLength(inputLength);

// Allocate buffer to hold the concatenated string
char *output = new char[totalLength + 1];  // +1 for null terminator

// Copy the prefix into the output buffer
strcpy(output, base64Prefix);

// Encode the framebuffer data into the output buffer after the prefix
base64::encode((const uint8_t *)fb->buf, inputLength, output + prefixLength);

// Ensure the output buffer is null-terminated
output[totalLength] = '\0';

Serial.println(prompt);
if (chatGPT_Client.vision_question("gpt-4o", "user", "text", prompt, "image_url", output, "auto", 200, true, chatGPTresult)) {

  Serial.print("[ChatGPT] Response: ");
  Serial.println(chatGPTresult);
} else {
  Serial.print("[ChatGPT] Error: ");
  Serial.println(chatGPTresult);
}

delete[] output;