tuanpmt / esp-request

This project is no longer supported, please use

Lightweight HTTP client for ESP32

Example

This project is no longer supported, please use

https://github.com/espressif/esp-idf/tree/master/components/esp_http_client

int download_callback(request_t *req, char *data, int len)
{
    req_list_t *found = req->response->header;
    while(found->next != NULL) {
        found = found->next;
        ESP_LOGI(TAG,"Response header %s:%s", (char*)found->key, (char*)found->value);
    }
    //or 
    found = req_list_get_key(req->response->header, "Content-Length");
    if(found) {
        ESP_LOGI(TAG,"Get header %s:%s", (char*)found->key, (char*)found->value);
    }
    ESP_LOGI(TAG,"%s", data);
    return 0;
}
static void request_task(void *pvParameters)
{
    request_t *req;
    int status;
    xEventGroupWaitBits(wifi_event_group, CONNECTED_BIT, false, true, portMAX_DELAY);
    ESP_LOGI(TAG, "Connected to AP, freemem=%d",esp_get_free_heap_size());
    // vTaskDelay(1000/portTICK_RATE_MS);
    req = req_new("http://httpbin.org/post"); 
    //or
    //request *req = req_new("https://google.com"); //for SSL
    req_setopt(req, REQ_SET_METHOD, "POST");
    req_setopt(req, REQ_SET_POSTFIELDS, "test=data&test2=data2");
    req_setopt(req, REQ_FUNC_DOWNLOAD_CB, download_callback);
    status = req_perform(req);
    req_clean(req);
    vTaskDelete(NULL);
}

Websocket

int websocket_cb(request_t *req, int status, void *buffer, int len)
{
    switch(status) {
        case WS_CONNECTED:
            ESP_LOGI(TAG, "websocket connected");
            req_write(req, "hello world", 11);
            break;
        case WS_DATA:
            ((char*)buffer)[len] = 0;
            ESP_LOGI(TAG, "websocket data = %s", (char*)buffer);
            req_close(req);
            break;
        case WS_DISCONNECTED:
            ESP_LOGI(TAG, "websocket disconnected");
            req_clean(req);
            req = NULL;
            break;
    }
    return 0;
}
void app()
{
    request_t *req = req_new("ws://echo.websocket.org"); // or wss://echo.websocket.org
    req_setopt(req, REQ_FUNC_WEBSOCKET, websocket_cb);
    req_perform(req);
}

Usage

• Create ESP-IDF application https://github.com/espressif/esp-idf-template
• Clone git submodule add https://github.com/tuanpmt/esp-request components/esp-request
• Example esp-request application: https://github.com/tuanpmt/esp-request-app
• OTA application using esp-request: https://github.com/tuanpmt/esp32-fota

API

Function

• req_new
• req_setopt
• req_clean

Options for req_setopt

• REQ_SET_METHOD - req_setopt(req, REQ_SET_METHOD, "GET");//or POST/PUT/DELETE
• REQ_SET_HEADER - req_setopt(req, REQ_SET_HEADER, "HeaderKey: HeaderValue");
• REQ_SET_HOST - req_setopt(req, REQ_SET_HOST, "google.com"); //or 192.168.0.1
• REQ_SET_PORT - req_setopt(req, REQ_SET_PORT, "80");//must be string
• REQ_SET_PATH - req_setopt(req, REQ_SET_PATH, "/path");
• REQ_SET_SECURITY
• REQ_SET_URI - req_setopt(req, REQ_SET_URI, "http://uri.com"); //will replace host, port, path, security and Auth if present
• REQ_SET_DATAFIELDS
• REQ_SET_UPLOAD_LEN - Not effect for now
• REQ_FUNC_DOWNLOAD_CB - req_setopt(req, REQ_FUNC_DOWNLOAD_CB, download_callback);
• REQ_FUNC_UPLOAD_CB
• REQ_FUNC_WEBSOCKET
• REQ_REDIRECT_FOLLOW - req_setopt(req, REQ_REDIRECT_FOLLOW, "true"); //or "false"

URI format

• Follow this: https://en.wikipedia.org/wiki/Uniform_Resource_Identifier

Todo

• Support URI parser
• Follow redirect
• Support SSL
• Support Set POST Fields (simple)
• Support Websocket & Websocket Secure
• Support Basic Auth
• Support Upload multipart
• Support Cookie

Known Issues

• Uri parse need more work

Authors

• Tuan PM

esp-request's Issues

HTTP Host header with or without port

I started using this library recently and noticed that the server I tried talking to does not accept (https) requests if the HTTP Host header is set as "hostname:port". The RFC says you can set the header like that and every other server I tried it with accepts it, but not mine.

So my question is, what was the intention behind this feature, as the code explicity sets this here and here?

Every server I tried also works without the port explicitly specified and it seems like other http clients do not send the port.

Would you accept a pull request that implements and option for turning this off?
If yes, how would you like the interface? As a new constant for req_setopt?

ram usage

Hi,
Using the code in the Example (post on server), the free ram size drecrease up to causing a reset after some minutes of work.
How can i solved it?
Thanks

How to get body of the response?

Hi, first of all good job!
How to get body or payload of http response besides status code? Trying to get something out of response inside the request but failed. Where is it? Thanks!

Update after few more testing. It seems response body saved in buffer->data, but not always....
For the same URL, bytes_read/bytes_write counter might have different value after long run. For quite high possibility in my testing (HTTP GET request, and JSON format payload in response body around 200Bytes), response content not completely saved into buffer. Is it caused by HTTP response in multiple TCP packets?

spelling error

Spelling error on line 677 "Error connnect" instead of "Error connect".

Unable to write headers / send HTTPS request

Hi Tuan!

A coworker of mine has already written in about this, and I wanted to create an issue here so that we can all be involved in / kept up to date on the discussion.

We're getting an error trying to send an HTTPS request to an API we're running, with a 2048 bit certificate from LetsEncrypt. The error we're getting is:

E (8691) HTTP_REQ: Error write header
E (8691) HTTP_REQ: Error send request

For the record, you already mentioned in your email reply that you have also seen this behaviour.

Please let us know if there's anything you find on this... thanks!

[improvement] Secure connection method

It would be great if the secure connection method could be set. Currently, the following line of code means that the SSL connection isn't very secure as lots of insecure cipher suites are given (checked using howsmyssl.com). It also can't make requests to various websites due to the TLS version being too low.
req->ctx = SSL_CTX_new(TLSv1_1_client_method());

A function should be added that changes which TLS method is used and allows client certificates to be used instead. To enable the latest TLS version, the following method needs to be passed to SSL_CTX_new
TLSv1_2_client_method
To add a client CA certificate, the following function should be used instead of SSL_CTX_new
SSL_CTX_add_client_CA(SSL_CTX *ctx, X509 *x)

I think it would make sense if the security method was set using req_setopt(req, REQ_SET_SECURITY, void *data), with the data field being one of the following:
"none"
"TLS 1.1"
"TLS 1.2"
"CA CERT"(X509 *x)

Protocol is required error

Hi you have a bug in the req_setopt function when setting REQ_SET_PROTOCOL as passing and enum as argument, will make the if(!req || !data) condition evaluate to false and the function to return not setting the protocol.
In any case also upon the creation of a new request with req_new, seems that the request has not the protocol set and any perform will fail with the protocol error.

func req_process_download in esp_request.c "eats" payload data

If some payload data is received in the same buffer where you receive http headers, that payload is missed.
The way I found to permit this funxtion to work correctly also in this situation is to avoid the check if the buffer is at eof:

line 637 and following:

if(process_header == 0)
        {
            //if(req->buffer->at_eof) {
				printf("***** BBB\n");
                fill_buffer(req);
            //}

cppcheck static code analyser tool detects memory leak

Hi Tuan,

I am using ESP32 dev kit. I am thinking to use esp-request https client library in one of my project for OTA feature.
I found this library very simplistic and useful for me. So before using it for production use, I just tried
executing cppcheck static analyser tool. cppcheck detects memory leak, and also give nice suggestion to reduce scope of the many variables.

Is it possible to optimise esp_request library considering low power mode usability ?
Any points/suggestions welcome.

cppcheck LOG:

$ cppcheck --enable=all esp-request
Checking esp-request/esp_request.c...
[esp-request/esp_request.c:135]: (style) The scope of the variable 'maskKey' can be reduced.
[esp-request/esp_request.c:293]: (style) The scope of the variable 'random_key' can be reduced.
[esp-request/esp_request.c:293]: (style) The scope of the variable 'b64_key' can be reduced.
[esp-request/esp_request.c:500]: (style) The scope of the variable 'bread' can be reduced.
[esp-request/esp_request.c:502]: (style) The scope of the variable 'buffer_free_bytes' can be reduced.
[esp-request/esp_request.c:300]: (error) Memory leak: req
[esp-request/esp_request.c:305]: (error) Memory leak: req
[esp-request/esp_request.c:313]: (error) Memory leak: req
[esp-request/esp_request.c:317]: (error) Memory leak: req
[esp-request/esp_request.c:353]: (error) Memory leak: host_w_port
1/3 files checked 33% done
Checking esp-request/req_list.c...
2/3 files checked 66% done
Checking esp-request/uri_parser.c...
[esp-request/uri_parser.c:51] -> [esp-request/uri_parser.c:52]: (warning) Either the condition '0==puri' is redundant or there is possible null pointer dereference: puri.
3/3 files checked 100% done
[esp-request/uri_parser.c:192]: (style) The function 'parse_uri_info' is never used.
[esp-request/esp_request.c:711]: (style) The function 'req_clean' is never used.
[esp-request/esp_request.c:703]: (style) The function 'req_close' is never used.
[esp-request/req_list.c:178]: (style) The function 'req_list_clear_key' is never used.
[esp-request/req_list.c:163]: (style) The function 'req_list_set_format' is never used.
[esp-request/esp_request.c:291]: (style) The function 'req_new' is never used.
[esp-request/esp_request.c:673]: (style) The function 'req_perform' is never used.
[esp-request/esp_request.c:707]: (style) The function 'req_write' is never used.
(information) Cppcheck cannot find all the include files (use --check-config for details)
$

Change LOGI to LOGd

Excellent lib @tuanpmt, thanks a lot for sharing it.
Should not most of the LOGI be LOGD ?
I am more than happy to prepare a PR if you agree.

websocket dont work;

i add a return statement:

        if(req->valid_websocket) {
            xTaskCreate(req_websocket_task, "req_websocket_task", 2*1024, req, 5, NULL);
            return 0;
        }

and then it works

No compile with newest esp-idf

Due to name collision in nossl_connect, i had to change the name of int socket -> int sock.

Alternately I tried changing the socket constructor to lwip_socket( xxx,xxx,xxx) which also worked.

Reuse of the list library

The files list.c and list.h are conflicting the include of the list library from the esp-idf sdk.
This means that if you use esp-request (wifi) along with blueooth, you'll get tonnes of multiple definition errors.

It's hacky, but I fixed it by renaming every reference to list as tuanslist .

Any chance you can work on this to fix it cleanly ?