Etcd v3 has been released for some days, and we should catch up and support the latest api

I modified his code to v3 version : https://github.com/chinazj/cetcd

In cetcd_send_request(),parser.json wasn't initiated, and it will freed if it's not null.
Normally it’s initiated in callback cetcd_parse_response(), but for failure case( e.g. got 503 response), it also won’t init. So we will free a pointer with garbage data which causes Segmentation.

The fix is:
init json in cetcd_parse_response() or cetcd_send_request()(preferred).

Code:
void *cetcd_send_request(CURL *curl, cetcd_request *req) {
cetcd_response_parser parser;
....
if (parser.json) {
yajl_free(parser.json);
cetcd_array_destroy(&parser.ctx.keystack);
cetcd_array_destroy(&parser.ctx.nodestack);
}

cetcd_parse_response()
{
if (ptr[i] == '\n') {
parser->st = blank_line_st;
if (parser->http_status >= 300 && parser->http_status < 400) { //error code is 503 in our run as below log shown
/this is a redirection, restart the state machine/
parser->st = request_line_start_st;
break;
}
continue;
}
}

Hi:
I use gcc 4.8.5 to compile example but i got this error:

cetcd-test.cpp:15:59: error: invalid conversion from ‘const void_’ to ‘void_’ [-fpermissive]
cetcd_array_append(&addrs, "http://192.168.4.247:2379");

elem is a pointer to pointer, so when do a destroy should also free(elem->[i]) before free(ca->elem).

typedef struct cetcd_array_t {
void **elem;
size_t count;
size_t cap;
} cetcd_array;

I perfer to change code like below.
int cetcd_array_destroy(cetcd_array ca) {
int i=0;
if (ca->elem != NULL && ca->cap != 0) {
++ for(;i< ca->count;i++){
++ if (ca-elem[i] != 0){
++ sdsfree(ca-elem[i]); / with this change, the cetcd_add_watcher must be changed also */
++ }
++ }
free(ca->elem);
ca->elem = NULL;
}
ca->count = 0;
ca->cap = 0;
return 0;
}

API

The documents is too simple now. There are some more details we need

• More about multiple watchers
• Add guides about how to use the C api
• Add guides about how to use the CPP api

cetcd_array.h: void cetcd_array_release();
cetcd_array.c: void cetcd_array_release(cetcd_array *ca)

Does these function not allow to use?

etcdctl is hard to use and misses a interactive interface. Maybe cetcd should build a cetcdctl as an alternative of etcdctl

the example code is like this below. I add a watcher and start async watch. Then stop it, add another watcher and start it again. I found that only the second watcher can receive watch event, the first is invalid.

cetcd_client m_cli;
cetcd_array m_addrs;
cetcd_array m_watchers;

cetcd_client_init(&m_cli, &m_addrs);
cetcd_array_init(&m_watchers, 1);
cetcd_watcher* watcher = cetcd_watcher_create(&m_cli, getKey1(), 0, 1 , 0, handleEvent1, NULL);
cetcd_add_watcher(&m_watchers, watcher);
cetcd_multi_watch_async(&m_cli, &m_watchers);
cetcd_multi_watch_async_stop(&m_cli, m_wid);

cetcd_watcher* watcher = cetcd_watcher_create(&m_cli, getKey2(), 0, 1 , 0, handleEvent2, NULL);
cetcd_add_watcher(&m_watchers, watcher);
cetcd_multi_watch_async(&m_cli, &m_watchers);

at cetcd.c during cetcd_del_watcher code checks if index is valid (i,e if if the watcher was deleted before)
318 if (watcher && index > 0)
but as far as i understood index (which is set from cetcd.317 watcher->array_index) can be in zero position. shouldn't this changed to index >= 0 ?

In example multi_watch.c file, I want to watch a dir for all change event. I try to set recursive =1 when call cetcd_watcher_create to create a watch. But it not work. Is there some way to watch a dir?

In case cetcd_multi_watch_async i sused and at any given point etcd is not reachable anymore, how is application notified ?

From what i saw there is no notification. After etcd is reachable again subscription is renewed but any notification is responced with
Error Code:1000
Error Message:not a json response
Error Cause:HTTP/1.1 200 OK

thank you

shall copy the http_status into err?

if (parser->http_status != 200 && parser->http_status != 201) {
resp->err = calloc(1, sizeof(cetcd_error));
parser->ctx.userdata = resp->err;
parser->json = yajl_alloc(&error_callbacks, 0, &parser->ctx);

Hi Shafreeck,

I found there is an issue withAPI cetcd_set(). If there is a '+' in the value, it will be replaced to a blank.

cetcd_response *cetcd_set(cetcd_client *cli, const char *key, const char *value, uint64_t ttl)

Here is the detailed analysis on this issue and proposed solution.

1) Problem description

For URL, it only allows a-z, A-Z, 0-9, '-', '.', '_' or '~' . For all other characters, it should be converted to their "URL escaped" version (%NN where NN is a two-digit hexadecimal number).
If without conversion, the set operation will either fail or being changed automatically. The following is the experiment(Note: sdcctl is the command that build with cetcd API, the set operation will call cetcd_set()):

1). The "+" will be automatically change to blank:

[root@etcd0-10-135-16-139 sdc]# sdcctl set myKey "plus++plus"
plus  plus
[root@etcd0-10-135-16-139 sdc]# sdcctl get myKey
plus  plus

2). It will fail if value has "%".

[root@etcd0-10-135-16-139 sdc]# sdcctl set myKey "plus%%plus"
Error code 210
Error message: Invalid POST form
Error cause: invalid URL escape "%%p"
[root@etcd0-10-135-16-139 sdc]# sdcctl set myKey "plus%plus"
Error code 210
Error message: Invalid POST form
Error cause: invalid URL escape "%pl"
[root@etcd0-10-135-16-139 sdc]#

2). The proposed solution

One of the easiest solution is to do a conversion by calling curl_easy_escape( CURL * curl , const char * string , int length ).

the following is example code:

cetcd_response *cetcd_set(cetcd_client *cli, const char *key, const char *value, uint64_t ttl) {
    cetcd_request req;
    cetcd_response *resp;
    cetcd_string params;
    char * value_ptr;

    memset(&req, 0, sizeof(cetcd_request));
    req.method = ETCD_HTTP_PUT;
    req.api_type = ETCD_KEYS;
    req.uri = sdscatprintf(sdsempty(), "%s%s", cli->keys_space, key);
    value_ptr = curl_easy_escape(cli->curl, value, strlen(value));  /* convert the value string */
    params = sdscatprintf(sdsempty(), "value=%s", value_ptr); /* use converted value string */
    if (ttl) {
        params = sdscatprintf(params, "&ttl=%lu", ttl);
    }
    req.data = params;
    resp = cetcd_cluster_request(cli, &req);
    sdsfree(req.uri);
    sdsfree(params);
    curl_free(value_ptr); /* free the memory */
    return resp;
}

Besides cetcd_set(),

• cetcd_update()
• cetcd_create()
• cetcd_create_in_order()
• cetcd_cmp_and_swap()
• cetcd_cmp_and_swap_by_index()
  also has the similar issue.

Do you think we should enhance it? if yes, I will make the changes and send a pull request. @shafreeck

Regards
Sam

Here is the testing code:

#include <stdio.h>
#include <stdlib.h>

#include "cetcd.h"

int main()
{
    cetcd_client cli;
    cetcd_response *resp;
    cetcd_array addrs;

    cetcd_array_init(&addrs, 3);
    cetcd_array_append(&addrs, (void *)"http://192.168.4.247:2379");
    cetcd_array_append(&addrs, (void *)"http://192.168.4.247:2381");
    cetcd_array_append(&addrs, (void *)"http://192.168.4.247:2383");

    cetcd_client_init(&cli, &addrs);

    resp = cetcd_get(&cli, "/test/test.conf");
    if(resp->err) {
        printf("error :%d, %s (%s)\n", resp->err->ecode, resp->err->message, resp->err->cause);
    }
    cetcd_response_print(resp);
    cetcd_response_release(resp);

    cetcd_array_destroy(&addrs);
    cetcd_client_destroy(&cli);

    return 0;
}

A segmentation fault happens when one or all of the etcd cluster server down.

after cetcd_multi_watch_async, if I want add a watcher, do I must cetcd_multi_watch_async_stop first?

In example/multi_watch.c, I see the code that initialize count of watchers before watch. But if the watcher and path are dynamic, how to handle it? Do I must cetcd_multi_watch_async_stop first, cetcd_add_watcher and cetcd_multi_watch_async again?

    cetcd_array_init(&watchers, 3);
    cetcd_add_watcher(&watchers, cetcd_watcher_create(&cli, "/test/watch", 0, 1, 1, watch, NULL));
    cetcd_add_watcher(&watchers, cetcd_watcher_create(&cli, "/test/watch", 0, 1, 1, watch, NULL));
    cetcd_add_watcher(&watchers, cetcd_watcher_create(&cli, "/test/watch", 0, 1, 1, watch, NULL));
    wid = cetcd_multi_watch_async(&cli, &watchers);
    while(!quit) {
        usleep(10000);
    }
    cetcd_multi_watch_async_stop(&cli, wid);

hi Shafreeck,

I found an issue with cetcd_set()/cetcd_parse_response(). If the value string is longer than 1024 bytes, the cetcd_parse_response() will report error code 1000.

Problem Description:

If the value string length is longer than the 1024, cetcd will response with error code 1000.
(sdcctl is a executable command line calling cetcd_set() function).

// It's ok for value string length < 1024
[root@Test ~]# sdcctl set /msg "Very long value string 1024 - looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong"
Very long value string 1024 - looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong

// It will report error response if the value string length > 1024
[root@Test ~]# sdcctl set /msg "Very long value string 1025  - loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong"
Error code 1000
Error message: not a json response
Error cause: HTTP/1.1 200 OK

Root cause:

If the value string length is longer than 1024, the curl will receive "HTTP/1.1 100 Continue" and then receive "HTTP/1.1 200 OK". But currently, there is no handling of "HTTP/1.1 100 Continue" in cetcd_parse_response(). It will result error code 1000.
See following experiment when using curl directly, it shows the messages from the ETCD server. There is "100 continue" before "200 OK" if the string length is 2015.

[root@Test ~]# curl http://10.135.16.138:2379/v2/keys/ab -XPUT -d value="Very long value string 1024  - loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong" -vv
* About to connect() to 10.135.16.138 port 2379 (#0)
*   Trying 10.135.16.138...
* Connected to 10.135.16.138 (10.135.16.138) port 2379 (#0)
> PUT /v2/keys/ab HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.135.16.138:2379
> Accept: */*
> Content-Length: 1024
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 1024 out of 1024 bytes
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Etcd-Cluster-Id: 8f872a24ebb263cc
< X-Etcd-Index: 237
< X-Raft-Index: 2256923
< X-Raft-Term: 50
< Date: Mon, 21 Nov 2016 06:15:22 GMT
< Transfer-Encoding: chunked
<
{"action":"set","node":{"key":"/ab","value":"Very long value string 1024  - loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong","modifiedIndex":237,"createdIndex":237},"prevNode":{"key":"/ab","value":"Very long value string 1024  - loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong","modifiedIndex":236,"createdIndex":236}}

[root@Test ~]# curl http://10.135.16.138:2379/v2/keys/ab -XPUT -d value="Very long value string 1025  - looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong" -vv
* About to connect() to 10.135.16.138 port 2379 (#0)
*   Trying 10.135.16.138...
* Connected to 10.135.16.138 (10.135.16.138) port 2379 (#0)
> PUT /v2/keys/ab HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.135.16.138:2379
> Accept: */*
> Content-Length: 1025
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Content-Type: application/json
< X-Etcd-Cluster-Id: 8f872a24ebb263cc
< X-Etcd-Index: 238
< X-Raft-Index: 2256964
< X-Raft-Term: 50
< Date: Mon, 21 Nov 2016 06:15:42 GMT
< Transfer-Encoding: chunked
<
{"action":"set","node":{"key":"/ab","value":"Very long value string 1025  - looooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong","modifiedIndex":238,"createdIndex":238},"prevNode":{"key":"/ab","value":"Very long value string 1025  - loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong","modifiedIndex":237,"createdIndex":237}}

Alternative solution:

According to my debugging, there will also be also a blank line between "HTTP/1.1 100 Continue" and "HTTP/1.1 200 OK", so we could restart the state machine like following:

size_t cetcd_parse_response(char *ptr, size_t size, size_t nmemb, void *userdata) {
        . . .
        if (parser->st == header_key_start_st) {
            if (ptr[i] == '\r') {
                ++i;
            }
            if (ptr[i] == '\n') {
                parser->st = blank_line_st;
                /* restart state machine for 1xx trying and 3xx redirection */
                if (**(parser->http_status >= 100 && parser->http_status < 200)**
                    || (parser->http_status >= 300 && parser->http_status < 400)) {
                    /*this is a redirection, restart the state machine*/
                    parser->st = request_line_start_st;
                    break;
                }
                continue;
            }
            parser->st = header_key_st;
        }
       . . . . . . 

Could you please review and let me know your opinion? if you are ok with this solution, I would be happy to send a you a pull request.

Have a good day!
Sam Zhi
@shafreeck

cc -g -O0 -Wall -I ../third-party/build/include -lcurl -lpthread cetcd_get.c ../cetcd.c ../cetcd_array.c ../sds/sds.c ../third-party/build/yajl.c.o ../third-party/build/yajl_alloc.c.o ../third-party/build/yajl_buf.c.o ../third-party/build/yajl_encode.c.o ../third-party/build/yajl_gen.c.o ../third-party/build/yajl_lex.c.o ../third-party/build/yajl_parser.c.o ../third-party/build/yajl_tree.c.o ../third-party/build/yajl_version.c.o -o cetcd_get
/tmp/ccNw5IP7.o: In function cetcd_client_init': /mnt/hdd1/work/cetcd/examples/../cetcd.c:58: undefined reference tocurl_global_init'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:64: undefined reference to curl_easy_init' /mnt/hdd1/work/cetcd/examples/../cetcd.c:92: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:95: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:96: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:98: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:99: undefined reference tocurl_easy_setopt'
/tmp/ccNw5IP7.o:/mnt/hdd1/work/cetcd/examples/../cetcd.c:100: more undefined references to curl_easy_setopt' follow /tmp/ccNw5IP7.o: In functioncetcd_client_destroy':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:114: undefined reference to curl_easy_cleanup' /mnt/hdd1/work/cetcd/examples/../cetcd.c:115: undefined reference tocurl_global_cleanup'
/tmp/ccNw5IP7.o: In function cetcd_setup_tls': /mnt/hdd1/work/cetcd/examples/../cetcd.c:165: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:168: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:171: undefined reference tocurl_easy_setopt'
/tmp/ccNw5IP7.o: In function cetcd_watcher_create': /mnt/hdd1/work/cetcd/examples/../cetcd.c:189: undefined reference tocurl_easy_init'
/tmp/ccNw5IP7.o: In function cetcd_watcher_release': /mnt/hdd1/work/cetcd/examples/../cetcd.c:206: undefined reference tocurl_easy_cleanup'
/tmp/ccNw5IP7.o: In function cetcd_watcher_reset': /mnt/hdd1/work/cetcd/examples/../cetcd.c:229: undefined reference tocurl_easy_reset'
/tmp/ccNw5IP7.o: In function cetcd_curl_setopt': /mnt/hdd1/work/cetcd/examples/../cetcd.c:267: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:272: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:274: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:276: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:277: undefined reference tocurl_easy_setopt'
/tmp/ccNw5IP7.o:/mnt/hdd1/work/cetcd/examples/../cetcd.c:279: more undefined references to curl_easy_setopt' follow /tmp/ccNw5IP7.o: In functioncetcd_reap_watchers':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:350: undefined reference to curl_easy_getinfo' /mnt/hdd1/work/cetcd/examples/../cetcd.c:360: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:362: undefined reference to curl_multi_remove_handle' /mnt/hdd1/work/cetcd/examples/../cetcd.c:363: undefined reference tocurl_easy_reset'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:365: undefined reference to curl_multi_add_handle' /mnt/hdd1/work/cetcd/examples/../cetcd.c:379: undefined reference tocurl_multi_remove_handle'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:392: undefined reference to curl_multi_remove_handle' /mnt/hdd1/work/cetcd/examples/../cetcd.c:398: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:401: undefined reference to curl_multi_add_handle' /mnt/hdd1/work/cetcd/examples/../cetcd.c:405: undefined reference tocurl_multi_remove_handle'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:347: undefined reference to curl_multi_info_read' /tmp/ccNw5IP7.o: In functioncetcd_multi_watch':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:422: undefined reference to curl_multi_init' /mnt/hdd1/work/cetcd/examples/../cetcd.c:426: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:427: undefined reference to curl_multi_add_handle' /mnt/hdd1/work/cetcd/examples/../cetcd.c:432: undefined reference tocurl_multi_perform'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:438: undefined reference to curl_multi_timeout' /mnt/hdd1/work/cetcd/examples/../cetcd.c:445: undefined reference tocurl_multi_fdset'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:450: undefined reference to curl_multi_perform' /tmp/ccNw5IP7.o: In functioncetcd_multi_watch_async':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:491: undefined reference to pthread_create' /tmp/ccNw5IP7.o: In functioncetcd_multi_watch_async_stop':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:501: undefined reference to pthread_cancel' /mnt/hdd1/work/cetcd/examples/../cetcd.c:502: undefined reference topthread_join'
/tmp/ccNw5IP7.o: In function cetcd_send_request': /mnt/hdd1/work/cetcd/examples/../cetcd.c:1099: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:1100: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:1102: undefined reference tocurl_easy_setopt'
/mnt/hdd1/work/cetcd/examples/../cetcd.c:1110: undefined reference to curl_easy_setopt' /mnt/hdd1/work/cetcd/examples/../cetcd.c:1113: undefined reference tocurl_easy_setopt'
/tmp/ccNw5IP7.o:/mnt/hdd1/work/cetcd/examples/../cetcd.c:1116: more undefined references to curl_easy_setopt' follow /tmp/ccNw5IP7.o: In functioncetcd_send_request':
/mnt/hdd1/work/cetcd/examples/../cetcd.c:1125: undefined reference to curl_easy_perform' /mnt/hdd1/work/cetcd/examples/../cetcd.c:1142: undefined reference tocurl_easy_strerror'
collect2: error: ld returned 1 exit status
make: *** [cetcd_get] Error 1

it appears that when value of the key is large (something larger than 1024 or so) Expect: 100-continue is used from libcurl (if i can recall by defult) and state machine is troubled(response_discard_st).
etcd replies with HTTP/1.1 100 Continue and then HTTP/1.1 200 OK.

error :1000, not a json response (HTTP/1.1 200 OK
)
Error Code:1000
Error Message:not a json response
Error Cause:HTTP/1.1 200 OK

One possible workarround is the disable the Expect: 100-continue in the send request

@@ -1122,9 +1129,15 @@ void *cetcd_send_request(CURL *curl, cetcd_request *req) {
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, cetcd_parse_response);
curl_easy_setopt(curl, CURLOPT_VERBOSE, req->cli->settings.verbose);
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, req->cli->settings.connect_timeout);

• struct curl_slist *chunk = NULL;
• chunk = curl_slist_append(chunk, "Expect:");
• res = curl_easy_setopt(curl, CURLOPT_HTTPHEADER, chunk);

res = curl_easy_perform(curl);

• curl_slist_free_all(chunk);

if that's OK i can send a pull request.

(gdb) thread apply all bt

Thread 3 (Thread 0x7ffff053f700 (LWP 34885)):
#0 0x00007ffff76cff4d in write () from /lib64/libc.so.6
#1 0x00007ffff765e363 in _IO_new_file_write () from /lib64/libc.so.6
#2 0x00007ffff765f7cc in __GI__IO_do_write () from /lib64/libc.so.6
#3 0x00007ffff765ea01 in __GI__IO_file_xsputn () from /lib64/libc.so.6
#4 0x00007ffff7630a4d in vfprintf () from /lib64/libc.so.6
#5 0x00007ffff7639c79 in printf () from /lib64/libc.so.6
#6 0x00007ffff7bcfffb in cetcd_send_request (curl=0x60aa50, req=0x7ffff053ed80) at cetcd.c:1186
#7 0x00007ffff7bd0154 in cetcd_cluster_request (cli=0x6020c0 , req=0x7ffff053ed80) at cetcd.c:1221
#8 0x00007ffff7bce188 in cetcd_lsdir (cli=0x6020c0 , key=0x400e01 "/test_query", sort=0, recursive=0) at cetcd.c:538
#9 0x00007ffff7bce0b8 in cetcd_get (cli=0x6020c0 , key=0x400e01 "/test_query") at cetcd.c:522
#10 0x0000000000400b84 in get_tread2 (ThreadId=0x0) at cetcd_set.c:65
#11 0x00007ffff79b0dc5 in start_thread () from /lib64/libpthread.so.0
#12 0x00007ffff76de71d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7ffff0d40700 (LWP 34884)):
#0 0x00007ffff761d5d7 in raise () from /lib64/libc.so.6
#1 0x00007ffff761ecc8 in abort () from /lib64/libc.so.6
#2 0x00007ffff765d2f7 in __libc_message () from /lib64/libc.so.6
#3 0x00007ffff76646d3 in _int_free () from /lib64/libc.so.6
#4 0x00007ffff7bc9b02 in sdsMakeRoomFor (s=0x7fffe8000983 "lication/json", addlen=1) at sds/sds.c:233
#5 0x00007ffff7bca131 in sdscatlen (s=0x7fffe8000983 "lication/json", t=0x613d20, len=1) at sds/sds.c:389
#6 0x00007ffff7bcf628 in cetcd_parse_response (ptr=0x613d20 "Content-Type: application/json\r\n", size=1, nmemb=32,
userdata=0x7ffff053ec10) at cetcd.c:1003
#7 0x00007ffff737b9c2 in Curl_client_chop_write () from /lib64/libcurl.so.4
#8 0x00007ffff7379d93 in Curl_http_readwrite_headers () from /lib64/libcurl.so.4
#9 0x00007ffff738cbc8 in Curl_readwrite () from /lib64/libcurl.so.4
#10 0x00007ffff7397351 in multi_runsingle () from /lib64/libcurl.so.4
#11 0x00007ffff7397df9 in curl_multi_perform () from /lib64/libcurl.so.4
#12 0x00007ffff738e745 in curl_easy_perform () from /lib64/libcurl.so.4
#13 0x00007ffff7bcfefe in cetcd_send_request (curl=0x60aa50, req=0x7ffff0d3fd80) at cetcd.c:1166
#14 0x00007ffff7bd0154 in cetcd_cluster_request (cli=0x6020c0 , req=0x7ffff0d3fd80) at cetcd.c:1221
#15 0x00007ffff7bce188 in cetcd_lsdir (cli=0x6020c0 , key=0x400e01 "/test_query", sort=0, recursive=0) at cetcd.c:538
#16 0x00007ffff7bce0b8 in cetcd_get (cli=0x6020c0 , key=0x400e01 "/test_query") at cetcd.c:522
#17 0x0000000000400b11 in get_tread1 (ThreadId=0x0) at cetcd_set.c:39
#18 0x00007ffff79b0dc5 in start_thread () from /lib64/libpthread.so.0
#19 0x00007ffff76de71d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7ffff7fe0840 (LWP 34867)):
#0 0x00007ffff76a596d in nanosleep () from /lib64/libc.so.6
#1 0x00007ffff76a5804 in sleep () from /lib64/libc.so.6
#2 0x0000000000400d32 in main (argc=1, argv=0x7fffffffe618) at cetcd_set.c:130
(gdb)

test code below:

#include "../cetcd.h"
#include <unistd.h>

pthread_mutex_t unTimerLock; /**/

#define ETCD_TIMER_LOCK() //pthread_mutex_lock(&unTimerLock)
#define ETCD_TIMER_UNLOCK() //pthread_mutex_unlock(&unTimerLock)

cetcd_client cli;

void *get_tread1(int *ThreadId)
{
cetcd_response *resp;
while(1)
{
ETCD_TIMER_LOCK();
resp = cetcd_get(&cli, "/test_query");
ETCD_TIMER_UNLOCK();
if (resp->err)
{
printf("get_tread1 cetcd_get error :%d, %s (%s)\n", resp->err->ecode, resp->err->message, resp->err->cause);
}
else
{
//printf("get_tread1 cetcd_tty OK:%s\n", resp->node->value);
}
cetcd_response_release(resp);

}
return 0;

}

void *get_tread2(int *ThreadId)
{
cetcd_response *resp;
sleep(2);

while(1)
{
    ETCD_TIMER_LOCK();
    resp = cetcd_get(&cli, "/test_query");
    ETCD_TIMER_UNLOCK();
    if (resp->err)
    {
        printf("get_tread2 cetcd_get error :%d, %s (%s)\n", resp->err->ecode, resp->err->message, resp->err->cause);
    }
    else
    {
        //printf("get_tread2 cetcd_tty OK:%s\n", resp->node->value);
    }
    cetcd_response_release(resp);
    
}
return 0;

}

int main(int argc, char *argv[]) {
cetcd_response *resp;
cetcd_array addrs;
pthread_t thread1;
pthread_t thread2;
pthread_mutex_init(&unTimerLock, 0);
int ThreadId1 = 1;
int ThreadId2 = 1;

cetcd_array_init(&addrs, 1);
cetcd_array_append(&addrs, "http://171.19.0.1:2379");

cetcd_client_init(&cli, &addrs);
printf("connnect to http://171.19.0.1:2379");

resp = cetcd_create(&cli, "/test_tty", "hello cetcd", 0);
if(resp->err) {
    printf("error :%d, %s (%s)\n", resp->err->ecode, resp->err->message, resp->err->cause);
}

resp = cetcd_create(&cli, "/test_query", "hello cetcd query", 0);
if(resp->err) {
    printf("error :%d, %s (%s)\n", resp->err->ecode, resp->err->message, resp->err->cause);
}

pthread_create(&thread1, NULL, (void *(*)(void *))get_tread1, 0);

pthread_create(&thread2, NULL, (void *(*)(void *))get_tread2, 0);

while(1)
{           
    sleep(1);
}

cetcd_array_destroy(&addrs);
cetcd_client_destroy(&cli);
return 0;

}

Do we support ssl, tls or https, or others?

We can just add/remove watchers before calling cetcd_multi_watch* so far. It is not flexible enough. What I expect is add or remove watchers at any time including after starting watching.

the recent hireds is using sds 2.0 ,while cetcd use sds 1.x. it cause the problem when they combine together in one project.

Add etcd auth support

Does cetcd support watch key synchronously? eg: watch the key and don't return until the key is changed, or reach timeout, or stop flag is being set, or network error.

Situation: we try to receive keys from etcd directory by cetcd_lsdir(...), where in directory more than ~200 keys... more specifically, we try to receive some curl data with size > 16KB.
Problem: we can`t parce full response of data more then 16 KB, so more than
~200-220 keys.

We try to write key-value pair but the value is binary, contain \0. It can not store all value into etcd.

Do you plan add support for CAS and ability to specify prev values for keys/ttl and other stuff?

Hi,

The cetcd has two APIs to writ the key/value, cetcd_update and cetcd_set, is it better to add the prevExist in the etcd_set? Then we only use cetcd_set to create a key, and use cetcd_update to update a key.

cetcd_response *cetcd_set(cetcd_client *cli, const char *key,
const char *value, uint64_t ttl) {
cetcd_request req;
cetcd_response *resp;
cetcd_string params;

memset(&req, 0, sizeof(cetcd_request));
req.method = ETCD_HTTP_PUT;
req.api_type = ETCD_KEYS;
req.uri = sdscatprintf(sdsempty(), "%s%s", cli->keys_space, key);
params = sdscatprintf(sdsempty(), "value=%s", value);
params = sdscatprintf(sdsempty(), "prevExist=false");
if (ttl) {
    params = sdscatprintf(params, "&ttl=%lu", ttl);
}
req.data = params;
resp = cetcd_cluster_request(cli, &req);
sdsfree(req.uri);
sdsfree(params);
return resp;

}

Connect etcd with HTTPS