xmidt-org / parodus

The XMiDT client.

parodus

C implementation of the XMiDT client coordinator

XMiDT is a highly scalable, highly available, generic message routing system.

Building and Testing Instructions

mkdir build
cd build
cmake ..
make
make test

Command line Arguments needed to start parodus

- /hw-model -The hardware model name.

- /hw-serial-number -The serial number.

- /hw-manufacturer -The device manufacturer.

- /hw-mac -The MAC address used to manage the device.

- /hw-last-reboot-reason -The last known reboot reason.

- /fw-name -The firmware name.

- /boot-time -The boot time in unix time.

- /webpa-url -The URL that the WRP client should reach out to. (required). Must begin with 'https://' or 'http://'.  May end with a port number. If no port specified, then 443 is assumed for https, 80 for http.

- /webpa-backoff-max -The maximum value of c in the binary backoff algorithm

- /webpa-ping-timeout -The maximum time to wait between pings before assuming the connection is broken.

- /webpa-interface-used -The device interface being used to connect to the cloud.

- /parodus-local-url -Parodus local server url (optional argument)

- /partner-id -  Partner ID of broadband gateway (optional argument)

- /ssl-cert-path -Provide the certs for establishing secure connection (optional argument)

- /force-ipv4 -Forcefully connect parodus to ipv4 address (optional argument)

- /force-ipv6 -Forcefully connect parodus to ipv6 address (optional argument)

- /client-cert-path -MTLS client cert location to request auth token for establishing secure connection [absolute path where client cert is present] (optional argument)

- /token-server-url -complete server url with API path to request new auth token for establishing secure connection (optional argument)

- /crud-config-file -Config json file to store objects during create, retrieve, update and delete (CRUD) operations -optional argument 


# if ENABLE_SESHAT is enabled
- /seshat-url - The seshat server url 

# if FEATURE_DNS_QUERY is enabled then below mentioned arguments are needed

- /acquire-jwt - this parameter (0 or 1) specifies whether there will be a dns lookup. If not, or if any problem occurs with the dns lookup, then webpa-url will be the target. 

- /dns-txt-url  - this parameter is used along with the hw_mac parameter to create the dns txt record id

- /jwt-algo -Allowed algorithm used for communication

- /jwt-public-key-file -JWT token validation key

# if ENABLE_MUTUAL_AUTH is enabled

- /mtls-client-cert-path - Provide the client cert for establishing a mutual auth secure websocket connection

- /mtls-client-key-path - Provide the client cert key for establishing a mutual auth secure websocket connection

Sample parodus start commands:

# Seshat & FEATURE_DNS_QUERY Enabled

./parodus --hw-model=TGXXX --hw-serial-number=E8GBUEXXXXXXXXX --hw-manufacturer=ARRIS --hw-mac=14cfexxxxxxx --hw-last-reboot-reason=unknown --fw-name=TG1682_DEV_master_20170512115046sdy --boot-time=1494590301 --webpa-ping-timeout=180 --webpa-interface-used=eth0 --webpa-url=https://somebody.net:8080 --webpa-backoff-max=9 --parodus-local-url=tcp://127.0.0.1:6666 --partner-id=comcast --ssl-cert-path=/etc/ssl/certs/ca-certificates.crt --acquire-jwt=1 --dns-txt-url=somebody.net --jwt-public-key-file=webpa-rs256.pem --jwt-algo=RS256 --seshat-url=tcp://127.0.0.1:7777 --client-cert-path=/tmp/clientcert.mch --token-server-url=https://somebody.net:8080/token --force-ipv4 --crud-config-file=/tmp/parodus_cfg.json


# Seshat is not enabled

./parodus --hw-model=TGXXX --hw-serial-number=E8GBUEXXXXXXXXX --hw-manufacturer=ARRIS --hw-mac=14cfexxxxxxx --hw-last-reboot-reason=unknown --fw-name=TG1682_DEV_master_20170512115046sdy --boot-time=1494590301 --webpa-ping-timeout=180 --webpa-interface-used=eth0 --webpa-url=https://somebody.net:8080 --webpa-backoff-max=9 --parodus-local-url=tcp://127.0.0.1:6666 --partner-id=comcast --ssl-cert-path=/etc/ssl/certs/ca-certificates.crt --acquire-jwt=1 --dns-txt-url=somebody.net --jwt-public-key-file=webpa-rs256.pem --jwt-algo=RS256 --client-cert-path=/tmp/clientcert.mch --token-server-url=https://somebody.net:8080/token --force-ipv4 --crud-config-file=/tmp/parodus_cfg.json


# When both Seshat & FEATURE_DNS_QUERY not Enabled

./parodus --hw-model=TGXXX --hw-serial-number=E8GBUEXXXXXXXXX --hw-manufacturer=ARRIS --hw-mac=14cfexxxxxxx --hw-last-reboot-reason=unknown --fw-name=TG1682_DEV_master_20170512115046sdy --boot-time=1494590301 --webpa-ping-timeout=180 --webpa-interface-used=eth0 --webpa-url=https://somebody.net:8080 --webpa-backoff-max=9 --parodus-local-url=tcp://127.0.0.1:6666 --partner-id=comcast --ssl-cert-path=/etc/ssl/certs/ca-certificates.crt --client-cert-path=/tmp/clientcert.mch --token-server-url=https://somebody.net:8080/token --force-ipv4 --crud-config-file=/tmp/parodus_cfg.json

parodus's Issues

about the problem that nopoll_conn_wait_for_status_until_connection_ready not found in the open source nopoll.

dear,
the function "nopoll_conn_wait_for_status_until_connection_ready" can not be found in the formal release version nopoll-1.0.0.tar.gz. only can be found in the test unit code.
is the original function not be merged with parodus?

OpenWrt package ?

is there possibly an OpenWrt package? and if so how would that interact with the device, i dont see where parodus speaks "uci"

Allow a partner-id list.

If there is a * in the partner-id field it should be treated as a wild card and taken as a match.
[Update] We have determined that doing a literal match on the character * is simpler and a bit safer. We are going this route.

cjwt build failure is deprecated: Since OpenSSL 3.0

consolidate compiler generated dependencies of target cjwt
[ 85%] Building C object src/CMakeFiles/cjwt.dir/cjwt.c.o
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c: In function ‘cjwt_create_rsa’:
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:206:9: error: ‘PEM_read_bio_RSA_PUBKEY’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
206 | rsa = PEM_read_bio_RSA_PUBKEY( keybio, &rsa, NULL, NULL );
| ^~~
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:31:
/usr/include/openssl/pem.h:449:1: note: declared here
449 | DECLARE_PEM_rw_attr(OSSL_DEPRECATEDIN_3_0, RSA_PUBKEY, RSA)
| ^~~~~~~~~~~~~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:208:9: error: ‘PEM_read_bio_RSAPrivateKey’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
208 | rsa = PEM_read_bio_RSAPrivateKey( keybio, &rsa, NULL, NULL );
| ^~~
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:31:
/usr/include/openssl/pem.h:447:1: note: declared here
447 | DECLARE_PEM_rw_cb_attr(OSSL_DEPRECATEDIN_3_0, RSAPrivateKey, RSA)
| ^~~~~~~~~~~~~~~~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c: In function ‘cjwt_verify_rsa’:
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:248:9: error: ‘RSA_free’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
248 | RSA_free( rsa );
| ^~~~~~~~
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:30:
/usr/include/openssl/rsa.h:293:28: note: declared here
293 | OSSL_DEPRECATEDIN_3_0 void RSA_free(RSA *r);
| ^~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:272:19: error: ‘RSA_verify’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
272 | ( NID_sha256, digest, SHA256_DIGEST_LENGTH, decoded_sig,
| ^
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:30:
/usr/include/openssl/rsa.h:351:27: note: declared here
351 | OSSL_DEPRECATEDIN_3_0 int RSA_verify(int type, const unsigned char *m,
| ^~~~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:278:19: error: ‘RSA_verify’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
278 | ( NID_sha384, digest, SHA384_DIGEST_LENGTH, decoded_sig,
| ^
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:30:
/usr/include/openssl/rsa.h:351:27: note: declared here
351 | OSSL_DEPRECATEDIN_3_0 int RSA_verify(int type, const unsigned char *m,
| ^~~~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:284:19: error: ‘RSA_verify’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
284 | ( NID_sha512, digest, SHA512_DIGEST_LENGTH, decoded_sig,
| ^
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:30:
/usr/include/openssl/rsa.h:351:27: note: declared here
351 | OSSL_DEPRECATEDIN_3_0 int RSA_verify(int type, const unsigned char *m,
| ^~~~~~~~~~
/home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:294:5: error: ‘RSA_free’ is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
294 | RSA_free( rsa );
| ^~~~~~~~
In file included from /home/dingo/parodus/build/_prefix/cjwt/src/cjwt/src/cjwt.c:30:
/usr/include/openssl/rsa.h:293:28: note: declared here
293 | OSSL_DEPRECATEDIN_3_0 void RSA_free(RSA *r);
| ^~~~~~~~
cc1: all warnings being treated as errors
make[5]: *** [src/CMakeFiles/cjwt.dir/build.make:76: src/CMakeFiles/cjwt.dir/cjwt.c.o] Error 1
make[4]: *** [CMakeFiles/Makefile2:156: src/CMakeFiles/cjwt.dir/all] Error 2
make[3]: *** [Makefile:136: all] Error 2
make[2]: *** [CMakeFiles/cjwt.dir/build.make:86: _prefix/cjwt/src/cjwt-stamp/cjwt-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:1156: CMakeFiles/cjwt.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
❯ openssl version
OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)

parse_webpa_url function doesn't correctly return server address if it doesn't have port number in the URL

If the redirect from Petasos returns an URL that has a default port, such as http (port 80) or https (port 443) with an URI, parse_webpa_url function doesn't correctly return the server address.

Pull request #195

webpa-url with IPv6 address as host does not work

When IPv6 address is passed like --webpa-url="http://[2001:558:fc18:2:f816:3eff:fe7f:6efa]:8080"
then due to ":" parsing in url this address will not work even if valid.
This IPv6 address parsing in URL needs to be fixed.

Retry Auth token fetch on every reconnect to server

Currently Auth token is fetched on the startup and whenever 403 is received.
As part of #301 the expired token in cache is cleared.
So re-attempt to fetch the auth token on next reconnect to server is necessary, whenever the cached token is empty. This is to ensure that every connect to server has a token and reduce cases where connect is made without token.

Remove "master" from test variable contents

parodus/tests/simple_connection.c

Line 50 in 880e4a8

headerValues[2] = "WebPA-1.6 (TG1682_DEV_master_2016000000sdy;TG1682/ARRISGroup,Inc.;)";

parodus/tests/test_config.c

Line 174 in 4085c56

"--fw-name=TG1682_DEV_master_2016000000sdy",

parodus/tests/test_crud_internal.c

Line 142 in f44fc2c

parStrncpy(cfg->fw_name, "TG1682_DEV_master_2016000000sdy", sizeof(cfg->fw_name));

On redirected server URL connect failure with 5xx re-attempt connection with default server URL

Currently when redirected server URL connect returns 5xx then connection is retried with same redirected server URL instead of re-fetching the server URL from default.
Because in case if redirected server is at full capacity then we need to fetch another redirect server by reaching to load balancers to get a good redirected server URL.

Use exponential backoff as seed vs. absolute time

At large scales, we see large spikes at the exponential back-off retry times. Ideally, we'd use the exponential back-off time calculated as a max seed to a random time generation.

Example:

n=5
2^n-1 = 2^5-1 = 31

-- here is the new part --
randomize from 0 to 31 inclusive. [0, 31]
-- end new part --

use the random value as the delay before retrying.

parodus client setup for webpa server getting TLS negation error

We are getting a TLS negotiation error from parodus when it's trying to connect to a server & need some support on resolving this.

Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.508328 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: ../../git/src/nopoll_conn.c:817 CERTIFICATE: subject: /CN=10.47.1.250
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.508381 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: ../../git/src/nopoll_conn.c:819 CERTIFICATE: error 19:self signed certificate in certificate chain
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.508642 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: ../../git/src/nopoll_conn.c:1105 there was an error with the TLS negotiation, ssl error (code:1) : error:00000001:lib(0):func(0):reason(1)
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.508719 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: ../../git/src/nopoll_conn.c:535 tls stack: err=337047686, error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed (find reason(code) at openssl/ssl.h)
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.508770 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: ../../git/src/nopoll_conn.c:556 details, run: openssl errstr 1416F086
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.509302 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: Error connecting to server
Dec 10 06:33:58 raspberrypi-rdk-mc "parodus"[11716]: 211210-06:33:58.509347 [mod=PARODUS, lvl=ERROR] [tid=11716] PARODUS: RDK-10037 - WebPA Connection Lost

Parodus compile error in test_token when Seshat is enabled

depending on ENABLE_SESHAT flag in CMakeList, seshat_interface.c is used and seshat_interface_stub.c is used.
currently, i have a compile error.
Scanning dependencies of target test_token
[ 41%] Building C object tests/CMakeFiles/test_token.dir/test_token.c.o
[ 42%] Building C object tests/CMakeFiles/test_token.dir//src/conn_interface.c.o
[ 42%] Building C object tests/CMakeFiles/test_token.dir//src/config.c.o
[ 42%] Building C object tests/CMakeFiles/test_token.dir//src/connection.c.o
[ 43%] Building C object tests/CMakeFiles/test_token.dir//src/spin_thread.c.o
[ 43%] Building C object tests/CMakeFiles/test_token.dir//src/service_alive.c.o
[ 44%] Building C object tests/CMakeFiles/test_token.dir//src/client_list.c.o
[ 44%] Building C object tests/CMakeFiles/test_token.dir//src/nopoll_handlers.c.o
[ 44%] Building C object tests/CMakeFiles/test_token.dir//src/nopoll_helpers.c.o
[ 45%] Building C object tests/CMakeFiles/test_token.dir//src/partners_check.c.o
[ 45%] Building C object tests/CMakeFiles/test_token.dir//src/ParodusInternal.c.o
[ 46%] Building C object tests/CMakeFiles/test_token.dir//src/upstream.c.o
[ 46%] Building C object tests/CMakeFiles/test_token.dir//src/downstream.c.o
[ 47%] Building C object tests/CMakeFiles/test_token.dir//src/networking.c.o
[ 47%] Building C object tests/CMakeFiles/test_token.dir//src/thread_tasks.c.o
[ 47%] Building C object tests/CMakeFiles/test_token.dir//src/time.c.o
[ 48%] Building C object tests/CMakeFiles/test_token.dir//src/string_helpers.c.o
[ 48%] Building C object tests/CMakeFiles/test_token.dir//src/mutex.c.o
[ 49%] Building C object tests/CMakeFiles/test_token.dir//src/seshat_interface.c.o
/stb/parodus/src/seshat_interface.c:27:23: fatal error: libseshat.h: No such file or directory

Add hw-last-reboot-reason-detailed

Add a command line option to accept a unique and detailed reboot reason and include it as metadata in the WRP messages.

The string shall be limited to 64 bytes by parodus.
The string shall have the following regex run on it s/[ ;!&?\\]/_/g to help prevent command injection mistakes.

--hw-last-reboot-reason-detailed - string - A detailed and possibly unique reboot reason.

The --hw-last-reboot-reason is still used but is focused on a more generic "high level" reason.

Why WEBPA protocol is more efficient than TR-69 or SNMP？

Hey, the idea behind Parodus is very inspiring. However, I still feel confused in some ways. Could you explain it further why "WEBPA protocol provides functionality of read/write access to device management parameters in an efficient manner as compared to TR-69 or SNMP" ? Thanks!

Sending this message seems to crash Parodus

I send this message via libparodus & there is a double free.

https://github.com/Comcast/parodus/blob/master/src/upstream.c#L424

JSON Version

{
  "msg_type": 6,
  "source": "mac:112233445500/fs/full/home/schmidtw/workspace/xmidt/parodus2fs/build/src/parodus2fs",
  "dest": "dns:weston",
  "transaction_uuid": "bd4ad2d1-5c9c-486f-8e25-52c242b38f71",
  "status": 400,
  "rdr": 0,
  "metadata": {
    "hw-model": "fs-testing",
    "hw-serial-number": "123456",
    "hw-manufacturer": "Wespa",
    "hw-mac": "112233445500",
    "hw-last-reboot-reason": "",
    "fw-name": "linux",
    "boot-time": "1",
    "webpa-last-reconnect-reason": "webpa_process_starts",
    "webpa-protocol": "PARODUS-2.0-1.0.1-25-g2c3b8fc",
    "webpa-uuid": "1234567-345456546",
    "webpa-inteface-used": "enp57s0u2u3",
    "partner-id": ""
  }
}

Msgpack Version

  0x73, 0x6f, 0x75, 0x72, 0x63, 0x65, 0xd9, 0x56, 0x6d, 0x61, 0x63, 0x3a,
  0x31, 0x31, 0x32, 0x32, 0x33, 0x33, 0x34, 0x34, 0x35, 0x35, 0x30, 0x30,
  0x2f, 0x66, 0x73, 0x2f, 0x66, 0x75, 0x6c, 0x6c, 0x2f, 0x68, 0x6f, 0x6d,
  0x65, 0x2f, 0x73, 0x63, 0x68, 0x6d, 0x69, 0x64, 0x74, 0x77, 0x2f, 0x77,
  0x6f, 0x72, 0x6b, 0x73, 0x70, 0x61, 0x63, 0x65, 0x2f, 0x78, 0x6d, 0x69,
  0x64, 0x74, 0x2f, 0x70, 0x61, 0x72, 0x6f, 0x64, 0x75, 0x73, 0x32, 0x66,
  0x73, 0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x2f, 0x73, 0x72, 0x63, 0x2f,
  0x70, 0x61, 0x72, 0x6f, 0x64, 0x75, 0x73, 0x32, 0x66, 0x73, 0xa4, 0x64,
  0x65, 0x73, 0x74, 0xaa, 0x64, 0x6e, 0x73, 0x3a, 0x77, 0x65, 0x73, 0x74,
  0x6f, 0x6e, 0xb0, 0x74, 0x72, 0x61, 0x6e, 0x73, 0x61, 0x63, 0x74, 0x69,
  0x6f, 0x6e, 0x5f, 0x75, 0x75, 0x69, 0x64, 0xd9, 0x24, 0x62, 0x64, 0x34,
  0x61, 0x64, 0x32, 0x64, 0x31, 0x2d, 0x35, 0x63, 0x39, 0x63, 0x2d, 0x34,
  0x38, 0x36, 0x66, 0x2d, 0x38, 0x65, 0x32, 0x35, 0x2d, 0x35, 0x32, 0x63,
  0x32, 0x34, 0x32, 0x62, 0x33, 0x38, 0x66, 0x37, 0x31, 0xa6, 0x73, 0x74,
  0x61, 0x74, 0x75, 0x73, 0xcd, 0x01, 0x90, 0xa3, 0x72, 0x64, 0x72, 0x00,
  0xa8, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x8c, 0xa8, 0x68,
  0x77, 0x2d, 0x6d, 0x6f, 0x64, 0x65, 0x6c, 0xaa, 0x66, 0x73, 0x2d, 0x74,
  0x65, 0x73, 0x74, 0x69, 0x6e, 0x67, 0xb0, 0x68, 0x77, 0x2d, 0x73, 0x65,
  0x72, 0x69, 0x61, 0x6c, 0x2d, 0x6e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0xa6,
  0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0xaf, 0x68, 0x77, 0x2d, 0x6d, 0x61,
  0x6e, 0x75, 0x66, 0x61, 0x63, 0x74, 0x75, 0x72, 0x65, 0x72, 0xa5, 0x57,
  0x65, 0x73, 0x70, 0x61, 0xa6, 0x68, 0x77, 0x2d, 0x6d, 0x61, 0x63, 0xac,
  0x31, 0x31, 0x32, 0x32, 0x33, 0x33, 0x34, 0x34, 0x35, 0x35, 0x30, 0x30,
  0xb5, 0x68, 0x77, 0x2d, 0x6c, 0x61, 0x73, 0x74, 0x2d, 0x72, 0x65, 0x62,
  0x6f, 0x6f, 0x74, 0x2d, 0x72, 0x65, 0x61, 0x73, 0x6f, 0x6e, 0xa0, 0xa7,
  0x66, 0x77, 0x2d, 0x6e, 0x61, 0x6d, 0x65, 0xa5, 0x6c, 0x69, 0x6e, 0x75,
  0x78, 0xa9, 0x62, 0x6f, 0x6f, 0x74, 0x2d, 0x74, 0x69, 0x6d, 0x65, 0xa1,
  0x31, 0xbb, 0x77, 0x65, 0x62, 0x70, 0x61, 0x2d, 0x6c, 0x61, 0x73, 0x74,
  0x2d, 0x72, 0x65, 0x63, 0x6f, 0x6e, 0x6e, 0x65, 0x63, 0x74, 0x2d, 0x72,
  0x65, 0x61, 0x73, 0x6f, 0x6e, 0xb4, 0x77, 0x65, 0x62, 0x70, 0x61, 0x5f,
  0x70, 0x72, 0x6f, 0x63, 0x65, 0x73, 0x73, 0x5f, 0x73, 0x74, 0x61, 0x72,
  0x74, 0x73, 0xae, 0x77, 0x65, 0x62, 0x70, 0x61, 0x2d, 0x70, 0x72, 0x6f,
  0x74, 0x6f, 0x63, 0x6f, 0x6c, 0xbd, 0x50, 0x41, 0x52, 0x4f, 0x44, 0x55,
  0x53, 0x2d, 0x32, 0x2e, 0x30, 0x2d, 0x31, 0x2e, 0x30, 0x2e, 0x31, 0x2d,
  0x32, 0x35, 0x2d, 0x67, 0x32, 0x63, 0x33, 0x62, 0x38, 0x66, 0x63, 0xaa,
  0x77, 0x65, 0x62, 0x70, 0x61, 0x2d, 0x75, 0x75, 0x69, 0x64, 0xb1, 0x31,
  0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x2d, 0x33, 0x34, 0x35, 0x34, 0x35,
  0x36, 0x35, 0x34, 0x36, 0xb3, 0x77, 0x65, 0x62, 0x70, 0x61, 0x2d, 0x69,
  0x6e, 0x74, 0x65, 0x66, 0x61, 0x63, 0x65, 0x2d, 0x75, 0x73, 0x65, 0x64,
  0xab, 0x65, 0x6e, 0x70, 0x35, 0x37, 0x73, 0x30, 0x75, 0x32, 0x75, 0x33,
  0xaa, 0x70, 0x61, 0x72, 0x74, 0x6e, 0x65, 0x72, 0x2d, 0x69, 0x64, 0xa0,
  0x0d, 0x0a```

Report failure on non-200 response code from auth token server

Behaviors

Current: parodus uses the response body from the auth token server even when the server responded with a non-200 status code.

Expected/Recommended: parodus should only use the response body as as an auth token when the server responded with a 200 (successful) code.

Link to relevant code: https://github.com/xmidt-org/parodus/blob/master/src/auth_token.c#L140

It would look something like

if(response_code == 200) {
    ParodusInfo("Received successful response from auth token server\n");
} else {
    ParodusError("Received failed response from auth token server\n"); 
    //perhaps log the code and the response body 

    return -1 // failure
}

...
return 0 // success

Change webpa-url command line argument format to <http/https>://<server_dns_name>:<port>

Add secure flag to parodus as command line argument to switch between http or https

Add additional CRUD fields

We should add a couple of useful items to the CRUD interface:

• /cloud-status - string Read only field. Results inonline or offline
• /cloud-disconnect - string Update only field. Character string of [a-zA-Z0-9 ]*. If a value is set, the action causes Parodus to disconnect from XMiDT cloud for 5 minutes and send back the string set in the call as the reason for disconnecting. Disconnection payload will contain "disconnection-reason": "field value from cloud-disconnect". During the time when Parodus is disconnected, all other operations take place normally. After 5 minutes, Parodus clears this field and reconnects to the cloud.

Fix cmake error observed while upgrade to nanomsg 1.1.5

https://travis-ci.org/Comcast/parodus/builds/523670805?utm_source=github_status&utm_medium=notification

https://api.travis-ci.org/v3/job/523670806/log.txt

[ 16%] [34m[1mCreating directories for 'nanomsg'[0m
[ 17%] [34m[1mPerforming download step (git clone) for 'nanomsg'[0m
Cloning into 'nanomsg'...
Note: checking out '1.1.5'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at 1749fd7b... Bump version to 1.1.5.
[ 17%] [34m[1mNo patch step for 'nanomsg'[0m
[ 17%] [34m[1mPerforming update step for 'nanomsg'[0m
[ 18%] [34m[1mPerforming configure step for 'nanomsg'[0m
-- The C compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Detected nanomsg ABI v5 (v5.1.0)
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- OS System is Linux
-- OS Version is 4.4.0-101-generic
-- Looking for gethrtime
-- Looking for gethrtime - not found
-- Looking for socketpair
-- Looking for socketpair - found
-- Looking for eventfd
-- Looking for eventfd - found
-- Looking for pipe
-- Looking for pipe - found
-- Looking for pipe2
-- Looking for pipe2 - found
-- Looking for accept4
-- Looking for accept4 - found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for kqueue
-- Looking for kqueue - not found
-- Looking for poll
-- Looking for poll - found
-- Looking for getaddrinfo_a in anl
-- Looking for getaddrinfo_a in anl - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for sem_wait in rt
-- Looking for sem_wait in rt - not found
-- Looking for sem_wait in pthread
-- Looking for sem_wait in pthread - found
-- Looking for gethostbyname in nsl
-- Looking for gethostbyname in nsl - found
-- Looking for socket in socket
-- Looking for socket in socket - not found
-- Looking for CLOCK_MONOTONIC
-- Looking for CLOCK_MONOTONIC - found
-- Looking for atomic_cas_32
-- Looking for atomic_cas_32 - not found
-- Looking for AF_UNIX
-- Looking for AF_UNIX - found
-- Looking for backtrace_symbols_fd
-- Looking for backtrace_symbols_fd - found
-- Performing Test NN_HAVE_MSG_CONTROL
-- Performing Test NN_HAVE_MSG_CONTROL - Success
-- Performing Test NN_HAVE_GCC_ATOMIC_BUILTINS
-- Performing Test NN_HAVE_GCC_ATOMIC_BUILTINS - Success
CMake Error at /usr/local/cmake-3.9.2/share/cmake-3.9/Modules/WriteBasicConfigVersionFile.cmake:40 (message):
No VERSION specified for WRITE_BASIC_CONFIG_VERSION_FILE()
Call Stack (most recent call first):
/usr/local/cmake-3.9.2/share/cmake-3.9/Modules/CMakePackageConfigHelpers.cmake:207 (write_basic_config_version_file)
src/CMakeLists.txt:390 (write_basic_package_version_file)

-- Configuring incomplete, errors occurred!
See also "/home/travis/build/Comcast/parodus/build/_prefix/nanomsg/src/nanomsg-build/CMakeFiles/CMakeOutput.log".
See also "/home/travis/build/Comcast/parodus/build/_prefix/nanomsg/src/nanomsg-build/CMakeFiles/CMakeError.log".
make[2]: *** [_prefix/nanomsg/src/nanomsg-stamp/nanomsg-configure] Error 1
make[1]: *** [CMakeFiles/nanomsg.dir/all] Error 2
make: *** [all] Error 2
travis_time:end:2f525be6:start=1556051667057267812,finish=1556051695577297624,duration=28520029812
[0K[31;1mThe command "make" exited with 2.[0m

travis_time:start:05cbed41
[0K$ export ARGS="-V"
travis_time:end:05cbed41:start=1556051695581647949,finish=1556051695584342372,duration=2694423
[0K[32;1mThe command "export ARGS="-V"" exited with 0.[0m

travis_time:start:052a75de

Parodus Support of QOS

Parodus will behave as follows:

A command line option for configuring the maximum number of messages queued shall be added. (A default value of 0 means do not queue, less than 0 is illegal.)

Low priority messages

Outgoing message queuing will occur until:

• a delivery attempt is made
• 15 minutes has elapsed since the message was enqueued
• queue memory is exhausted, then qos value shall be used to determine and keep the most valuable messages

Delivery Attempts:

• Upon a cloud connection establishment.

Medium priority messages

Outgoing message queuing will occur until:

• a delivery attempt is made & an ACK has been received from the cloud
• 20 minutes has elapsed
• queue memory is exhausted, then qos value shall be used to determine and keep the most valuable messages

Delivery Attempts:

• Upon a cloud connection establishment.
• A ping/pong was received from the cloud & at least 7 seconds have elapsed.

High priority messages

Same as Medium except the expiration time in the queue is 25 minutes.

Critical priority messages

Same as Medium except the expiration time in the queue is 30 minutes.

Parodus CRUD Interface

Parodus needs to implement the CRUD model it defined here:
https://github.com/Comcast/parodus/wiki/WRP-CRUD-Model-Details

Abort parodus start without hw-mac command line argument

hw-mac is required command line argument for parodus so if that is not passed then abort the process.

Process messages before cloud-disconnect

When parodus is doing force disconnect from XMIDT cloud (using /cloud-disconnect UPADTE request etc), upstream and downstream messages in queue need to be handled.

If cloud-disconnect is in progress, incoming requests should be handled and parodus should send a response saying "Parodus is about to shut down & will function after 5mins" with proper error code.

Existing messages in queue should be processed and send response to server before proceeding to disconnect from cloud.

Parodus not compiling on Ubuntu 12.04

Hi Wes ,

We are trying to build webpa on Ubuntu but facing compilation issues with respect to clock_gettime symbol undefined reference . I have ensured that -lrt library in included .
Just for FYI ; tools version which I am using

Ubuntu 12.04.5 64bit
Cmake 3.7.2
Make 3.81
Gcc 4.6.3
G++ 4.6.3
and cunit framework installation for webpa ucnit framework based tests .

time.h is already included along with required included RT library .

Can you please confirm if this environment is enough for compilation or is there any missing dependency :
Here are logs :
/CMakeFiles/test_conn_interface.dir/__/src/mutex.c.o -o test_conn_interface -L/home/rajesh/WebPA/parodus/build/_install/lib -L/home/rajesh/WebPA/parodus/build/_install/lib64 -Wl,-rpath,/home/rajesh/WebPA/parodus/build/_install/lib:/home/rajesh/WebPA/parodus/build/_install/lib64 -rdynamic -lcmocka -lgcov -lcunit -lwrp-c -lpthread -lm -lmsgpackc -lcjson -ltrower-base64 -lnopoll -lnanomsg -lrt -lcimplog
/home/rajesh/WebPA/parodus/build/_install/lib/libcimplog.so: undefined reference to clock_gettime' collect2: ld returned 1 exit status make[2]: *** [tests/test_conn_interface] Error 1 make[2]: Leaving directory /home/rajesh/WebPA/parodus/build'
make[1]: *** [tests/CMakeFiles/test_conn_interface.dir/all] Error 2
make[1]: Leaving directory `/home/rajesh/WebPA/parodus/build'
make: *** [all] Error 2

Regards
Nidhi

Remove dependency on comcast infrastructure

All tests, documentation and similar need to remove dependency on comcast infrastructure.

Clear the config auth token when 403 is received and new token fetch fails

Clear the config auth token when 403 is received and new token fetch fails.

Ensure Correct Xmidt Header Usage

Parodus creates a cURL request with the incorrect header format: X-Midt-*. This should be updated to use the correct header format: X-Xmidt-* to reduce confusion.

webpa messages and payloads

Hello.

My name is Vitor Paulino and I work at NOS Inovação and we at NOS Inovação are starting to explore this and others repositories regarding webpa protocol.
To understand the messages payload that are sent and received by parodus, it would be great if there was any documentation regarding the use cases and the messages that are exchanged. Can you provide something so we can start working on this?

Best regards
Vitor Paulino

Observed syntax error `*-*-beos*' while running make

Followed the build instruction mentioned in the page. Getting error while executing make command

./config.status: line 561: syntax error near unexpected token *-*-beos*' ./config.status: line 561: --beos*)'
CMakeFiles/nopoll.dir/build.make:105: recipe for target '_prefix/nopoll/src/nopoll-stamp/nopoll-configure' failed
make[2]: *** [_prefix/nopoll/src/nopoll-stamp/nopoll-configure] Error 1
CMakeFiles/Makefile2:1187: recipe for target 'CMakeFiles/nopoll.dir/all' failed
make[1]: *** [CMakeFiles/nopoll.dir/all] Error 2
Makefile:140: recipe for target 'all' failed
make: *** [all] Error 2

Rename command line options for MTLS cert and Key

There is name conflict for these command line options

• /client-cert-path -MTLS client cert location to request auth token from MTLS HTTPS server for establishing secure connection [absolute path where client cert is present] (optional argument)

Websocket MTLS options

• /client-cert-path - Provide the client cert for establishing a mutual auth secure websocket connection

• /client-key-path - Provide the client cert key for establishing a mutual auth secure websocket connection

So, rename MTLS websocket connect client cert and key path to mtls-client-cert-path and mtls-client-key-path

build error

Hi. My gcc/g++ version is 5.3

I use make, the errors are like this:

config.status: executing depfiles commands
config.status: executing libtool commands
config.status: executing nopoll_config.h commands
config.status: nopoll_config.h is unchanged
------------------------------------------
--       LibNoPoll (0.4.6.b400) LIBRARY SETTINGS     --
------------------------------------------
   Installation prefix:            
      select(2) support:           yes
      poll(2) support:             yes
      epoll(2) support:            yes
   OpenSSL TLS protocol versions detected:
      SSLv3:   no
      SSLv23:  no
      TLSv1.0: no
      TLSv1.1: no
      TLSv1.2: no
      TLS flx: no
------------------------------------------
--     NOW TYPE: make; make install     --
------------------------------------------
--     Mira papa! Sin manos!!!          --
------------------------------------------
[ 23%] Performing build step for 'nopoll'
Making all in src
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -Wstrict-prototypes -Wall -Werror -ansi -I.. -DVERSION=\"0.4.6.b400\" -DPACKAGE_DTD_DIR=\"/share\" -DPACKAGE_TOP_DIR=\"..\" -DVERSION=\"0.4.6.b400\" -DSHOW_DEBUG_LOG -fstack-protector-all -Wstack-protector -g -O2 -MT nopoll.lo -MD -MP -MF .deps/nopoll.Tpo -c nopoll.c  -fPIC -DPIC -o .libs/nopoll.o
In file included from ./nopoll.h:41:0,
                 from nopoll.c:38:
./nopoll_decl.h:546:1: error: expected identifier before '}' token
 } noPollSslProtocol ;
 ^
In file included from ./nopoll.h:52:0,
                 from nopoll.c:38:
./nopoll_conn_opts.h:47:64: error: unknown type name 'noPollSslProtocol'
 void nopoll_conn_opts_set_ssl_protocol (noPollConnOpts * opts, noPollSslProtocol ssl_protocol);
                                                                ^
In file included from ./nopoll_hostname_validation.h:28:0,
                 from ./nopoll.h:59,
                 from nopoll.c:38:
./nopoll_private.h:403:2: error: unknown type name 'noPollSslProtocol'
  noPollSslProtocol    ssl_protocol;
  ^
make[5]: *** [nopoll.lo] Error 1
make[4]: *** [all-recursive] Error 1
make[3]: *** [all] Error 2
make[2]: *** [_prefix/nopoll/src/nopoll-stamp/nopoll-build] Error 2
make[1]: *** [CMakeFiles/nopoll.dir/all] Error 2
make: *** [all] Error 2

On getaddrinfo error in redirected URL, re-attempt with default server URL

[1541539952][PARODUS][Debug]: ...............Inside checkHostIp..............talaria-prod-00101-184-204-77t.xmidt.comcast.net
[1541539952][PARODUS][Error]: getaddrinfo: No address associated with hostname
[1541539952][PARODUS][Info]: Waiting with backoffRetryTime 9 seconds
[1541539961][PARODUS][Debug]: secure true
[1541539961][PARODUS][Debug]: mutex init successfully
[1541539961][PARODUS][Info]: Connecting in Ipv6 mode
[1541539961][PARODUS][Debug]: Mutex destroyed
[1541539961][PARODUS][Debug]: ...............Inside checkHostIp..............talaria-prod-00101-184-204-77t.xmidt.comcast.net
[1541539961][PARODUS][Error]: getaddrinfo: No address associated with hostname
[1541539961][PARODUS][Debug]: secure true
[1541539961][PARODUS][Debug]: mutex init successfully
[1541539961][PARODUS][Info]: Connecting in Ipv4 mode
[1541539961][PARODUS][Debug]: Mutex destroyed
[1541539961][PARODUS][Debug]: ...............Inside checkHostIp..............talaria-prod-00101-184-204-77t.xmidt.comcast.net
[1541539961][PARODUS][Error]: getaddrinfo: No address associated with hostname
[1541539961][PARODUS][Info]: Waiting with backoffRetryTime 9 seconds

Switching between ipv6 and ipv4 but not retrying with default url.

DNS TXT JWT expiration should disconnect client

When the JWT acquired via the DNS TXT record expires, parodus should disconnect from the URL specified in that JWT and restart the reconnection process fresh.

This shall include attempting to acquire an updated DNS TXT record with a JWT. All normal & default semantics apply.

test

test

Rename master branch to main

Also have to change references to the branch in .travis.yml, README, and CONTRIBUTING. Double check any other markdown files as well - sometimes links have the branch name in them.

TLS negotiation error in parodus

We are getting a TLS negotiation error from parodus when it's trying to connect to a local talaria server & need some support on resolving this.

• Parodus is running in a DEV XG1v3 & talaria runs from a local server.
• The local servers are not configured with any certificate so we added --force-ipv4 --secure-flag=http parameters as parodus command line arguments to disable certificate negotiation & https.
• Also Same error is encountered with and without the --ssl-cert-path argument.

Command:
/usr/bin/parodus --hw-mac=84E058575831 --webpa-ping-time=180 --webpa-interface-used= --webpa-url=192.168.30.105 --partner-id=comcast --webpa-backoff-max=9 --force-ipv4 --secure-flag=http --ssl-cert-path=/etc/ssl/certs/ca-certificates.crt

Parodus log:
... pacexg1v3 "parodus"[16560]: /usr/bin/parodus: unrecognized option --secure-flag=http'
2018 Jan 30 10:01:25.582268 pacexg1v3 parodus[16560]: 180130-10:01:25.582199 mod=PARODUS, lvl=INFO] [tid=16560] PARODUS: nopoll_conn.c:795 Enabling certificate peer verification
2018 Jan 30 10:01:25.584535 pacexg1v3 parodus[16560]: 180130-10:01:25.584430 mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: nopoll_conn.c:1001 there was an error with the TLS negotiation, ssl error (code:1) : error:00000001:lib(0):func(0):reason(1)
2018 Jan 30 10:01:25.584703 pacexg1v3 parodus[16560]: 180130-10:01:25.584657 [mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: nopoll_conn.c:478 tls stack: error:1408F10B:lib(20):func(143):reason(267) (find reason(code) at openssl/ssl.h)
2018 Jan 30 10:01:25.584773 pacexg1v3 parodus[16560]: 180130-10:01:25.584734 [mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: nopoll_conn.c:493 details, run: openssl errstr 1408F10B
2018 Jan 30 10:01:25.584870 pacexg1v3 parodus[16560]: 180130-10:01:25.584831 [mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: nopoll_conn.c:498 noPoll id=2, socket: 4 (after testing errno: 0)
2018 Jan 30 10:01:25.585222 pacexg1v3 parodus[16560]: 180130-10:01:25.585170 [mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: Error connecting to server
2018 Jan 30 10:01:25.585277 pacexg1v3 parodus[16560]: 180130-10:01:25.585241 [mod=PARODUS, lvl=ERROR] [tid=16560] PARODUS: RDK-10037 - WebPA Connection Lost

parodus for openwrt system

Hi
is there any version of parodus for openwrt system ?

Backoff retry is not working

Back off retry is fixed to one value and not iterating like 3,5, 7 exponentially during connection retry.

[1544467387][PARODUS][Info]: Ping received with payload mac:666666667777, opcode 9
[1544467392][PARODUS][Info]: No clients are registered, waiting ..

[1544467432][PARODUS][Info]: Ping received with payload mac:666666667777, opcode 9

[1544467613][PARODUS][Info]: heartBeatTimer 180857 webpa_ping_timeout_ms 180000
[1544467613][PARODUS][Error]: ping wait time > 180 . Terminating the connection with WebPA server and retrying
[1544467613][PARODUS][Info]: Reconnect detected, setting Ping_Miss reason for Reconnect
[1544467613][PARODUS][Info]: close_retry is 1, hence closing the connection and retrying
[1544467613][PARODUS][Info]: nopoll_ctx.c:340 Returning, unlock of mutex is not required
[1544467613][PARODUS][Info]: cloud_status set as offline after connection close
[1544467613][PARODUS][Info]: Received reboot_reason as:
[1544467613][PARODUS][Info]: Received reconnect_reason as:Ping_Miss
[1544467613][PARODUS][Info]: User-Agent: PARODUS-2.0-1.0.1-69-gf4d544f (test_shilpa; unknown/unknown;)
[1544467613][PARODUS][Error]: Failed to GET Reboot reason value
[1544467613][PARODUS][Info]: X-WebPA-Convey Header: [198]{"hw-serial-number":"shilpa","fw-name":"test_shilpa","boot-time":1494590301,"webpa-protocol":"PARODUS-2.0-1.0.1-69-gf4d544f","webpa-inteface-used":"enp0s5","webpa-last-reconnect-reason":"Ping_Miss"}
[1544467613][PARODUS][Info]: Device_id mac:666666667777
[1544467613][PARODUS][Info]: full url: https://fabric-beta.xmidt.comcast.net:8080
[1544467613][PARODUS][Info]: server address copied from url
[1544467613][PARODUS][Info]: server fabric-beta.xmidt.comcast.net, port 8080, http_match 0
[1544467613][PARODUS][Info]: default server_Address fabric-beta.xmidt.comcast.net
[1544467613][PARODUS][Info]: default port 8080
[1544467613][PARODUS][Info]: Connecting in Ipv4 mode
[1544467613][PARODUS][Error]: getaddrinfo: Temporary failure in name resolution
[1544467613][PARODUS][Info]: Waiting with backoffRetryTime 3 seconds
[1544467616][PARODUS][Info]: full url: https://fabric-beta.xmidt.comcast.net:8080
[1544467616][PARODUS][Info]: server address copied from url
[1544467616][PARODUS][Info]: server fabric-beta.xmidt.comcast.net, port 8080, http_match 0
[1544467616][PARODUS][Info]: default server_Address fabric-beta.xmidt.comcast.net
[1544467616][PARODUS][Info]: default port 8080
[1544467616][PARODUS][Info]: Connecting in Ipv4 mode
[1544467616][PARODUS][Error]: getaddrinfo: Temporary failure in name resolution
[1544467616][PARODUS][Info]: Waiting with backoffRetryTime 3 seconds
[1544467619][PARODUS][Info]: full url: https://fabric-beta.xmidt.comcast.net:8080
[1544467619][PARODUS][Info]: server address copied from url
[1544467619][PARODUS][Info]: server fabric-beta.xmidt.comcast.net, port 8080, http_match 0
[1544467619][PARODUS][Info]: default server_Address fabric-beta.xmidt.comcast.net
[1544467619][PARODUS][Info]: default port 8080
[1544467619][PARODUS][Info]: Connecting in Ipv4 mode
[1544467619][PARODUS][Error]: getaddrinfo: Temporary failure in name resolution
[1544467619][PARODUS][Info]: Waiting with backoffRetryTime 3 seconds
[1544467622][PARODUS][Info]: full url: https://fabric-beta.xmidt.comcast.net:8080
[1544467622][PARODUS][Info]: server address copied from url
[1544467622][PARODUS][Info]: server fabric-beta.xmidt.comcast.net, port 8080, http_match 0
[1544467622][PARODUS][Info]: default server_Address fabric-beta.xmidt.comcast.net
[1544467622][PARODUS][Info]: default port 8080
[1544467622][PARODUS][Info]: Connecting in Ipv4 mode
[1544467622][PARODUS][Error]: getaddrinfo: Temporary failure in name resolution
[1544467622][PARODUS][Info]: Waiting with backoffRetryTime 3 seconds