plgd-dev / hub Goto Github PK

Secure and Interoperable Internet of Things

License: Apache License 2.0

Dockerfile 0.46% Makefile 1.08% Go 77.18% Shell 1.57% JavaScript 0.80% HTML 5.20% SCSS 0.08% Python 0.38% Smarty 0.81% TypeScript 12.45%

ocf cloud iot-cloud iotivity-lite iotivity openconnectivityfoundation coap iot

hub's People

Contributors

Stargazers

Watchers

Forkers

foxxnuaa opencollective yorickcoleu 00mjk infinera makothap infirr kurzgood wavdbeek fossabot jokerbro rlopinfn soyoo iq-scm oursland

hub's Issues

Resource & Device notifications on plgd Dashboard

Feature Request

What is the purpose of the feature?

Being notified about changes using toasts directly in the plgd Dashboard.

What specific challenge does this feature solve?

Easy to use.

What alternatives did I consider?

None

Optimize keep-alive between the device and CoAP Gateway

Pods not starting due to 'too many open files' error

Description

When deploying PLGD it seems if the number of pods being deployed is high enough then some will not run. For example if 20 coap gw (replicas) pods are configured to be deployed then 7 will never reach running state.

How can it be reproduced?

deploy using command:
microk8s.helm3 install am-1 -f withMock.yaml plgd/plgd-hub --create-namespace --namespace=am-1
withMock.yaml content:

global:
  domain: "host.domain.com"
  hubId: "d03a1bb4-0a77-428c-b78c-1c46efe6a38e"
mockoauthserver:
  enabled: true
resourcedirectory:
  log:
    debug: true
resourceaggregate:
  log:
    debug: true
httpgateway:
  log:
    debug: true
grpcgateway:
  log:
    debug: true
coapgateway:
  log:
    debug: true
  replicas: 20
  port: 5683
  service:
    type: NodePort
    nodePort: 5683
  apis:
    coap:
      tls:
        enabled: false

Logs, error output, etc.

Log from a coap gw pod that does not reach running state (continuously restarting):

2022-01-19T16:34:13.332Z        FATAL   log/log.go:129  cannot create service: cannot create identity-store client: cannot create connection to identity-store: cannot create cert manager: too many open files
github.com/plgd-dev/hub/v2/pkg/log.Fatalf
        /go/src/github.com/plgd-dev/hub/pkg/log/log.go:129
main.main
        /go/src/github.com/plgd-dev/hub/coap-gateway/cmd/service/main.go:25
runtime.main
        /usr/local/go/src/runtime/proc.go:255

The pods that fail to reach running state are not always coap gw pods. Sometimes there are other pods, which fail in a similar fashion and give the same error message. What seems constant is the number of pods that do not run after PLGD installation.
The OS Kernel open file settings used are the following:

#ulimit -Hn
262144
#ulimit -Sn  (increased this one from 1024 to 250K but did not made any difference)
250000 
#cat /proc/sys/fs/file-max
1613280

I also see that the number of entries in lsof command jumps about 60K with PLGD deployed (far more than everything else in the OS)

Environment

microk8s v1.22.4 (snap) running on CentOS Linux release 8.5.2111
microk8s helm3 v3.5.0
plgd hub tag: 2.2.3
IoTivity version: none (issue is reproducible with no IOT devices connected to PLGD)
Deployment option: ( helm )

Support subscription to different events through one WebSocket connection

Feature Request

What is the purpose of the feature?

The goal is to be able to drive what events are received through one WS connection. In the current implementation, the new WS has to be open per each event type and device we are interested in.

What specific challenge does this feature solve?

In case we need to subscribe to resource content changes of 1000 devices, we need to open 1000 WS connections.

What alternatives did I consider?

The gRPC Gateway streams would work, but not on the UI.

cloud2cloud-connector can not pull devices and create devices because of nats invalid argument.

@jkralik @ondrejtomcik

cloud2cloud-connector can not pull devices and create devices because of nats invalid argument.

I'm using commit 4604d31 of v2 branch now.
And I have tested plgd cloud with CTT Target Cloud in the local network.

How can I solve this issue ?
ERROR log/log.go:117 cannot pull devices for linked linkedAccount({d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}): [[cannot add device b6167c31-b593-43e1-9007-b6066ea9b49b to devicesSubscription: cannot create device b6167c31-b593-43e1-9007-b6066ea9b49b pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument cannot add device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 to devicesSubscription: cannot create device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument] [cannot notifyResourceChanged {Href:/oic/p Representation:map[mnmn:OCF pi:220b0ec7-3dca-4631-953f-25bf14d695e8]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/BinarySwitchResURI Representation:map[value:false]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/oic/d Representation:map[di:99e5e72b-a9f5-40f8-998a-f3e7a23dd258 dmn:[map[language:en-US value:Comarch]] dmv:ocf.res.1.3.0, ocf.sh.1.3.0 icv:ocf.2.2.0 n:CTT oic.d.smartplug simulator piid:2f3227d9-30e1-40fd-b3a0-f12b567eb3e7]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/oic/p Representation:map[mnmn:OCF pi:0b59268f-a2a2-46ff-9a8a-b7fb6cfba866]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/BinarySwitchResURI Representation:map[value:false]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded]]

This is what I found error:

2021-08-05 17:07:13 | 2021-08-05T08:07:13.509Z	DEBUG	log/log.go:102	pulling devices for {d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}
-- | --
  |   | 2021-08-05 17:07:16 | 2021-08-05T08:07:15.999Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/d: {"di":"b6167c31-b593-43e1-9007-b6066ea9b49b","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.light simulator","piid":"d438e20f-1417-4af4-9b1e-8d6451ade5a3"}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.509Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/p: {"mnmn":"OCF","pi":"220b0ec7-3dca-4631-953f-25bf14d695e8"}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.510Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.510Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/d: {"di":"99e5e72b-a9f5-40f8-998a-f3e7a23dd258","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.smartplug simulator","piid":"2f3227d9-30e1-40fd-b3a0-f12b567eb3e7"}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.672Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/p: {"mnmn":"OCF","pi":"0b59268f-a2a2-46ff-9a8a-b7fb6cfba866"}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.833Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.873Z	ERROR	log/log.go:117	cannot pull devices for linked linkedAccount({d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}): [[cannot add device b6167c31-b593-43e1-9007-b6066ea9b49b to devicesSubscription: cannot create device b6167c31-b593-43e1-9007-b6066ea9b49b pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument cannot add device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 to devicesSubscription: cannot create device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument] [cannot notifyResourceChanged {Href:/oic/p Representation:map[mnmn:OCF pi:220b0ec7-3dca-4631-953f-25bf14d695e8]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/BinarySwitchResURI Representation:map[value:false]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/oic/d Representation:map[di:99e5e72b-a9f5-40f8-998a-f3e7a23dd258 dmn:[map[language:en-US value:Comarch]] dmv:ocf.res.1.3.0, ocf.sh.1.3.0 icv:ocf.2.2.0 n:CTT oic.d.smartplug simulator piid:2f3227d9-30e1-40fd-b3a0-f12b567eb3e7]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/oic/p Representation:map[mnmn:OCF pi:0b59268f-a2a2-46ff-9a8a-b7fb6cfba866]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/BinarySwitchResURI Representation:map[value:false]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded]]
  |   | 2021-08-05 17:07:18 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-08-05 17:07:18 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-08-05 17:07:18 | github.com/plgd-dev/cloud/cloud2cloud-connector/service.pullDevices.func1
  |   | 2021-08-05 17:07:18 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-connector/service/pull.go:381
  |   | 2021-08-05 17:07:18 | 2021-08-05T08:07:18.878Z	DEBUG	log/log.go:102	pulling devices for {d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}
  |   | 2021-08-05 17:07:22 | 2021-08-05T08:07:22.945Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/d: {"di":"b6167c31-b593-43e1-9007-b6066ea9b49b","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.light simulator","piid":"d438e20f-1417-4af4-9b1e-8d6451ade5a3"}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.066Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/p: {"mnmn":"OCF","pi":"220b0ec7-3dca-4631-953f-25bf14d695e8"}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.076Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.633Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/d: {"di":"99e5e72b-a9f5-40f8-998a-f3e7a23dd258","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.smartplug simulator","piid":"2f3227d9-30e1-40fd-b3a0-f12b567eb3e7"}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.884Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/p: {"mnmn":"OCF","pi":"0b59268f-a2a2-46ff-9a8a-b7fb6cfba866"}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.908Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.908Z	ERROR	log/log.go:117	cannot pull devices for linked linkedAccount({d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}): [[cannot add device b6167c31-b593-43e1-9007-b6066ea9b49b to devicesSubscription: cannot create device b6167c31-b593-43e1-9007-b6066ea9b49b pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument cannot add device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 to devicesSubscription: cannot create device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument] [cannot notifyResourceChanged {Href:/oic/p Representation:map[mnmn:OCF pi:0b59268f-a2a2-46ff-9a8a-b7fb6cfba866]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded cannot notifyResourceChanged {Href:/BinarySwitchResURI Representation:map[value:false]}: rpc error: code = DeadlineExceeded desc = context deadline exceeded]]
  |   | 2021-08-05 17:07:23 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-08-05 17:07:23 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-08-05 17:07:23 | github.com/plgd-dev/cloud/cloud2cloud-connector/service.pullDevices.func1
  |   | 2021-08-05 17:07:23 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-connector/service/pull.go:381
  |   | 2021-08-05 17:07:23 | 2021-08-05T08:07:23.908Z	DEBUG	log/log.go:102	pulling devices for {d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.543Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/d: {"di":"b6167c31-b593-43e1-9007-b6066ea9b49b","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.light simulator","piid":"d438e20f-1417-4af4-9b1e-8d6451ade5a3"}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.551Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/oic/p: {"mnmn":"OCF","pi":"220b0ec7-3dca-4631-953f-25bf14d695e8"}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.561Z	DEBUG	log/log.go:102	notifyResourceChanged b6167c31-b593-43e1-9007-b6066ea9b49b/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.567Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/d: {"di":"99e5e72b-a9f5-40f8-998a-f3e7a23dd258","dmn":[{"language":"en-US","value":"Comarch"}],"dmv":"ocf.res.1.3.0, ocf.sh.1.3.0","icv":"ocf.2.2.0","n":"CTT oic.d.smartplug simulator","piid":"2f3227d9-30e1-40fd-b3a0-f12b567eb3e7"}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.573Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/oic/p: {"mnmn":"OCF","pi":"0b59268f-a2a2-46ff-9a8a-b7fb6cfba866"}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.641Z	DEBUG	log/log.go:102	notifyResourceChanged 99e5e72b-a9f5-40f8-998a-f3e7a23dd258/BinarySwitchResURI: {"value":false}
  |   | 2021-08-05 17:07:27 | 2021-08-05T08:07:27.641Z	ERROR	log/log.go:117	cannot pull devices for linked linkedAccount({d7219308-a575-4aae-b6db-e91cc1b4b7e1 a45c0763-f013-48c7-9a34-5de92a6705c5 auth0\|61095976fc3c6c00757ae79d {dyjcu0zx.5y5 owifdxvy.yd5 2021-08-05 08:11:57.8647477 +0000 UTC m=+1850.572177201}}): [[cannot add device b6167c31-b593-43e1-9007-b6066ea9b49b to devicesSubscription: cannot create device b6167c31-b593-43e1-9007-b6066ea9b49b pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument cannot add device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 to devicesSubscription: cannot create device 99e5e72b-a9f5-40f8-998a-f3e7a23dd258 pending subscription: cannot subscribe: cannot subscribe to topics: nats: invalid argument]]
  |   | 2021-08-05 17:07:27 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-08-05 17:07:27 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-08-05 17:07:27 | github.com/plgd-dev/cloud/cloud2cloud-connector/service.pullDevices.func1
  |   | 2021-08-05 17:07:27 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-connector/service/pull.go:381

[Http-Gateway] Unsupported parsing format of application/json of http gateway when I requested GET/POST devices API of origin cloud toward target cloud's devices.

@jkralik, @ondrejtomcik

As you know, there was an unsupported parsing issue of application/json of http gateway when I requested GET/POST devices API of origin cloud toward target cloud's devices.

See the error result of http-gateway devices APIs below :

{"err":"cannot get resource: rpc error: code = InvalidArgument desc = cannot decode response: not a CBOR content format: application/json"}

I wish you could fix this issue as soon as possible.

Gateways receive pending commands from EventBus

Feature Request

What is the purpose of the feature?

plgd Gateways are receiving events about pending commands from the Resource Directory, instead of being subscribed directly to the EventBus. The goal is to not use this middleware forwarding EventBus events through the gRPC stream but directly subscribe to the EventBus.

What specific challenge does this feature solve?

Reduces unnecessary dependency introducing additional complexity during scaling.

What alternatives did I consider?

None

Force device deletion directly from the plgd Cloud

Feature Request

What is the purpose of the feature?

Being able to delete the device from the plgd cloud if it's online, or cleanup the list of devices which are already disconnected and will never connect again.

What specific challenge does this feature solve?

Devices can be currently deleted from the plgd only using the onboarding tool, on the local network. But even when using this path it's not guaranteed that the device will be removed from the plgd cloud. If the device is disconnected from the plgd cloud (network issue) but OBT can reach it and reset, this device will never connect back to plgd cloud but it's shadow will be still in the list. User needs a possibility to delete stale devices or trigger deregistration of online device directly from the plgd cloud.

What alternatives did I consider?

None are available

Device status online when coap gateway is down

Description

When coap gateway is not running, plgd hub continues to report the previous online/offline device status of the device even though it cannot communicate with it (and so cannot be sure that is the actual state of the device).

How can it be reproduced?

deploy using command:
microk8s.helm3 install am-1 -f withMock.yaml plgd/plgd-hub --create-namespace --namespace=am-1 --version 2.2.2

withMock.yaml content:

global:
  domain: "host.domain.com"
  hubId: "d03a1bb4-0a77-428c-b78c-1c46efe6a38e"
mockoauthserver:
  enabled: true
coapgateway:
  port: 5683
  service:
    type: NodePort
    nodePort: 5683
  apis:
    coap:
      tls:
        enabled: false

onboard devices and check their status

export ACCESS_TOKEN=$(curl -ks "https://${HOSTNAME}:443/oauth/token?client_id=test&audience=test" | jq -r .access_token)
curl -ks -XGET "https://${HOSTNAME}:443/api/v1/devices" --header 'Content-Type: application/json' --header "Authorization: Bearer ${ACCESS_TOKEN}"

stop coap gateway (by scaling it down to zero replicas)
microk8s.kubectl scale deployment am-1-plgd-hub-coap-gateway --replicas=0 --namespace am-1
Check devices status again

curl -ks -XGET "https://${HOSTNAME}:443/api/v1/devices" --header 'Content-Type: application/json' --header "Authorization: Bearer ${ACCESS_TOKEN}"

Environment

microk8s v1.22.4 (snap) running on CentOS Linux release 8.5.2111
microk8s helm3 v3.5.0
plgd hub tag: 2.2.2

Change CoAP Gateway href prefix

CoAP Gateway uses /oic/route as href prefix when accessing device resources. This should be changed as only the OCF standardized hrefs contains /oic. This should be changed to /api/v1/devices/

Jetstream as an EventBus

Feature Request

What is the purpose of the feature?

Some users are interested in persisting streams of events. JetStream which is built on top of NATS can be used to achieve this in a way it won't affect the plgd cloud architecture.

What specific challenge does this feature solve?

Providing persisted streams of events what is interested for some clients subscribed directly to NATS/JetStream.

What alternatives did I consider?

None

[cloud2cloud-gateway] cloud2cloud-gateway exposes token is expired error after some time of starting the service

@ondrejtomcik @jkralik

Description

cloud2cloud-gateway exposes token is expired error after some time of starting the service even if the used token is newly obtained access token from oauth provider. The expired time of new access token has enough time util being expired and the access token has no problem to request APIs of http-gateway service.

After restarting cloud2cloud-gateway service, the access token that I used before has no problem to request APIs any more. It's wired situation that only c2c-gateway service has this kind of issue.

How can I handle this issue? Is it known issue or not ?

Logs, error output, etc.

2021-09-06T06:31:16.471Z        DEBUG   log/log.go:102  GET /api/v1/devices
2021-09-06T06:31:16.480Z        ERROR   log/log.go:117  cannot retrieve all devices: cannot retrieve all devices[base]: cannot get devices: rpc error: code = Unauthenticated desc = invalid token: could not parse token: token is expired
github.com/plgd-dev/kit/log.Errorf
        /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
github.com/plgd-dev/cloud/cloud2cloud-gateway/service.logAndWriteErrorResponse
        /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/httpApi.go:46
github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*RequestHandler).RetrieveDevices
        /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/retrieveDevices.go:172
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2069
github.com/plgd-dev/cloud/pkg/net/http.CreateAuthMiddleware.func1.1
        /go/src/github.com/plgd-dev/cloud/pkg/net/http/auth.go:81
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2069
github.com/plgd-dev/cloud/cloud2cloud-gateway/service.loggingMiddleware.func1
        /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/httpApi.go:143
net/http.HandlerFunc.ServeHTTP
        /usr/local/go/src/net/http/server.go:2069
github.com/gorilla/mux.(*Router).ServeHTTP
        /go/pkg/mod/github.com/gorilla/[email protected]/mux.go:210
net/http.serverHandler.ServeHTTP
        /usr/local/go/src/net/http/server.go:2887
net/http.(*conn).serve
        /usr/local/go/src/net/http/server.go:1952

Environment

plgd source : commit e019708
IoTivity version: jakarta branch merged with master
Deployment option: bundle on my own

Additional context

[cloud2cloud-gateway] Resource APIs can not retrieve resource data with query parameter (i.e. ?if=oic.if.baseline).

Description

@ondrejtomcik @jkralik
Recently, I'm doing test plgd cloud with CTT Tool. I found that Resource API(i.e. /api/v1/devices/{deviceId}/{resourceHref}) of cloud2cloud-gateway can not retrieve resource data with query parameter (i.e. ?if=oic.if.baseline).

Logs

22:39:47 VERBOSE: CTT HTTP Client -> [request ID=4E07D12D] GET https://c2c-api.ttaocfcloud.org/api/v1/devices/a0253411-a758-4a30-6efc-27b5061bbd58/platform?if=oic.if.baseline
Headers: [Accept=application/json, Authorization=Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImxfcUtMNlE3djFRNTNyZ3dSSFN1aiJ9.eyJpc3MiOiJodHRwczovL3R0YW9jZi51cy5hdXRoMC5jb20vIiwic3ViIjoiZ29vZ2xlLW9hdXRoMnwxMDQ0MzAyMzI5NjAyMzA1ODY1OTUiLCJhdWQiOiJodHRwczovL3d3dy50dGFvY2ZjbG91ZC5vcmciLCJpYXQiOjE2MzA0ODI4MTYsImV4cCI6MTYzMDU2OTIxNiwiYXpwIjoieXhGOXJQQ3B0OFRrcVJEbmpYZ1NPa0YxR3dFUVVaYVoiLCJzY29wZSI6InI6ZGV2aWNlaW5mb3JtYXRpb246KiByOnJlc291cmNlczoqIHc6cmVzb3VyY2VzOioifQ.DTdhrd0FyzDIdQpj_IAFWvAmhDc4gwkogYBIRf7_8QJX0aAtqKVwdN6LGipVVLDhdZfaVi6UvGy0Qu2nw-U_kW68iQtP3oQF_ISEWGxRn3vOzLiyD6ppzqV6l5g4dLZXwd9KuEcdQ7rLOEb0Kz3PSd1tU8PUlPDsSm2_NDVl_Hm3PU4xnhMk_ReL11sATOlS3uKo11fUKciHPjbWAXuJEZ3SX6uFUEqFnVomwh592FpBaGpXLAEGm6jgOWwF05YikoeXys9EeyFA6CIcmHUzdAvyTCcRX7hzY9QGWFTc2uaA_L56Sw9sDZBkxExjUh0LgAwFvpUw82iUC-7TPlW5RA]
22:39:47 VERBOSE: CTT HTTP Client <- [response ID=4E07D12D] GET https://c2c-api.ttaocfcloud.org/api/v1/devices/a0253411-a758-4a30-6efc-27b5061bbd58/platform?if=oic.if.baseline
Code: 404 NotFound
Headers: [Connection=keep-alive, Content-Length=213, Content-Type=text/plain; charset=UTF-8, Date=Wed, 01 Sep 2021 13:39:47 GMT, Via=kong/2.5.0, X-Kong-Proxy-Latency=0, X-Kong-Upstream-Latency=3]
Body: cannot retrieve resource: cannot retrieve resource(deviceID: a0253411-a758-4a30-6efc-27b5061bbd58, Href: /platform?if=oic.if.baseline): rpc error: code = NotFound desc = cannot retrieve resources values: not found
22:39:47 ERROR: CT5.2.2_Check_3: Status code in response ID=4E07D12D is 404 while 200 was expected

CTT is expecting below step:
Step 3.5: If the Resource is a Collection or Atomic Measurement, the CTT Origin Cloud also sends a HTTPS GET message to "/api/v1/devices/{link href}" where {link href} is the "href" Link Parameter of the first Link returned in the response received in Step 2. The request contains a baseline OCF Interface query.

Environment

plgd source : commit e019708
IoTivity version: jakarta branch merged with master
Deployment option: bundle on my own

Reduce number of goroutines in processing requests

Feature Request

What is the purpose of the feature?

Optimize mem and CPU footprint of the plgd. We've observed that the number of goroutines spawned when some API is requested is too high.

What specific challenge does this feature solve?

Better mem and cpu footprint.

What alternatives did I consider?

None

[cloud2cloud-gateway] Events APIs for registering subscriptions asks the client to have "w:subscriptions:* which is not matched with the spec and CTT requirement.

@ondrejtomcik @jkralik

In the mail thread with OCF staff about "Why are CT5.3.1~CT5.3.3 test cases using only Retrieval Scope for Account Linking to get an access token ?", I have a lot of discussion about this. Finally the reason why I didn't know the scope for subscription was changed to only retrieval scope and the implementation of plgd was not updated to the final spec.

Description

[cloud2cloud-gateway] Events APIs for registering subscriptions asks the client to have "w:subscriptions:* which is not matched with the final spec and CTT requirement. So I think it need to be fixed as soon as possible.

Support branding of plgd Dashboard

Feature Request

What is the purpose of the feature?

Demonstrate plgd cloud in colors and with the logo of our company.

What specific challenge does this feature solve?

Convincing PoC for client's stakeholders.

What alternatives did I consider?

None

coap gateway do not start with fatal randomly

Not reproducible all the time but on some occasions, after the start, the system fails to onboard devices and this can be seen in the log's from coap-gateway.log

{"level":"fatal","ts":1630674629.136271,"caller":"log/log.go:129","msg":"cannot load config: invalid config: yaml: unmarshal errors:\n line 2: cannot unmarshal !!str false into bool","stacktrace":"github.com/plgd-dev/cloud/pkg/log.Fatalf\n\t/go/src/github.com/plgd-dev/cloud/pkg/log/log.go:129\nmain.main\n\t/go/src/github.com/plgd-dev/cloud/coap-gateway/cmd/service/main.go:15\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:225"}

The system is started using docker-compose. PLGD bundle uses the following flags:

FQDN: ${IP}
LOG_DEBUG: "false"
JETSTREAM: "true"

Generically stopping and just restarting using the same docker images solves the issue.

The full log's collected are attached
archive.tar.gz

Support tree view for resource browsing

Feature Request

What is the purpose of the feature?

Support easier browsing through device resources.

What specific challenge does this feature solve?

If the device has e.g. 100 resources that have a hierarchy, e.g.

/channels/1
/channels/1
/channesl/1/calibrations/32
/channesl/3/calibrations/1
It's hard to browse them in a table. Having an expandable tree view would be very nice!

What alternatives did I consider?

None

Provide k8s Helm chart for core plgd services

Feature Request

What is the purpose of the feature?

Deploy plgd cloud in a standardized way in k8s.

What specific challenge does this feature solve?

Unable to deploy plgd in a cloud infrastructure.

What alternatives did I consider?

None

Docs improvement

Right now I feel like the docs are lacking a bit. I know this is constant development but I wanted to know if there was any plan in improving mostly the following aspects:

Getting started guide: 2 - onboard, 3 - interact
dashboard: example auth_config.json doesn't work (not sure if it should, but gives unknown client error). An explanation on how to set up the auth_config or a description of what each parameter means
A full example, start to finish on how to set up and use the whole system (maybe even with a device set up from the getting started guide from iotivty, this way it would be a fitting follow-up)
How plgd compares to other platforms
Document reference architectures
Fix C4 diagram and dependencies between GWs and EB

I understand documentation might not be high priority right now but I thought I would create the issue either way.

Support wildcard filter in subscription APIs

Feature Request

What is the purpose of the feature?

Subscribe to all devices events without a need to list them and receive events from devices that came online after the subscription was created.

What specific challenge does this feature solve?

Without supporting wildcard subscription our client has to firstly query a list of devices and only then subscribe to events including all retrieved device ids. Additionally, this solution won't send us events of a device that was registered after the subscription was created as its id is not part of the filter.

What alternatives did I consider?

Subscribing to device registered events and updating filter of latter subscription.

[Account Linking] Infinite looping between origin and target cloud when same account who has different devices was linked in both clouds

@ondrejtomcik,

As you mentioned before, there has been an Account Linking issue related to Infinite looping between origin cloud and target cloud when the same account who has different devices was linked in both clouds each other.

I wish this issue would be solved as soon as possible.

Support JWT tokens on devices

Feature Request

What is the purpose of the feature?

The current version of the plgd cloud authorizes device connections using opaque tokens. By using JWT tokens, service tokens between plgd Gateways, RA and RD could be dropped and communication between CoAP Gateway and OAuth2.0 Server optimized.

What specific challenge does this feature solve?

reduce deployment complexity by not requiring OAuth 2.0 Service clients for plgd services
reduce the number of requests towards an OAuth 2.0 Server from the CoAP Gateway, introspecting each opaque token received from all connected devices

What alternatives did I consider?

None

Overview of user's devices on plgd Dashboard

Feature Request

What is the purpose of the feature?

To better understand how plgd cloud works, a basic UI with device overview shall be provided.

What specific challenge does this feature solve?

Get an understanding of how plgd cloud works.

What alternatives did I consider?

None

cloud2cloud-gateway subscriptions has some issues.

@jkralik @ondrejtomcik

I have tested plugged cloud with CTT for C2C API on local network.

cloud2cloud-gateway subscriptions has some sequence number issues not to increment error and to emit duplicate sequence number like 1, 2 sequence number (each 3 times) toward same device_online event.

What is problem? First of all, I want to solve "not increment sequence number" issue.

I have witnessed two kind of issue. See below logs :

2021-07-22 17:10:00 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:111
-- | --
  |   | 2021-07-22 17:10:00 | github.com/plgd-dev/cloud/grpc-gateway/client.NewDevicesSubscription.func1
  |   | 2021-07-22 17:10:00 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:159
  |   | 2021-07-22 17:10:00 | github.com/plgd-dev/cloud/grpc-gateway/client.(*DevicesSubscription).runRecv
  |   | 2021-07-22 17:10:00 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:82
  |   | 2021-07-22 17:10:00 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesOnlineHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:10:00 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:37
  |   | 2021-07-22 17:10:00 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesSubsciptionHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:10:00 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-07-22 17:10:00 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-07-22 17:10:00 | 2021-07-22T08:10:00.003Z	ERROR	log/log.go:117	devicesSubsciptionHandler.HandleDeviceMetadataUpdated: cannot emit event: cannot increment sequence number: cannot increment sequence number for subscription: cannot increment sequence number for 25c6ff5c-dba5-41f3-9219-20eb31438fbd: mongo: no documents in result


  |   | 2021-07-22 17:10:00 | 2021-07-22T08:10:00.000Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:b49c3cb7-5f37-4446-9ae5-375257227b9d URL:https://[FE80:0000:0000:0000:20D7:D98C:FBF8:7C3F]:55551/events CorrelationID:c4abf7ad-65df-4d8e-b2da-3fe726f6fafb Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:2 UserID:google-oauth2\|104342780807563784155 SigningSecret:zUppMMTAhbpTcDbUZuemivWsvLZVixKv Initialized:false}


  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:111
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/grpc-gateway/client.NewDevicesSubscription.func1
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:159
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/grpc-gateway/client.(*DevicesSubscription).runRecv
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:82
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesOnlineHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:37
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesSubsciptionHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:59 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.993Z	ERROR	log/log.go:117	devicesSubsciptionHandler.HandleDeviceMetadataUpdated: cannot emit event: cannot increment sequence number: cannot increment sequence number for subscription: cannot increment sequence number for f39dce95-6cde-4d1d-a2c6-f5270742d5b6: mongo: no documents in result


  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.991Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:5da77666-0e8a-4d85-ae61-ceca79285313 URL:https://743a2cecdc50.ngrok.io/events CorrelationID:951ff40c-436d-4de9-95b6-fba9adaddb1a Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:1 UserID:google-oauth2\|104342780807563784155 SigningSecret:qlYlEFvMODAQVgRWeaLBtUDRzbJxEKsO Initialized:false}
  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.992Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:144904a3-c2ac-4664-9e83-4945cb69d5df URL:https://743a2cecdc50.ngrok.io/events CorrelationID:5ae61304-ccdf-4e41-8bcc-1caa86e30c23 Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:2 UserID:google-oauth2\|104342780807563784155 SigningSecret:TjxnmxRyglRELZhxyXAvRPNYhgKuSwzY Initialized:false}
  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.966Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:25c6ff5c-dba5-41f3-9219-20eb31438fbd URL:https://743a2cecdc50.ngrok.io/events CorrelationID:2c5efdfa-8abc-488e-a045-36742bee6669 Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:1 UserID:google-oauth2\|104342780807563784155 SigningSecret:YqGkRgCNNvHAUSBdIZwlrTDPpgeKVQJj Initialized:false}


  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:111
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/grpc-gateway/client.NewDevicesSubscription.func1
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:159
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/grpc-gateway/client.(*DevicesSubscription).runRecv
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:82
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesOnlineHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:59 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:37
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesSubsciptionHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:59 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-07-22 17:09:59 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.940Z	ERROR	log/log.go:117	devicesSubsciptionHandler.HandleDeviceMetadataUpdated: cannot emit event: cannot increment sequence number: cannot increment sequence number for subscription: cannot increment sequence number for 760c6baa-16ed-4527-ab62-ca5994cf55f1: mongo: no documents in result


  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.930Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:f39dce95-6cde-4d1d-a2c6-f5270742d5b6 URL:https://743a2cecdc50.ngrok.io/events CorrelationID:ce23e56c-ceaa-4417-b9d0-673332f41317 Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:1 UserID:google-oauth2\|104342780807563784155 SigningSecret:zNvfQezSwszAjVKrwvznzVaUvwbXaVav Initialized:false}


  |   | 2021-07-22 17:09:59 | 2021-07-22T08:09:59.929Z	DEBUG	log/log.go:102	emitEvent: devices_online: {ID:760c6baa-16ed-4527-ab62-ca5994cf55f1 URL:https://743a2cecdc50.ngrok.io/events CorrelationID:abb0b858-518c-4130-9240-6c1dd95413a1 Type:devices Accept:[application/json] EventTypes:[devices_online] DeviceID: Href: SequenceNumber:2 UserID:google-oauth2\|104342780807563784155 SigningSecret:seSDcvwcibEhGXSzoyARAlgvapAtJSuh Initialized:false}


  |   | 2021-07-22 17:09:50 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:111
  |   | 2021-07-22 17:09:50 | github.com/plgd-dev/cloud/grpc-gateway/client.NewDevicesSubscription.func1
  |   | 2021-07-22 17:09:50 | /go/src/github.com/plgd-dev/cloud/grpc-gateway/client/devicesSubscription.go:159
  |   | 2021-07-22 17:09:50 | github.com/plgd-dev/cloud/grpc-gateway/client.(*DevicesSubscription).runRecv
  |   | 2021-07-22 17:09:50 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:93
  |   | 2021-07-22 17:09:50 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesOfflineHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:50 | /go/src/github.com/plgd-dev/cloud/cloud2cloud-gateway/service/devicesSubscriptions.go:37
  |   | 2021-07-22 17:09:50 | github.com/plgd-dev/cloud/cloud2cloud-gateway/service.(*devicesSubsciptionHandler).HandleDeviceMetadataUpdated
  |   | 2021-07-22 17:09:50 | /go/pkg/mod/github.com/plgd-dev/[email protected]/log/log.go:117
  |   | 2021-07-22 17:09:50 | github.com/plgd-dev/kit/log.Errorf
  |   | 2021-07-22 17:09:50 | 2021-07-22T08:09:50.786Z	ERROR	log/log.go:117	devicesSubsciptionHandler.HandleDeviceMetadataUpdated: cannot emit event: cannot increment sequence number: cannot increment sequence number for subscription: cannot increment sequence number for 12511240-cdb7-423a-87c3-cb3095e36e09: mongo: no documents in result


  |   | 2021-07-22 17:09:50 | 2021-07-22T08:09:50.784Z	DEBUG	log/log.go:102	emitEvent: devices_offline: {ID:12511240-cdb7-423a-87c3-cb3095e36e09 URL:https://743a2cecdc50.ngrok.io/events CorrelationID:608c3c8c-3dd4-4c14-8744-01a46c58df33 Type:devices Accept:[application/json] EventTypes:[devices_offline] DeviceID: Href: SequenceNumber:3 UserID:google-oauth2\|104342780807563784155 SigningSecret:GwGRnVwKXAfiFUOCrKLzMQgANDZFbOfJ Initialized:false}

Read & Update resources on plgd Dashboard

Feature Request

What is the purpose of the feature?

Being able to retrieve and update device resources using plgd Dashboard.

What specific challenge does this feature solve?

Easy to use.

What alternatives did I consider?

None

Coap gateway replicas parameter is being ignored with helm charts

Description

When deploying plgd via helm chart and a file to override some parameters the replicas value for COAP gateway is being ignored.
(PS: looks like it is being read from resource directory instead, because if one increases it's replicas value, them the COAP gateway also increases the number of replicas by the same amount)

How can it be reproduced?

deploy using command:
microk8s.helm3 install am-1 -f withMock.yaml plgd/plgd-hub --create-namespace --namespace=am-1 --version 2.2.2

withMock.yaml content:

global:
  domain: "host.domain.com"
  hubId: "d03a1bb4-0a77-428c-b78c-1c46efe6a38e"
mockoauthserver:
  enabled: true
coapgateway:
  replicas: 3
  port: 5683
  service:
    type: NodePort
    nodePort: 5683
  apis:
    coap:
      tls:
        enabled: false

Check deployed pods:
microk8s.kubectl get pods -A

Logs, error output, etc.

Only one plgd-hub-coap-gateway pod is running after deploy of the helm chart with replicas: 3 overridden.

Environment

microk8s v1.22.4 (snap) running on CentOS Linux release 8.5.2111
microk8s helm3 v3.5.0
plgd hub tag: 2.2.2

http-gateway websocket seems to have writing message issue.

Below log is http-gateway log when I opened http-gateway web UI running on the my local PC. And also things menu does not show anything on the page.

I used latest commit 4604d31 you did commit yesterday.

Should I avoid to run plgd cloud on local PC ? or Other problem ?
Could you look into this problem @jkralik ?

  |   | 2021-07-20 23:36:36 | time="2021-07-20T14:36:36Z" level=warning msg="[read] error writing message to upstream http server: io: read/write on closed pipe"
-- | -- | -- | --
  |   | 2021-07-20 23:36:36 |  
  |   | 2021-07-20 23:36:36 |  
  |   | 2021-07-20 23:36:36 | X-Scheme: https
  |   | 2021-07-20 23:36:36 | X-Request-Id: 3bef749e96be5efc07f39914ffe6d60a
  |   | 2021-07-20 23:36:36 | X-Real-Ip: 192.168.65.6
  |   | 2021-07-20 23:36:36 | X-Forwarded-Proto: https
  |   | 2021-07-20 23:36:36 | X-Forwarded-Port: 443
  |   | 2021-07-20 23:36:36 | X-Forwarded-Host: www.ocfcloud.com
  |   | 2021-07-20 23:36:36 | X-Forwarded-For: 192.168.65.6
  |   | 2021-07-20 23:36:36 | User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36
  |   | 2021-07-20 23:36:36 | Upgrade: websocket
  |   | 2021-07-20 23:36:36 | Sec-Websocket-Version: 13
  |   | 2021-07-20 23:36:36 | Sec-Websocket-Key: 1pFXTEEaBYPdqib7fF+/AQ==
  |   | 2021-07-20 23:36:36 | Sec-Websocket-Extensions: permessage-deflate; client_max_window_bits
  |   | 2021-07-20 23:36:36 | Pragma: no-cache
  |   | 2021-07-20 23:36:36 | Origin: https://www.ocfcloud.com
  |   | 2021-07-20 23:36:36 | Cookie: _legacy_auth0.is.authenticated=true; auth0.is.authenticated=true
  |   | 2021-07-20 23:36:36 | Connection: upgrade
  |   | 2021-07-20 23:36:36 | Cache-Control: no-cache
  |   | 2021-07-20 23:36:36 | Accept-Language: ko-KR,ko;q=0.9,en-US;q=0.8,en;q=0.7
  |   | 2021-07-20 23:36:36 | Accept-Encoding: gzip, deflate, br
  |   | 2021-07-20 23:36:36 | Host: www.ocfcloud.com
  |   | 2021-07-20 23:36:36 | 2021-07-20T14:36:36.509Z	INFO	log/log.go:107	Request: GET /api/v1/ws/devices HTTP/1.1

Fix snapshot creation when snapshotTreshold set to 1

coap-gateway does not receive accesstoken in DELETE Uri-Query of /oic/sec/account (SignOff).

@jkralik @ondrejtomcik

coap-gateway does not receive accesstoken in DELETE Uri-Query of /oic/sec/account (SignOff). It receives only di and uid even though iotivity-lite device sends full uri query which has 1015 bytes.

Is it already known issue ? So, did you get accesstoken from client connection context that device signed in ?

This is iotivity-lite logs:

DEBUG: ../../messaging/coap/coap.c <coap_serialize_array_option:248>: OPTION type 15, delta 0, len 1015, part [di=67eefa66-026a-41a8-5a6c-aa084b110fec]
DEBUG: ../../messaging/coap/coap.c <coap_serialize_options:453>: Uri-Query [accesstoken=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6Imdvb2dsZS1vYXV0aDJ8MTA0MzQyNzgwODA3NTYzNzg0MTU1IiwiYXVkIjpbImh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tL3VzZXJpbmZvIl0sImlhdCI6MTYyNzY1NzA1MSwiZXhwIjoxNjI3NzQzNDUxLCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoib3BlbmlkIHI6ZGV2aWNlaW5mb3JtYXRpb246KiByOnJlc291cmNlczoqIHc6cmVzb3VyY2VzOiogdzpzdWJzY3JpcHRpb25zOiogb2ZmbGluZV9hY2Nlc3MifQ.2rSLxMQpl3xdBoExhyCZr-5PxHBycOVvlVJQIMsDK1P3whTM62D6n4eBfgX9tqhqHUfaGLoZGn4CaPihyVWaarL7rfa1c6XTialF57Lb9ntPQM9XrmtKLJpnJFkhFbQe8RoZVoKk9YsA4uQl_vKcGTDyadOmAv7xghXMBvr_efd4x6YJ6j9Jf5y9Bh-7jx3JlVPxSAxhh1dLNGEL0nkkCy_WwF_3-ZG6O6bUeLKMCfHu5Jn-cU52juectr2W0mwJOdxc1v57aAPrz653yZAcnnHkmAljBldzxe-uYGshPaej9fwG3noSj5976hs7l3LlnZc0ZcDxw-6hrJ_vj_PopQ&uid=google-oauth2|104342780807563784155&di=67eefa66-026a-41a8-5a6c-aa084b110fec]

And this is plgd cloud logs received from iotivity-lite device:

2021-07-31 00:20:06 | 2021-07-30T15:20:06.764Z	DEBUG	log/log.go:66
-- | --
2021-07-31 00:20:06 | -------------------RECEIVED-COMMAND------------------
2021-07-31 00:20:06 | DeviceId: 67eefa66-026a-41a8-5a6c-aa084b110fec
2021-07-31 00:20:06 | Token: 0e8d579180e3d5c1
2021-07-31 00:20:06 | Path: oic/sec/account
2021-07-31 00:20:06 | Code: DELETE
2021-07-31 00:20:06 | Query: [uid=google-oauth2\|104342780807563784155 di=67eefa66-026a-41a8-5a6c-aa084b110fec]
2021-07-31 00:20:06 | Body is EMPTY
2021-07-31 00:20:06 | 2021-07-30T15:20:06.778Z	ERROR	log/log.go:117	cannot handle sign off for 67eefa66-026a-41a8-5a6c-aa084b110fec: invalid accesstoken('')
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/pkg/log.Errorf
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:117
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/coap-gateway/service.(*Client).logAndWriteErrorResponse
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/clientLogAndWriteErrorResponse.go:14
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/coap-gateway/service.signOffHandler
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signOff.go:77
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/coap-gateway/service.signUpHandler
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:83
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/coap-gateway/service.validateCommand.func1
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:266
2021-07-31 00:20:06 | github.com/plgd-dev/cloud/pkg/sync/task/queue.(*Queue).Submit.func1
2021-07-31 00:20:06 | /go/src/github.com/plgd-dev/cloud/pkg/sync/task/queue/queue.go:69
2021-07-31 00:20:06 | github.com/panjf2000/ants/v2.(*goWorker).run.func1
2021-07-31 00:20:06 | /go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:70
2021-07-31 00:20:06 | 2021-07-30T15:20:06.851Z	DEBUG	log/log.go:66
2021-07-31 00:20:06 | -------------------SEND-ERROR------------------
2021-07-31 00:20:06 | DeviceId: 67eefa66-026a-41a8-5a6c-aa084b110fec
2021-07-31 00:20:06 | Token: 0e8d579180e3d5c1
2021-07-31 00:20:06 | Path:
2021-07-31 00:20:06 | Code: BadRequest
2021-07-31 00:20:06 | Query: []
2021-07-31 00:20:06 | Error Message:
2021-07-31 00:20:06 | cannot handle sign off for 67eefa66-026a-41a8-5a6c-aa084b110fec: invalid accesstoken('')

I have modified some code to check if it gets accesstoken from SignOff Coap message.

plgd-hub-coap-gateway pod error

Description

When deploying plgd hub using helm chart (from https://charts.plgd.dev) with version 2.2.0 or 2.2.1 the pod 'plgd-hub-coap-gateway' fails to run. In version 2.1.2 this issue does not happen.

How can it be reproduced?

create withMock.yaml with following content:

global:
  domain: "host.domain.com"
  hubId: "d03a1bb4-0a77-428c-b78c-1c46efe6a38e"
mockoauthserver:
  enabled: true
coapgateway:
  port: 5683
  service:
    type: NodePort
    nodePort: 5683
  apis:
    coap:
      tls:
        enabled: false

deploy using command:
microk8s.helm3 install am-1 -f withMock.yaml plgd/plgd-hub --create-namespace --namespace=am-1 --version 2.2.1
check pods status:
microk8s.kubectl get pods -A

Logs, error output, etc.

# microk8s.kubectl get pods -A
NAMESPACE      NAME                                                  READY   STATUS             RESTARTS      AGE
kube-system    hostpath-provisioner-5c65fbdb4f-zkxdz                 1/1     Running            1 (20d ago)   21d
cert-manager   cert-manager-cainjector-7974c84449-wbh4p              1/1     Running            1 (20d ago)   21d
cert-manager   cert-manager-77fd97f598-zpp52                         1/1     Running            1 (20d ago)   21d
kube-system    calico-node-7bbrw                                     1/1     Running            1 (20d ago)   21d
kube-system    coredns-7f9c69c78c-w27d9                              1/1     Running            0             20d
kube-system    calico-kube-controllers-678fb876bf-pnfx9              1/1     Running            1 (20d ago)   21d
cert-manager   cert-manager-webhook-59d6cfd784-l9w62                 1/1     Running            1 (20d ago)   21d
ingress        nginx-ingress-microk8s-controller-sxqg8               1/1     Running            0             20d
am-1           am-1-plgd-hub-mock-oauth-server-69fb9c89f7-h2tz9      1/1     Running            0             11m
am-1           am-1-nats-0                                           3/3     Running            0             11m
am-1           mongodb-0                                             1/1     Running            0             11m
am-1           am-1-plgd-hub-resource-directory-7b96b8b8d5-g8lxj     1/1     Running            2 (10m ago)   11m
am-1           am-1-plgd-hub-identity-store-6dd67db5b5-n69fp         1/1     Running            2 (10m ago)   11m
am-1           am-1-plgd-hub-http-gateway-d778c75b-bzbjr             1/1     Running            2 (10m ago)   11m
am-1           am-1-plgd-hub-certificate-authority-c8dd7876b-9pff8   1/1     Running            2 (10m ago)   11m
am-1           am-1-plgd-hub-grpc-gateway-7979868684-pdjdh           1/1     Running            2 (10m ago)   11m
am-1           am-1-plgd-hub-resource-aggregate-7889f685b5-7wvj5     1/1     Running            2 (10m ago)   11m
am-1           mongodb-1                                             1/1     Running            0             10m
am-1           mongodb-2                                             1/1     Running            0             10m
am-1           am-1-plgd-hub-coap-gateway-6d84bfd897-lp965           0/1     CrashLoopBackOff   7 (16s ago)   11m


# microk8s.kubectl logs am-1-plgd-hub-coap-gateway-6d84bfd897-lp965 --namespace am-1
{"level":"fatal","ts":1641235717.5614443,"caller":"log/log.go:129","msg":"cannot load config: invalid config: yaml: unmarshal errors:\n  line 11: cannot unmarshal !!str `1000` into int","stacktrace":"github.com/plgd-dev/hub/pkg/log.Fatalf\n\t/go/src/github.com/plgd-dev/hub/pkg/log/log.go:129\nmain.main\n\t/go/src/github.com/plgd-dev/hub/coap-gateway/cmd/service/main.go:15\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:255"}

Environment

microk8s v1.22.4 (snap) runnning on CentOS Linux release 8.5.2111
microk8s helm3 v3.5.0
plgd hub tag: 2.2.0 and '2.2.1'

Sign-In notifications are propagated to gRPC clients even if the device didn't change its state

Device before its token expires requests a new one and calls the Sign In to keep the TCP connection authorized. This notification is internally important, but shouldn't be propagated to gRPC clients as the state of the device wasn't really changed.

Configurable device owner claim

Feature Request

What is the purpose of the feature?

Devices, before they can be used and onboarded to the cloud, are owned by some subject. In the current implementation, this subject is limited to the sub claim of the token device uses. Claim used to identify device owner shall be configurable.

What specific challenge does this feature solve?

In a multitenant system, devices might be owned by the tenant and not by the user who provisions them. Limiting owner to "sub" set the device owner always to that user and not to e.g. tenant.

What alternatives did I consider?

None, sub claim cannot be modified.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

dockerfile

Dockerfile.test

ubuntu 22.04

bundle/Dockerfile

docker/dockerfile 1

golang 1.22.3-alpine

ubuntu 22.04

http-gateway/Dockerfile

docker/dockerfile 1

golang 1.22.3-alpine

alpine 3.19

http-gateway/Dockerfile.www

docker/dockerfile 1

node 18

tools/cert-tool/Dockerfile

golang 1.22.3-alpine

alpine 3.19

tools/docker/Dockerfile.in

docker/dockerfile 1

golang 1.22.3-alpine

alpine 3.19

github-actions

.github/actions/cleanup-stale/action.yaml

snok/container-retention-policy v2

docker/setup-buildx-action v3

docker/login-action v3

Chizkiyahu/delete-untagged-ghcr-action v4

.github/workflows/build-publish-cfg.yaml

actions/checkout v4

actions/download-artifact v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/login-action v3

docker/metadata-action v5

docker/build-push-action v5

docker/build-push-action v5

.github/workflows/build-publish.yaml

actions/checkout v4

actions/upload-artifact v4

actions/checkout v4

actions/setup-node v4

anchore/sbom-action v0

.github/workflows/builds.yml

actions/checkout v4

actions/setup-go v5

.github/workflows/checkFormat.yml

actions/checkout v4

actions/setup-go v5

.github/workflows/ghcr-cleanup.yaml

snok/container-retention-policy v2

snok/container-retention-policy v2

snok/container-retention-policy v2

actions/checkout v4

.github/workflows/golangci-lint.yml

actions/checkout v4

actions/setup-go v5

golangci/golangci-lint-action v5

.github/workflows/measureMemory.yml

actions/download-artifact v4

actions/checkout v4

actions/upload-artifact v4

.github/workflows/publishDockerImagesGhcr.yml

actions/checkout v4

docker/setup-qemu-action v3

docker/setup-buildx-action v3

docker/login-action v3

docker/metadata-action v5

docker/build-push-action v5

.github/workflows/releaseHelmCharts.yml

actions/checkout v4

.github/workflows/run-bundle.yml

actions/checkout v4

luisboto/ngrok-tunnel-action v0.1.7.2

docker/metadata-action v5

actions/upload-artifact v4

actions/upload-artifact v4

.github/workflows/staticAnalysis.yml

actions/checkout v4

actions/setup-go v5

.github/workflows/test.yml

actions/checkout v4

docker/login-action v3

actions/upload-artifact v4

actions/checkout v4

actions/download-artifact v4

http-gateway/web/playwright/.github/workflows/playwright.yml

actions/checkout v4

actions/setup-node v4

actions/upload-artifact v4

gomod

go.mod

go 1.22

go 1.22.0

github.com/favadi/protoc-go-inject-tag v1.4.0

github.com/felixge/httpsnoop v1.0.4

github.com/fsnotify/fsnotify v1.7.0

github.com/fullstorydev/grpchan v1.1.1

github.com/fxamacker/cbor/v2 v2.6.0

github.com/go-co-op/gocron/v2 v2.3.0

github.com/gocql/gocql v1.6.0

github.com/golang-jwt/jwt/v5 v5.2.1

github.com/golang/snappy v0.0.4

github.com/google/go-querystring v1.1.0

github.com/google/uuid v1.6.0

github.com/gorilla/mux v1.8.1

github.com/gorilla/websocket v1.5.1

github.com/grpc-ecosystem/go-grpc-middleware v1.4.0

github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1

github.com/hashicorp/go-multierror v1.1.1

github.com/jessevdk/go-flags v1.5.0

github.com/json-iterator/go v1.1.12

github.com/jtacoma/uritemplates v1.0.0

github.com/karrick/tparse/v2 v2.8.2

github.com/lestrrat-go/jwx/v2 v2.0.21

github.com/nats-io/nats.go v1.34.1

github.com/panjf2000/ants/v2 v2.9.1

github.com/pion/dtls/v2 v2.2.8-0.20240501061905-2c36d63320a0@2c36d63320a0

github.com/pion/logging v0.2.2

github.com/plgd-dev/device/v2 v2.5.1-0.20240513064831-b553d1a87e1c@b553d1a87e1c

github.com/plgd-dev/go-coap/v3 v3.3.4

github.com/plgd-dev/kit/v2 v2.0.0-20211006190727-057b33161b90@057b33161b90

github.com/pseudomuto/protoc-gen-doc v1.5.1

github.com/sirupsen/logrus v1.9.3

github.com/stretchr/testify v1.9.0

github.com/tidwall/gjson v1.17.1

github.com/tidwall/sjson v1.2.5

github.com/ugorji/go/codec v1.2.12

github.com/vincent-petithory/dataurl v1.0.0

github.com/web-of-things-open-source/thingdescription-go v0.0.0-20240510130416-741fef736e1e@741fef736e1e

go.mongodb.org/mongo-driver v1.15.0

go.opentelemetry.io/contrib/instrumentation/go.mongodb.org/mongo-driver/mongo/otelmongo v0.49.0

go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0

go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0

go.opentelemetry.io/otel v1.24.0

go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0

go.opentelemetry.io/otel/metric v1.24.0

go.opentelemetry.io/otel/sdk v1.24.0

go.opentelemetry.io/otel/trace v1.24.0

go.uber.org/atomic v1.11.0

go.uber.org/zap v1.27.0

golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f@fe59bbe5cc7f

golang.org/x/net v0.24.0

golang.org/x/oauth2 v0.19.0

golang.org/x/sync v0.7.0

google.golang.org/genproto/googleapis/api v0.0.0-20240429193739-8cf5692501f6@8cf5692501f6

google.golang.org/genproto/googleapis/rpc v0.0.0-20240429193739-8cf5692501f6@8cf5692501f6

google.golang.org/grpc v1.63.2

google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.3.0

google.golang.org/protobuf v1.34.0

gopkg.in/yaml.v3 v3.0.1

github.com/pion/dtls/v2 v2.2.8-0.20240501061905-2c36d63320a0@2c36d63320a0

github.com/youmark/pkcs8 v0.0.0-20201027041543-1326539a0a0a@1326539a0a0a

go.opentelemetry.io/contrib/instrumentation/go.mongodb.org/mongo-driver/mongo/otelmongo v0.49.0

go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0

go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.49.0

go.opentelemetry.io/otel v1.24.0

go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0

go.opentelemetry.io/otel/metric v1.24.0

go.opentelemetry.io/otel/sdk v1.24.0

go.opentelemetry.io/otel/trace v1.24.0

helm-values

charts/plgd-hub/values.yaml

docker.io/bitnami/mongodb 5.0.10-debian-11-r3

helmv3

charts/plgd-hub/Chart.yaml

nats 1.1.9

mongodb 13.18.3

scylla 1.10.0

npm

http-gateway/web/package.json

http-gateway/web/packages/mock-server/package.json

http-gateway/web/packages/ws-mocker/package.json

http-gateway/web/playwright/package.json

Check this box to trigger a request for Renovate to run again on this repository

[coap-gateway] all coap client connection was always closed whenever keep alive was reached.

@jkralik @ondrejtomcik

Description

All coap client connection was always closed whenever keep alive was reached even if coap client like cloud server, server certification test sample of iotivity-lite jakarta branch works fine. Because of that, cloud server and the other devices are offline and online repeatably.

What is your keep-alive logic of coap-gateway? Why this happen to client connection?

Logs, error output, etc.

2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  close client 211.204.163.221:36818 for device 783afcdb-5f94-41a7-7325-5c732a9f433b
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36818 for device 783afcdb-5f94-41a7-7325-5c732a9f433b
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cannot cancel resource('2713892058') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cannot cancel resource('757679238') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cannot cancel resource('318534269') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cannot cancel resource('136750592') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:29:28.985Z        DEBUG   log/log.go:108  cannot cancel resource('510901701') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:29:28.986Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36818, device , user 
2021-08-23T12:29:28.996Z        DEBUG   log/log.go:108  plgd/go-coap: tcp: 211.204.163.221:36818: cannot read from connection: EOF
2021-08-23T12:31:13.019Z        ERROR   log/log.go:98   DeviceId: 0882f552-a331-4a49-7d1e-a9280e100f2d: keep alive was reached fail limit:: closing connection
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.New.func8
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:189
github.com/plgd-dev/go-coap/v2/net/monitor/inactivity.(*KeepAlive).OnInactive
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/net/monitor/inactivity/keepalive.go:34
github.com/plgd-dev/go-coap/v2/net/monitor/inactivity.(*inactivityMonitor).CheckInactivity
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/net/monitor/inactivity/inactivitymonitor.go:57
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2.1
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:216
github.com/plgd-dev/go-coap/v2/net.(*Conn).ReadWithContext
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/net/conn.go:205
github.com/plgd-dev/go-coap/v2/tcp.(*Session).Run
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/session.go:369
github.com/plgd-dev/go-coap/v2/tcp.(*ClientConn).Run
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/clientconn.go:472
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:228
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  close client 211.204.163.221:36676 for device 0882f552-a331-4a49-7d1e-a9280e100f2d
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36676 for device 0882f552-a331-4a49-7d1e-a9280e100f2d
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  cannot cancel resource('2946766110') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  cannot cancel resource('510901701') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  cannot cancel resource('2058517680') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  cannot cancel resource('3878770302') observation: cannot write request: cannot write to connection: context canceled
2021-08-23T12:31:13.019Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36676, device , user 
2021-08-23T12:31:40.513Z        DEBUG   log/log.go:108  close client 211.204.163.221:36872 for device 
2021-08-23T12:31:40.513Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36872 for device 
2021-08-23T12:31:40.513Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36872, device , user 
2021-08-23T12:31:40.514Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36872: cannot send CSM: cannot write to connection: cannot TLS handshake: tls: no cipher suite supported by both client and server
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:31:49.555Z        DEBUG   log/log.go:108  close client 211.204.163.221:36874 for device 
2021-08-23T12:31:49.556Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36874 for device 
2021-08-23T12:31:49.556Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36874, device , user 
2021-08-23T12:31:49.556Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36874: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36874: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:31:52.932Z        DEBUG   log/log.go:108  close client 211.204.163.221:36876 for device 
2021-08-23T12:31:52.932Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36876 for device 
2021-08-23T12:31:52.932Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36876, device , user 
2021-08-23T12:31:52.932Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36876: cannot send CSM: cannot write to connection: cannot TLS handshake: tls: no cipher suite supported by both client and server
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:32:00.242Z        DEBUG   log/log.go:108  close client 211.204.163.221:36878 for device 
2021-08-23T12:32:00.242Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36878 for device 
2021-08-23T12:32:00.242Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36878, device , user 
2021-08-23T12:32:00.243Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36878: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36878: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:32:10.567Z        DEBUG   log/log.go:108  close client 211.204.163.221:36880 for device 
2021-08-23T12:32:10.568Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36880 for device 
2021-08-23T12:32:10.568Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36880, device , user 
2021-08-23T12:32:10.568Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36880: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36880: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:33:09.847Z        DEBUG   log/log.go:108  close client 211.204.163.221:36890 for device 
2021-08-23T12:33:09.847Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36890 for device 
2021-08-23T12:33:09.847Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36890, device , user 
2021-08-23T12:33:09.847Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36890: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36890: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:33:38.869Z        DEBUG   log/log.go:108  close client 211.204.163.221:36892 for device 
2021-08-23T12:33:38.869Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36892 for device 
2021-08-23T12:33:38.869Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36892, device , user 
2021-08-23T12:33:38.869Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36892: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36892: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:33:47.088Z        DEBUG   log/log.go:108  close client 211.204.163.221:36896 for device 
2021-08-23T12:33:47.088Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36896 for device 
2021-08-23T12:33:47.088Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36896, device , user 
2021-08-23T12:33:47.089Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36896: cannot send CSM: cannot write to connection: cannot TLS handshake: tls: received unexpected handshake message of type *tls.certificateMsg when waiting for *tls.clientHelloMsg
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:33:59.216Z        DEBUG   log/log.go:108  close client 211.204.163.221:36900 for device 
2021-08-23T12:33:59.217Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36900 for device 
2021-08-23T12:33:59.218Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36900, device , user 
2021-08-23T12:33:59.219Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36900: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36900: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:34:00.321Z        DEBUG   log/log.go:108  close client 211.204.163.221:36902 for device 
2021-08-23T12:34:00.321Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36902 for device 
2021-08-23T12:34:00.321Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36902, device , user 
2021-08-23T12:34:00.321Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36902: cannot send CSM: cannot write to connection: cannot TLS handshake: tls: no cipher suite supported by both client and server
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:34:10.425Z        DEBUG   log/log.go:108  close client 211.204.163.221:36904 for device 
2021-08-23T12:34:10.425Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36904 for device 
2021-08-23T12:34:10.425Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36904, device , user 
2021-08-23T12:34:10.425Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36904: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36904: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:34:22.318Z        DEBUG   log/log.go:108  close client 211.204.163.221:36908 for device 
2021-08-23T12:34:22.319Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36908 for device 
2021-08-23T12:34:22.319Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36908, device , user 
2021-08-23T12:34:22.319Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36908: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36908: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:34:59.959Z        DEBUG   log/log.go:108  close client 211.204.163.221:36912 for device 
2021-08-23T12:34:59.960Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36912 for device 
2021-08-23T12:34:59.960Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36912, device , user 
2021-08-23T12:34:59.960Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36912: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36912: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:35:18.219Z        DEBUG   log/log.go:108  close client 211.204.163.221:36918 for device 
2021-08-23T12:35:18.219Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36918 for device 
2021-08-23T12:35:18.219Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36918, device , user 
2021-08-23T12:35:18.219Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36918: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36918: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:36:17.899Z        DEBUG   log/log.go:108  close client 211.204.163.221:36924 for device 
2021-08-23T12:36:17.899Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36924 for device 
2021-08-23T12:36:17.900Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36924, device , user 
2021-08-23T12:36:17.900Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36924: cannot send CSM: cannot write to connection: cannot TLS handshake: read tcp 172.19.0.8:5684->211.204.163.221:36924: i/o timeout
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:36:24.232Z        DEBUG   log/log.go:108  close client 211.204.163.221:36926 for device 
2021-08-23T12:36:24.233Z        DEBUG   log/log.go:108  cleanUp client 211.204.163.221:36926 for device 
2021-08-23T12:36:24.233Z        DEBUG   log/log.go:108  Authorization context replaced for client 211.204.163.221:36926, device , user 
2021-08-23T12:36:24.233Z        ERROR   log/log.go:98   plgd/go-coap: tcp: 211.204.163.221:36926: cannot send CSM: cannot write to connection: cannot TLS handshake: tls: no cipher suite supported by both client and server
github.com/plgd-dev/cloud/pkg/log.Error
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
        /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
github.com/plgd-dev/cloud/pkg/log.Errorf
        /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
github.com/plgd-dev/cloud/coap-gateway/service.(*Service).setupCoapServer.func7
        /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:430
github.com/plgd-dev/go-coap/v2/tcp.NewServer.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:136
github.com/plgd-dev/go-coap/v2/tcp.(*Server).Serve.func2
        /go/pkg/mod/github.com/plgd-dev/go-coap/[email protected]/tcp/server.go:230
2021-08-23T12:36:38.123Z        DEBUG   log/log.go:83

Environment

plgd source : commit e019708
IoTivity version: jakarta branch
Deployment option: bundle on my own

Allow user to disable device twin synchronization (resource shadow)

Feature Request

What is the purpose of the feature?

Support mode when the application is not interested in changes produced by the device. The application shall be able to stop observation - the creation of the device/resource shadow.

What specific challenge does this feature solve?

The device in case it's in maintenance mode (application-specific mode) produces thousands of changes. If the device would be observed by the plgd cloud during the maintenance, plgd cloud would have to handle and store thousands of non-relevant messages. This is what we want to prevent.

What alternatives did I consider?

Disconnect the device and connect it again, but then we cannot configure it remotely.

Support device name change

Feature Request

What is the purpose of the feature?

User shall be able to change the device name in the plgd Dashboard.

What specific challenge does this feature solve?

If a user has more than one device and they have always the same name, it's hard to identify the one she wants to configure.

What alternatives did I consider?

None

problem receiving notifications by GRPC

Hi,

Could you please advise if there are changes from release v2next-77de0a1 ( itself this version doesn't even let connection for GRPC as far as I could test it )

v2next-d387ac2 - does not trigger notifications automatically on empty startup
v2next-77de0a1 - does not connect to grpc
v2next-b88a2ca - worked

behaviour:
1)start a PLGD bundle from scratch.
2)launch secure devices
3)launch a GRPCclient (not using the certificate). Create a subscription for Online device status change
4)onboard devices using the SDK provided by the team

expected: online notification is caught by the subscription created ( tested up to v2next-b88a2ca with success)

observed: if no device existed before online notification is not triggered from v2next-d387ac2 up to latest. If the GRPC client subscription is launched after a device was onboarded/offboarded it will correctly receive all notifications

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Support Creation & Deletion of device resources

Feature Request

What is the purpose of the feature?

Being able to create and delete resources directly in the UI.

What specific challenge does this feature solve?

Easy to use.

What alternatives did I consider?

None

Cancellation of pending commands

Feature Request

What is the purpose of the feature?

UPDATE, CREATE, DELETE and RETRIEVE_FROM_DEVICE commands are registered as pending commands and will stay there forever if the device is offline. We would like to be able to cancel these pending commands using plgd gRPC / HTTP Gateways.

What specific challenge does this feature solve?

Command became invalid for some reason and we need a possibility to cancel it before device comes online and processes it.

What alternatives did I consider?

Track if the device is online and only then send the command, but the problem is that if we send the command, the device can at the same time go offline.

[cloud2cloud-connector] Account Linking API cannot exchange origin cloud authorization code caused by oauth 403 forbidden error.

Description

@ondrejtomcik @jkralik @Danielius1922
[cloud2cloud-connector] Account Linking API cannot exchange origin cloud authorization code caused by oauth 403 forbidden error. This is because handleOAuth() function of account linking API (addLinkedAccount.go) was not handle properly. The logic exchanges to origin cloud oauth provider with auth code of target cloud now. So you should change the logic in handleOAuth() in the addLinkedAccount.go to fixed codes below :

Error Logs, error output, etc.

cannot process oauth callback: cannot exchange origin cloud authorization code for access token: oauth2: cannot fetch token: 403 Forbidden
Response: {"error":"invalid_grant","error_description":"Invalid authorization code"}

Fixed Codes for proper OAuth exchange for both origin/target cloud account linking

func (rh *RequestHandler) handleOAuth(w http.ResponseWriter, r *http.Request, linkedAccount store.LinkedAccount, linkedCloud store.LinkedCloud) (int, error) {
	linkedCloud, ok := rh.store.LoadCloud(linkedAccount.LinkedCloudID)
	if !ok {
		return http.StatusBadRequest, fmt.Errorf("cannot find linked cloud with ID %v: not found", linkedAccount.LinkedCloudID)
	}
	t, err := generateRandomString(32)
	if err != nil {
		return http.StatusInternalServerError, fmt.Errorf("cannot generate token")
	}
	err = rh.provisionCache.Add(t, provisionCacheData{
		linkedAccount: linkedAccount,
		linkedCloud:   linkedCloud,
	}, cache.DefaultExpiration)
	if err != nil {
		return http.StatusInternalServerError, fmt.Errorf("cannot store key - collision")
	}

	if !linkedAccount.Data.HasOrigin() {
		url := rh.provider.OAuth2.AuthCodeURL(t)
		http.Redirect(w, r, url, http.StatusTemporaryRedirect)
	}

	if !linkedAccount.Data.HasTarget() {
		oauthCfg := linkedCloud.OAuth
		if oauthCfg.RedirectURL == "" {
			oauthCfg.RedirectURL = rh.provider.Config.RedirectURL
		}
		url := oauthCfg.AuthCodeURL(t)
		http.Redirect(w, r, url, http.StatusTemporaryRedirect)
	}
	return http.StatusOK, nil
}

Environment

plgd source: commit ba6d959
Deployment option: own deployment for each service

Additional context

Use MongoDB bucket pattern in the EventStore

Feature Request

What is the purpose of the feature?

The way how the current MongoDB model schema looks is very inefficient. It's hard to scale and due to mapping each event to document the number of file descriptors grows to thousands. After having few sessions with the MongoDB team the goal is to improve the schema by changing it to a bucket pattern.

What specific challenge does this feature solve?

Scalability, footprint, and performance improvements.

What alternatives did I consider?

None

Configurable command timeout

Feature Request

What is the purpose of the feature?

UPDATE, CREATE, DELETE and RETRIEVE_FROM_DEVICE commands are registered as pending commands and will stay there forever if the device is offline. We would like to be able to configure the command time to live, so it expires at some point and won't stay as a pending one forever.

What specific challenge does this feature solve?

Update command we send to the device shall expire if the device is offline for more than 1 minute.

What alternatives did I consider?

Cancel the command on our own after one minute, but there is currently no API to cancel the pending command.

Introduce new /resources endpoint

Feature Request

What is the purpose of the feature?

The /devices endpoint limits the client to retrieve resources only of a specific device. As the gRPC gateway supports retrieval of resources filtered by the resource type or href, the same functionality shall be added also to the HTTP gateway.

What specific challenge does this feature solve?

Retrieve of resources of a specific type from all devices connected to the plgd.

What alternatives did I consider?

None

[coap-gateway] coap gateway receive duplicate command from cloud server of iotivity-lite which causes invalid authorization code when device registration.

@ondrejtomcik @jkralik

I found weired situation on plgd coap-gateway.
Registration of device does not work properly and does not save access token, refresh token on user devices table.

Description

When I tried to register device, Post /oic/sec/account send proper response with access token, refresh token, uid at first but in a row, it sends error response to device about invalid authorization code. I think there seems to be some logic to exchange token twice in a request of sign up.

More over, access token and refresh token was not saved in user devices table on mongo db after using latest plgd open source (commit 0f2c187).

See below second token exchange and error response sending logs:

  |   | 2021-08-13 02:14:45 | 2021-08-12T17:14:45.061Z	ERROR	log/log.go:98	cannot handle sign up: oauth2: cannot fetch token: 403 Forbidden
  |   | 2021-08-13 02:14:45 | Response: {"error":"invalid_grant","error_description":"Invalid authorization code"}
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.Error
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.Errorf
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.(*Client).logAndWriteErrorResponse
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/clientLogAndWriteErrorResponse.go:36
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpPostHandler.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:72
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpPostHandler
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:94
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpHandler
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:127
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.validateCommand.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:290
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/sync/task/queue.(*Queue).Submit.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/sync/task/queue/queue.go:68
  |   | 2021-08-13 02:14:45 | github.com/panjf2000/ants/v2.(*goWorker).run.func1
  |   | 2021-08-13 02:14:45 | /go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:70
  |   | 2021-08-13 02:14:45 | 2021-08-12T17:14:45.066Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:45 | -------------------SEND-ERROR------------------
  |   | 2021-08-13 02:14:45 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:45 | Token: 7c8f474fa0db85e3
  |   | 2021-08-13 02:14:45 | Path:
  |   | 2021-08-13 02:14:45 | Code: Unauthorized
  |   | 2021-08-13 02:14:45 | Query: []
  |   | 2021-08-13 02:14:45 | error Message:
  |   | 2021-08-13 02:14:45 | cannot handle sign up: oauth2: cannot fetch token: 403 Forbidden
  |   | 2021-08-13 02:14:45 | Response: {"error":"invalid_grant","error_description":"Invalid authorization code"}

This is full logs from sign-up to resource publish

2021-08-13 02:14:44 | 2021-08-12T17:14:44.161Z	DEBUG	log/log.go:83
-- | --
  |   | 2021-08-13 02:14:44 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:44 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:44 | Token: 860f28cdf0f3c0d6
  |   | 2021-08-13 02:14:44 | Path: oic/sec/account
  |   | 2021-08-13 02:14:44 | Code: POST
  |   | 2021-08-13 02:14:44 | Query: []
  |   | 2021-08-13 02:14:44 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:44 | CBOR:
  |   | 2021-08-13 02:14:44 | {"accesstoken":"bJ4ywwOl9Kv-r_mI","authprovider":"auth0","devicetype":"device","di":"6ec91ff4-60e9-47db-6257-3f05c6867e10"}
  |   | 2021-08-13 02:14:44 | 2021-08-12T17:14:44.651Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:44 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:44 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:44 | Token: 7c8f474fa0db85e3
  |   | 2021-08-13 02:14:44 | Path: oic/sec/account
  |   | 2021-08-13 02:14:44 | Code: POST
  |   | 2021-08-13 02:14:44 | Query: []
  |   | 2021-08-13 02:14:44 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:44 | CBOR:
  |   | 2021-08-13 02:14:44 | {"accesstoken":"bJ4ywwOl9Kv-r_mI","authprovider":"auth0","devicetype":"device","di":"6ec91ff4-60e9-47db-6257-3f05c6867e10"}
  |   | 2021-08-13 02:14:44 | 2021-08-12T17:14:44.989Z	INFO	zap/options.go:212	finished unary call with code OK	{"grpc.start_time": "2021-08-12T17:14:44Z", "system": "grpc", "span.kind": "server", "grpc.service": "ocf.cloud.auth.pb.AuthorizationService", "grpc.method": "AddDevice", "peer.address": "10.1.3.0:34394", "grpc.code": "OK", "grpc.time_ms": 6.389999866485596}
  |   | 2021-08-13 02:14:44 | 2021-08-12T17:14:44.992Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:44 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:44 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:44 | Token: 860f28cdf0f3c0d6
  |   | 2021-08-13 02:14:44 | Path:
  |   | 2021-08-13 02:14:44 | Code: Changed
  |   | 2021-08-13 02:14:44 | Query: []
  |   | 2021-08-13 02:14:44 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:44 | CBOR:
  |   | 2021-08-13 02:14:44 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ2NCwiZXhwIjoxNjI4ODc0ODY0LCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.zZqbbNpyc2tbwVr2jCEs9MjyajvL06A0da0CirJdAfsiR3fMPSdAsz7F39hkskSIjN1xSYY7KRl4NDDli5EABKHnSeXGOUnA0NCV1ILgLA8J-F0gwq_kYyVwJQ5oQ6L97nO2uZ5RpXxqkmQ8jBdTdR9_aD2dNccLRxxHxqbNN7ZomkgRJC13RhRv_SY2oZdxw8GjBOZQt3cs-PdQKJP0P0ZOWSx8l_ftAXniHZxS8JltmyBtYjWHfe-KMrAOCT6WceLqgO1ctnAPF0hK_1cSP7CetKNxkL55zG7jq-d1mbp7V_5uQIJcYUrz85xuWswIEJF135oT0oq5kd1J1iDTlw","expiresin":86399,"redirecturi":"","refreshtoken":"v1.MXnl0zc_bmZtpRJfIA1GeylefPnaenUaWvZY-sqOqLwjhSYRtXmBFz-9R18fnChVZYDETv35j57QjWb3EX2-WvM","uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:45 | 2021-08-12T17:14:45.061Z	ERROR	log/log.go:98	cannot handle sign up: oauth2: cannot fetch token: 403 Forbidden
  |   | 2021-08-13 02:14:45 | Response: {"error":"invalid_grant","error_description":"Invalid authorization code"}
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.Error
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:98
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.LogAndReturnError
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/logAndReturnError.go:35
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/log.Errorf
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/log/log.go:124
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.(*Client).logAndWriteErrorResponse
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/clientLogAndWriteErrorResponse.go:36
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpPostHandler.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:72
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpPostHandler
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:94
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.signUpHandler
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/signUp.go:127
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/coap-gateway/service.validateCommand.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/coap-gateway/service/service.go:290
  |   | 2021-08-13 02:14:45 | github.com/plgd-dev/cloud/pkg/sync/task/queue.(*Queue).Submit.func1
  |   | 2021-08-13 02:14:45 | /go/src/github.com/plgd-dev/cloud/pkg/sync/task/queue/queue.go:68
  |   | 2021-08-13 02:14:45 | github.com/panjf2000/ants/v2.(*goWorker).run.func1
  |   | 2021-08-13 02:14:45 | /go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:70
  |   | 2021-08-13 02:14:45 | 2021-08-12T17:14:45.066Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:45 | -------------------SEND-ERROR------------------
  |   | 2021-08-13 02:14:45 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:45 | Token: 7c8f474fa0db85e3
  |   | 2021-08-13 02:14:45 | Path:
  |   | 2021-08-13 02:14:45 | Code: Unauthorized
  |   | 2021-08-13 02:14:45 | Query: []
  |   | 2021-08-13 02:14:45 | error Message:
  |   | 2021-08-13 02:14:45 | cannot handle sign up: oauth2: cannot fetch token: 403 Forbidden
  |   | 2021-08-13 02:14:45 | Response: {"error":"invalid_grant","error_description":"Invalid authorization code"}
  |   | 2021-08-13 02:14:45 | 2021-08-12T17:14:45.461Z	INFO	zap/options.go:212	finished streaming call with code OK	{"grpc.start_time": "2021-08-12T17:14:45Z", "grpc.request.deadline": "2021-08-12T17:15:00Z", "system": "grpc", "span.kind": "server", "grpc.service": "ocf.cloud.auth.pb.AuthorizationService", "grpc.method": "GetUserDevices", "peer.address": "10.1.2.248:43570", "grpc.code": "OK", "grpc.time_ms": 7.1020002365112305}
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.032Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:46 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:46 | DeviceId: unknown(10.1.2.223:54490)
  |   | 2021-08-13 02:14:46 | Token: ed9734cfe02f335b
  |   | 2021-08-13 02:14:46 | Path: oic/sec/session
  |   | 2021-08-13 02:14:46 | Code: POST
  |   | 2021-08-13 02:14:46 | Query: []
  |   | 2021-08-13 02:14:46 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:46 | CBOR:
  |   | 2021-08-13 02:14:46 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ2NCwiZXhwIjoxNjI4ODc0ODY0LCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.zZqbbNpyc2tbwVr2jCEs9MjyajvL06A0da0CirJdAfsiR3fMPSdAsz7F39hkskSIjN1xSYY7KRl4NDDli5EABKHnSeXGOUnA0NCV1ILgLA8J-F0gwq_kYyVwJQ5oQ6L97nO2uZ5RpXxqkmQ8jBdTdR9_aD2dNccLRxxHxqbNN7ZomkgRJC13RhRv_SY2oZdxw8GjBOZQt3cs-PdQKJP0P0ZOWSx8l_ftAXniHZxS8JltmyBtYjWHfe-KMrAOCT6WceLqgO1ctnAPF0hK_1cSP7CetKNxkL55zG7jq-d1mbp7V_5uQIJcYUrz85xuWswIEJF135oT0oq5kd1J1iDTlw","di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","login":true,"uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.032Z	DEBUG	log/log.go:108	close client 10.1.2.223:54490 for device
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.033Z	DEBUG	log/log.go:108	cleanUp client 10.1.2.223:54490 for device
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.033Z	DEBUG	log/log.go:108	Authorization context replaced for client 10.1.2.223:54490, device , user
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.037Z	DEBUG	log/log.go:108	Authorization context replaced for client 10.1.2.223:54490, device 6ec91ff4-60e9-47db-6257-3f05c6867e10, user auth0\|6112467f3a83a80070bc699d
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.037Z	DEBUG	log/log.go:108	temporary error: cannot handle sign in: cannot update cloud device status: rpc error: code = Canceled desc = context canceled
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.039Z	DEBUG	log/log.go:108	cannot send error to 6ec91ff4-60e9-47db-6257-3f05c6867e10: cannot write to connection: context canceled
  |   | 2021-08-13 02:14:46 | 2021-08-12T17:14:46.039Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:46 | -------------------SEND-ERROR------------------
  |   | 2021-08-13 02:14:46 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:46 | Token: ed9734cfe02f335b
  |   | 2021-08-13 02:14:46 | Path:
  |   | 2021-08-13 02:14:46 | Code: ServiceUnavailable
  |   | 2021-08-13 02:14:46 | Query: []
  |   | 2021-08-13 02:14:46 | error Message:
  |   | 2021-08-13 02:14:46 | temporary error: cannot handle sign in: cannot update cloud device status: rpc error: code = Canceled desc = context canceled
  |   | 2021-08-13 02:14:51 | 2021-08-12T17:14:51.402Z	INFO	zap/options.go:212	finished streaming call with code OK	{"grpc.start_time": "2021-08-12T17:14:51Z", "grpc.request.deadline": "2021-08-12T17:15:06Z", "system": "grpc", "span.kind": "server", "grpc.service": "ocf.cloud.auth.pb.AuthorizationService", "grpc.method": "GetUserDevices", "peer.address": "10.1.2.249:60340", "grpc.code": "OK", "grpc.time_ms": 1.9819999933242798}
  |   | 2021-08-13 02:14:51 | 2021-08-12T17:14:51.645Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:51 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:51 | DeviceId: unknown(10.1.2.223:54580)
  |   | 2021-08-13 02:14:51 | Token: 9d37c28ee1a53eb2
  |   | 2021-08-13 02:14:51 | Path: oic/sec/tokenrefresh
  |   | 2021-08-13 02:14:51 | Code: POST
  |   | 2021-08-13 02:14:51 | Query: []
  |   | 2021-08-13 02:14:51 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:51 | CBOR:
  |   | 2021-08-13 02:14:51 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","granttype":"refresh_token","refreshtoken":"v1.MXnl0zc_bmZtpRJfIA1GeylefPnaenUaWvZY-sqOqLwjhSYRtXmBFz-9R18fnChVZYDETv35j57QjWb3EX2-WvM","uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:52 | 2021-08-12T17:14:52.137Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:52 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:52 | DeviceId: unknown(10.1.2.223:54580)
  |   | 2021-08-13 02:14:52 | Token: 255fc8ea50c9d196
  |   | 2021-08-13 02:14:52 | Path: oic/sec/tokenrefresh
  |   | 2021-08-13 02:14:52 | Code: POST
  |   | 2021-08-13 02:14:52 | Query: []
  |   | 2021-08-13 02:14:52 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:52 | CBOR:
  |   | 2021-08-13 02:14:52 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","granttype":"refresh_token","refreshtoken":"v1.MXnl0zc_bmZtpRJfIA1GeylefPnaenUaWvZY-sqOqLwjhSYRtXmBFz-9R18fnChVZYDETv35j57QjWb3EX2-WvM","uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:52 | 2021-08-12T17:14:52.149Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:52 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:52 | DeviceId: unknown(10.1.2.223:54580)
  |   | 2021-08-13 02:14:52 | Token: 9d37c28ee1a53eb2
  |   | 2021-08-13 02:14:52 | Path:
  |   | 2021-08-13 02:14:52 | Code: Changed
  |   | 2021-08-13 02:14:52 | Query: []
  |   | 2021-08-13 02:14:52 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:52 | CBOR:
  |   | 2021-08-13 02:14:52 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ3MSwiZXhwIjoxNjI4ODc0ODcxLCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.vYrpnKLzxx0kurIZd3VI8fu93Tb5LyP85H8iDQtYkhuGwVNi-nmFr_h0GQoucssIvFkjEsKi9_HYVtO2duhlLAWpiFHqfV24NqymynpllUCAETicdtg6_OV1j4G02GRefPL-wN1EGHutHnG2h8UMxfphwA70PgO29eVmz49EXPJMsbj7l8DfhFvBPOEOfyXgYq-_JDVO52JTFeq1lK7CC53OM76TA2wIkdt5wSDKY5NT3vbiO47Sda0vz86d-LcGCkbUg3_iyPV5NeupFLGKASk1QwfiT9bG4rCHpPJs3a9Pomrn3xw88AtTeXmSLc6dCr2gutP4KP85qM3alPGMzQ","expiresin":86399,"refreshtoken":"v1.Mnnl0zc_bmZtpRJfIA1Geyma8mI209786Qy-cGMyBXBK9Dxn7LML3pIEmrmUsieI5Ah2s5wXZS1HxWgIgybmDtg"}
  |   | 2021-08-13 02:14:52 | 2021-08-12T17:14:52.804Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:52 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:52 | DeviceId: unknown(10.1.2.223:54580)
  |   | 2021-08-13 02:14:52 | Token: 255fc8ea50c9d196
  |   | 2021-08-13 02:14:52 | Path:
  |   | 2021-08-13 02:14:52 | Code: Changed
  |   | 2021-08-13 02:14:52 | Query: []
  |   | 2021-08-13 02:14:52 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:52 | CBOR:
  |   | 2021-08-13 02:14:52 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ3MSwiZXhwIjoxNjI4ODc0ODcxLCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.vYrpnKLzxx0kurIZd3VI8fu93Tb5LyP85H8iDQtYkhuGwVNi-nmFr_h0GQoucssIvFkjEsKi9_HYVtO2duhlLAWpiFHqfV24NqymynpllUCAETicdtg6_OV1j4G02GRefPL-wN1EGHutHnG2h8UMxfphwA70PgO29eVmz49EXPJMsbj7l8DfhFvBPOEOfyXgYq-_JDVO52JTFeq1lK7CC53OM76TA2wIkdt5wSDKY5NT3vbiO47Sda0vz86d-LcGCkbUg3_iyPV5NeupFLGKASk1QwfiT9bG4rCHpPJs3a9Pomrn3xw88AtTeXmSLc6dCr2gutP4KP85qM3alPGMzQ","expiresin":86399,"refreshtoken":"v1.Mnnl0zc_bmZtpRJfIA1GeyktJqLcGHiPt9sdaDV_OQE0mrblhtlJa6-SRlGFfsLgkNWMLcmrIP237l0Xd-L0Hps"}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.218Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:55 | DeviceId: unknown(10.1.2.223:54580)
  |   | 2021-08-13 02:14:55 | Token: 2a9c76708d96ff7f
  |   | 2021-08-13 02:14:55 | Path: oic/sec/session
  |   | 2021-08-13 02:14:55 | Code: POST
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ3MSwiZXhwIjoxNjI4ODc0ODcxLCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.vYrpnKLzxx0kurIZd3VI8fu93Tb5LyP85H8iDQtYkhuGwVNi-nmFr_h0GQoucssIvFkjEsKi9_HYVtO2duhlLAWpiFHqfV24NqymynpllUCAETicdtg6_OV1j4G02GRefPL-wN1EGHutHnG2h8UMxfphwA70PgO29eVmz49EXPJMsbj7l8DfhFvBPOEOfyXgYq-_JDVO52JTFeq1lK7CC53OM76TA2wIkdt5wSDKY5NT3vbiO47Sda0vz86d-LcGCkbUg3_iyPV5NeupFLGKASk1QwfiT9bG4rCHpPJs3a9Pomrn3xw88AtTeXmSLc6dCr2gutP4KP85qM3alPGMzQ","di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","login":true,"uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.219Z	DEBUG	log/log.go:108	Authorization context replaced for client 10.1.2.223:54580, device 6ec91ff4-60e9-47db-6257-3f05c6867e10, user auth0\|6112467f3a83a80070bc699d
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.373Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: 2a9c76708d96ff7f
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Changed
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"expiresin":86375}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.394Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: 013a4230a1a5b18f
  |   | 2021-08-13 02:14:55 | Path: oic/rd
  |   | 2021-08-13 02:14:55 | Code: POST
  |   | 2021-08-13 02:14:55 | Query: [rt=oic.wk.rdpub]
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","links":[{"href":"/oic/p","if":["oic.if.r","oic.if.baseline"],"ins":564140495,"p":{"bm":3},"rel":"hosts","rt":["oic.wk.p"]},{"href":"/oic/d","if":["oic.if.r","oic.if.baseline"],"ins":3110858707,"p":{"bm":3},"rel":"hosts","rt":["oic.d.smartplug","oic.wk.d"]},{"href":"/plug/1","if":["oic.if.a","oic.if.baseline"],"ins":3159645952,"p":{"bm":3},"rel":"hosts","rt":["oic.r.switch.binary"]},{"href":"/switches","if":["oic.if.ll","oic.if.create","oic.if.b","oic.if.baseline"],"ins":2934548123,"p":{"bm":3},"rel":"hosts","rt":["oic.wk.col"]}],"n":"Smart Plug","ttl":0}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.409Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/oic/p" requested
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.414Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: 013a4230a1a5b18f
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Changed
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","links":[{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/oic/p","if":["oic.if.r","oic.if.baseline"],"ins":318534269,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.wk.p"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/oic/d","if":["oic.if.r","oic.if.baseline"],"ins":136750592,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.d.smartplug","oic.wk.d"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/plug/1","if":["oic.if.a","oic.if.baseline"],"ins":2713892058,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.r.switch.binary"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/switches","if":["oic.if.ll","oic.if.create","oic.if.b","oic.if.baseline"],"ins":757679238,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.wk.col"]}],"lt":0,"ttl":0}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.435Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-NOTIFICATION------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: b909b4621000df0b
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Content
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | Observe: 0
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"if":["oic.if.r","oic.if.baseline"],"mnmn":"ocfcloud.com","pi":"abefb29a-886d-42dc-5881-5170c6884204","rt":["oic.wk.p"]}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.436Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/oic/d" requested
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.466Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-NOTIFICATION------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: aa7c23e860eee873
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Content
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | Observe: 0
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","dmn":[{"language":"en","value":"ocfcloud.com"}],"dmv":"ocf.res.1.3.0,ocf.sh.1.3.0","icv":"ocf.2.0.5","if":["oic.if.r","oic.if.baseline"],"n":"Smart Plug","piid":"c9b9e646-3ffd-4da5-73ec-eef78d0e36ca","rt":["oic.d.smartplug","oic.wk.d"]}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.466Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/plug/1" requested
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.497Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-NOTIFICATION------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: 755444bfc16b86e6
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Content
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | Observe: 0
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"if":["oic.if.a","oic.if.baseline"],"n":"Switch","rt":["oic.r.switch.binary"],"value":false}
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.501Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/switches" requested
  |   | 2021-08-13 02:14:55 | 2021-08-12T17:14:55.525Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:55 | -------------------RECEIVED-NOTIFICATION------------------
  |   | 2021-08-13 02:14:55 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:55 | Token: 1d1515190ba7b5ad
  |   | 2021-08-13 02:14:55 | Path:
  |   | 2021-08-13 02:14:55 | Code: Content
  |   | 2021-08-13 02:14:55 | Query: []
  |   | 2021-08-13 02:14:55 | Observe: 0
  |   | 2021-08-13 02:14:55 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:55 | CBOR:
  |   | 2021-08-13 02:14:55 | {"if":["oic.if.ll","oic.if.create","oic.if.b","oic.if.baseline"],"links":[],"rt":["oic.wk.col"],"rts":["oic.r.switch.binary"],"rts-m":["oic.r.switch.binary"],"x.org.openconnectivity.bl":94}
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.200Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:58 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:58 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:58 | Token: 7d709897839984d6
  |   | 2021-08-13 02:14:58 | Path: oic/sec/session
  |   | 2021-08-13 02:14:58 | Code: POST
  |   | 2021-08-13 02:14:58 | Query: []
  |   | 2021-08-13 02:14:58 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:58 | CBOR:
  |   | 2021-08-13 02:14:58 | {"accesstoken":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6ImhPeHdxSzJCM2k5amZjSFo4dVh6QyJ9.eyJodHRwczovL3d3dy5vY2ZjbG91ZC5jb20vdGVuYW50IjoiKiIsImlzcyI6Imh0dHBzOi8vdHJpc2hpYTEyOTguYXV0aDAuY29tLyIsInN1YiI6ImF1dGgwfDYxMTI0NjdmM2E4M2E4MDA3MGJjNjk5ZCIsImF1ZCI6Imh0dHBzOi8vd3d3Lm9jZmNsb3VkLmNvbSIsImlhdCI6MTYyODc4ODQ3MSwiZXhwIjoxNjI4ODc0ODcxLCJhenAiOiJEZ2V6THlXczFoSFJmNjkxNHRreVVkcDRRRG1rVm1ueiIsInNjb3BlIjoicjpkZXZpY2VpbmZvcm1hdGlvbjoqIHI6cmVzb3VyY2VzOiogdzpyZXNvdXJjZXM6KiB3OnN1YnNjcmlwdGlvbnM6KiBvZmZsaW5lX2FjY2VzcyJ9.vYrpnKLzxx0kurIZd3VI8fu93Tb5LyP85H8iDQtYkhuGwVNi-nmFr_h0GQoucssIvFkjEsKi9_HYVtO2duhlLAWpiFHqfV24NqymynpllUCAETicdtg6_OV1j4G02GRefPL-wN1EGHutHnG2h8UMxfphwA70PgO29eVmz49EXPJMsbj7l8DfhFvBPOEOfyXgYq-_JDVO52JTFeq1lK7CC53OM76TA2wIkdt5wSDKY5NT3vbiO47Sda0vz86d-LcGCkbUg3_iyPV5NeupFLGKASk1QwfiT9bG4rCHpPJs3a9Pomrn3xw88AtTeXmSLc6dCr2gutP4KP85qM3alPGMzQ","di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","login":true,"uid":"auth0\|6112467f3a83a80070bc699d"}
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.201Z	DEBUG	log/log.go:108	Authorization context replaced for client 10.1.2.223:54580, device 6ec91ff4-60e9-47db-6257-3f05c6867e10, user auth0\|6112467f3a83a80070bc699d
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.220Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:58 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:58 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:58 | Token: 7d709897839984d6
  |   | 2021-08-13 02:14:58 | Path:
  |   | 2021-08-13 02:14:58 | Code: Changed
  |   | 2021-08-13 02:14:58 | Query: []
  |   | 2021-08-13 02:14:58 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:58 | CBOR:
  |   | 2021-08-13 02:14:58 | {"expiresin":86372}
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.240Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/oic/d" requested
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.241Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/plug/1" requested
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.241Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/switches" requested
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.242Z	DEBUG	log/log.go:108	observation of resource device_id:"6ec91ff4-60e9-47db-6257-3f05c6867e10"  href:"/oic/p" requested
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.244Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:58 | -------------------RECEIVED-COMMAND------------------
  |   | 2021-08-13 02:14:58 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:58 | Token: 1cd105e25508f80e
  |   | 2021-08-13 02:14:58 | Path: oic/rd
  |   | 2021-08-13 02:14:58 | Code: POST
  |   | 2021-08-13 02:14:58 | Query: [rt=oic.wk.rdpub]
  |   | 2021-08-13 02:14:58 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:58 | CBOR:
  |   | 2021-08-13 02:14:58 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","links":[{"href":"/oic/p","if":["oic.if.r","oic.if.baseline"],"ins":318534269,"p":{"bm":3},"rel":"hosts","rt":["oic.wk.p"]},{"href":"/oic/d","if":["oic.if.r","oic.if.baseline"],"ins":136750592,"p":{"bm":3},"rel":"hosts","rt":["oic.d.smartplug","oic.wk.d"]},{"href":"/plug/1","if":["oic.if.a","oic.if.baseline"],"ins":2713892058,"p":{"bm":3},"rel":"hosts","rt":["oic.r.switch.binary"]},{"href":"/switches","if":["oic.if.ll","oic.if.create","oic.if.b","oic.if.baseline"],"ins":757679238,"p":{"bm":3},"rel":"hosts","rt":["oic.wk.col"]}],"n":"Smart Plug","ttl":0}
  |   | 2021-08-13 02:14:58 | 2021-08-12T17:14:58.281Z	DEBUG	log/log.go:83
  |   | 2021-08-13 02:14:58 | -------------------SEND-RESPONSE------------------
  |   | 2021-08-13 02:14:58 | DeviceId: 6ec91ff4-60e9-47db-6257-3f05c6867e10
  |   | 2021-08-13 02:14:58 | Token: 1cd105e25508f80e
  |   | 2021-08-13 02:14:58 | Path:
  |   | 2021-08-13 02:14:58 | Code: Changed
  |   | 2021-08-13 02:14:58 | Query: []
  |   | 2021-08-13 02:14:58 | ContentFormat: application/vnd.ocf+cbor
  |   | 2021-08-13 02:14:58 | CBOR:
  |   | 2021-08-13 02:14:58 | {"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","links":[{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/oic/p","if":["oic.if.r","oic.if.baseline"],"ins":318534269,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.wk.p"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/oic/d","if":["oic.if.r","oic.if.baseline"],"ins":136750592,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.d.smartplug","oic.wk.d"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/plug/1","if":["oic.if.a","oic.if.baseline"],"ins":2713892058,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.r.switch.binary"]},{"di":"6ec91ff4-60e9-47db-6257-3f05c6867e10","href":"/switches","if":["oic.if.ll","oic.if.create","oic.if.b","oic.if.baseline"],"ins":757679238,"p":{"bm":3,"port":0,"sec":false,"x.org.iotivity.tcp":0,"x.org.iotivity.tls":0},"rt":["oic.wk.col"]}],"lt":0,"ttl":0}

Environment

plgd source : commit 0f2c187
IoTivity version: jakarta commit 803f85585ff9f7306f4342009d8db7a8088241bc
Deployment option: build on my own

Enable observability by using OpenTelemetry

Feature Request

What is the purpose of the feature?

As we need to solve availability and performance quickly, the observability framework has to be integrated into plgd cloud. Commands and events tracing, have more insights into delays, blocking chains, bottlenecks is crucial.

What specific challenge does this feature solve?

Understand the behavior of the system better under the load.

What alternatives did I consider?

None

Publish resource links in batches

Feature Request

What is the purpose of the feature?

The device publishes its resource links in one publish resources message. API of the Resource Aggregate is currently designed to support the publishing of only one resource link per command. In case the device has 500 resources, one connected device triggers 500 command executions on the Resource Aggregate. This needs to be optimized.

What specific challenge does this feature solve?

Performance improvement by supporting the publishing of multiple resource links within one command.

What alternatives did I consider?

None

Publish device Registered / Unregistered events to the EventBus

Feature Request

What is the purpose of the feature?

Our application using plgd is event-driven, subscribed directly to NATS events. We're able to find all resource-related events except device REGISTERED and UNREGISTERED events as they are defined here.

What specific challenge does this feature solve?

Building a fully event-driven application on top of plgd. Currently, as 2 events are missing it's not possible.

What alternatives did I consider?

We could pull information from the authorization server, but such a design doesn't meet our requirements.

Optimize PLGD application logs of Kubernetes deployment.

Feature Request

Optimize Logs of Kubernetes deployment.

What is the purpose of the feature?

Optimize logging by making better utilization of resources and address the following issues:

Coap-Gateway and other PLGD app logs in debug mode are being written in stderr stream only.
Coap-Gateway logs has entries (for notifications) that use more than 10 lines.

What specific challenge does this feature solve?

By optimizing the logs to produce a single line per timestamp entry (for example putting it in a json) it allows log aggregator tools like Loki to better provide better log analysis (by reducing the amount of lines being produced in logs). It also helps if not all log entries are being sent to the same std stream. Errors should go to stderr, other log entries should go to stdout.