telefonicaid / lightweightm2m-iotagent

IoT Agent accepting COAP requests. Designed to be a bridge between the OMA Lightweight M2M protocol and the NGSI interface.

FIWARE IoT Agent for OMA LightWeight Machine2Machine

This Internet of Things Agent is designed to be a bridge between the OMA Lightweight M2M protocol and the NGSI interface of a context broker

It is based on the FIWARE IoT Agent Node.js Library. Further general information about the FIWARE IoT Agents framework, its architecture and the common interaction model can be found in this repository.

This project is part of FIWARE. For more information check the FIWARE Catalogue entry for the IoT Agents.

📚 Documentation	🎓 Academy	quay.io	🐳 Docker Hub	🎯 Roadmap

Contents

• Background
• Install
• Usage
• API
• Contributing
• Testing
• Roadmap
• License

Background

Description

An Internet of Things Agent is a component that lets groups of devices send their data to and be managed from a FIWARE NGSI Context Broker using their own native protocols. This project provides the IoT Agent for the Lightweight M2M protocol, i.e. the bridge between OMA Lightweight M2M enabled devices and an NGSI Context Broker.

For more information on what an IoT Agent is or how it should work, please check the documentation on the IoT Agent Node.js Library.

For more information on OMA Lightweight M2M you can check the Node.js OMA Lightweight M2M library we are using.

If you just want to start using the agent, jump to the Quick Start Guide.

You will find some more general considerations about the LWM2M Mapping we are using in the following subsections.

Type configuration

In order to assign the proper configuration for each type of device, a mechanism to assign types to new arriving devices should be used. This mechanism is based on Prefixes for the registrarion URL. For each type configured in the lwm2m configuration section in the config.js config file, a URL prefix has to be defined. Whenever a registration arrives to a URL with that prefix, the device will be assigned the corresponding type.

Mappings

One of the features to provide through the IoT Agent is the mapping between the protocol specific names of the South Port traffic to the application-specific names in the North Port. In the case of Lightweight M2M, this means to map two things:

• OMA Registry objects and resources from their URIs (e.g.: '') to their common names (e.g.: '').
• Custom device objects to the names defined by the user.

In order to accomplish this task, the agent supports:

• An aditional property lwm2mResourceMapping that lets the user configure names for to each of the particular resources exposed by a device Type
• Two optional configuration file, omaRegistry.json containing the automatic mappings that will be applied in case there are no custom mappings.

Custom mappings defined by the user in the config.js file or by preprovisioning devices take precedence over any other available mapping.

OMA Registry mapping

The IoT Agent provides a mean to map Lightweight M2M objects supported by the client without the need to map them in the type or prevoprovision information. The mapping works as follows: whenever a device registration arrives to the IoT Agent if there is no configured mapping for any of the objects supported by the decive (that should appear in the registration payload), then all the resources supported by the object in the OMA Registry are configured as passive resources offered by the object.

The OMA Registry information is read from two files: omaRegistry.json and omaInverseRegistry.json. This two files can be generated with the last information in the OMA Registry with the command bin/downloadOmaRegistry.js. Notice that the generated files do not have the same name than the original ones (so the result can be double-checked before changing them). In order to use the freshly downloaded ones, just remove the former and rename the latter.

Preprovisioning

For individual provisioning of devices, LWM2M devices can be preprovisioned to the server, sending all the required information to the IoT Agent Provisioning API. Preprovisioned devices should target the standard '/rd' URL instead of a type URL. The preprovision configuration will be identified by the Endpoint name sent by the device.

A note about security

IoT Agent security is still in development, so no Southbound or Northbound security mechanisms are provided. The NGSI Context Broker can be secured with a PEP Proxy anyway, so the IoT Agent should be able to deal with token based security. This mechanism is achieved with the use of Keystone Trust Tokens. For more information on how to use them, please read the Security Section in Node IoT Agent Library.

Install

Information about how to install the OMA Lightweight M2M IoT Agent can be found at the corresponding section of the Installation & Administration Guide.

A Dockerfile is also available for your use - further information can be found here

Usage

Information about how to use the IoT Agent can be found in the User & Programmers Manual.

The following features are listed as deprecated

API

Apiary reference for the Configuration API can be found here. More information about IoT Agents and their APIs can be found in the IoT Agent Library documentation.

The latest OMA Lightweight M2M IoT Agent documentation is also available on ReadtheDocs

Contributing

If you'd like to contribute, start by searching through the issues and pull requests to see whether someone else has raised a similar idea or question.

Before contributing, please check out contribution guidelines

Testing

Mocha Test Runner + Should.js Assertion Library. The test environment is preconfigured to run BDD testing style. Module mocking during testing can be done with proxyquire To run tests, type

npm test

Roadmap

The roadmap of this FIWARE GE is described here

License

The IoT Agent for Lightweight Machine 2 Machine is licensed under Affero General Public License (GPL) version 3.

© 2023 Telefonica Investigación y Desarrollo, S.A.U

Further information on the use of the AGPL open source license

Are there any legal issues with AGPL 3.0? Is it safe for me to use?

There is absolutely no problem in using a product licensed under AGPL 3.0. Issues with GPL (or AGPL) licenses are mostly related with the fact that different people assign different interpretations on the meaning of the term “derivate work” used in these licenses. Due to this, some people believe that there is a risk in just using software under GPL or AGPL licenses (even without modifying it).

For the avoidance of doubt, the owners of this software licensed under an AGPL-3.0 license wish to make a clarifying public statement as follows:

Please note that software derived as a result of modifying the source code of this software in order to fix a bug or incorporate enhancements is considered a derivative work of the product. Software that merely uses or aggregates (i.e. links to) an otherwise unmodified version of existing software is not considered a derivative work, and therefore it does not need to be released as under the same license, or even released as open source.

lightweightm2m-iotagent's Issues

Object structure in the Device Groups and config file type configurations should be homogeneus

There are currently some differences between both objects, mainly concerning the LWM2M Mappings.

Trying to register an unexistent device without a mapped makes the agent crash

When a LWM2M device that has not been previously registered access the IOTAgent using the generic URL (that does not map the device to any type) the IOTAgent crash with an unhandled exception.

Lazy attributes

Dear Friends,

I noticed that lazy attributes are not in Orion Context Broker when query the Entity. I believe this project use old NGSI version then iotagent-ul. How can it be solved?

Thanks,
Caio Thomas

Device provisioning not working.

Whenever I execute this:

#!/bin/bash

(curl localhost:4041/iot/devices -s -S --header 'Content-Type: application/json' \
  --header 'Accept: application/json' --header 'fiware-service: testing' --header 'fiware-servicepath: /Robot' \
  -d @- | python -mjson.tool) <<EOF
{
    "name": "robot1",
    "entity_type": "Robot",
    "attributes": [
      {
        "name": "Battery",
        "type": "number"
      }
    ],
    "lazy": [
      {
        "name": "Message",
        "type": "string"
      }
    ],
    "commands": [
      {
        "name": "Position",
        "type": "location"
      }
    ],
  "internal_attributes": {
    "lwm2mResourceMapping": {
      "Battery" : {
        "objectType": 7392,
        "objectInstance": 0,
        "objectResource": 1
      },
      "Message" : {
        "objectType": 7392,
        "objectInstance": 0,
        "objectResource": 2
      },
      "Position" : {
        "objectType": 7392,
        "objectInstance": 0,
        "objectResource": 3
      }
    }
  }
}
EOF

I obtain this:

{
    "message": "Cannot read property 'map' of undefined",
    "name": "TypeError"
}

With this log:

time=2015-12-01T12:26:33.292Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Request for path [/iot/devices] from [localhost:4041]
time=2015-12-01T12:26:33.292Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "name": "robot1",
    "entity_type": "Robot",
    "attributes": [
        {
            "name": "Battery",
            "type": "number"
        }
    ],
    "lazy": [
        {
            "name": "Message",
            "type": "string"
        }
    ],
    "commands": [
        {
            "name": "Position",
            "type": "location"
        }
    ],
    "internal_attributes": {
        "lwm2mResourceMapping": {
            "Battery": {
                "objectType": 7392,
                "objectInstance": 0,
                "objectResource": 1
            },
            "Message": {
                "objectType": 7392,
                "objectInstance": 0,
                "objectResource": 2
            },
            "Position": {
                "objectType": 7392,
                "objectInstance": 0,
                "objectResource": 3
            }
        }
    }
}


time=2015-12-01T12:26:33.293Z | lvl=DEBUG | corr=n/a | trans=n/a | op=n/a | msg=Handling device provisioning request.
time=2015-12-01T12:26:33.295Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [TypeError] handing request: Cannot read property 'map' of undefined

Add required files for integration with Sonar

A sonar-project.properties file is needed in order to extract metrics for the repository. It should be filled following the example of other of our Node.js projects:

https://github.com/telefonicaid/fiware-pep-steelskin/blob/master/sonar-project.properties

Observations not managed after an empty "Update Registration"

Hello,

We have detected that after send an empty "Update registration" packet to the LwM2M Server, the next observations packets are not processed (received and sent to the Orion Context Broker).

Next image illustrates how the observations are being ignored after the update registration (with empty payload):

This occurs because Lightweight-IOTAgent expects receive always the objects in the "Update registration" packet payload, but this is an optional case, which is usually used when new instances or objects are enabled/disabled/created after the registration process. The most common case is avoid this overload on each "Update registration" packet.

Currently, the behaviour when an "Update registration" packet is received is:

• Subscriptions are removed.
• Update device information.
• Create again the subscriptions (based on the objects which appear in the "Update registration" packet payload).

Can be found in "lib/services/lwm2mHandlers/updateRegistration.js"

async.waterfall([		 
      removePreviousSubscriptions,		
      apply(iotAgentLib.getDevice, device.name),		 
      apply(commonLwm2m.observeActiveAttributes, payload)
], callback);

In order to avoid a process overload on each "Update registration" packet and the current problem, we purpose avoid to remove/create the subscriptions if the payload is empty.

Next image illustrate how the observations are now received after the update registration (with empty payload):

Tests are passed correctly.

Note: As future work, will be ideal check the existence the objects in the payload, because any other information (mode, lifetime, ...) can be also updated to the LwM2M Server through this way.

BR,
David

Devices in a group do not register their active attributes

When a device group is created with active attributes, no observation is created from the IoT Agent, so no information is ever sent from the client.

Add support for generic Resources of the OMA Registry

The IoT Agent currently gives support for OMA Registry object types, and the resources defined specifically for those types in their DDFs. For the third party objects, a list of generic Resources is available that should be supported from the agent.

404 error when register new app

Hello, am using the config-arduino, am gettting a 404 error when trying do a post on /devices. Is it the right way of doing it?

Any help on it?

Regards,

Create an empty entity when registering a Device as a Context Provider

Create an empty entity when registering a Device as a Context Provider to let the Context Broker count the number of Context Entities related to devices.

Create Quick Start Guides for the typical working modes

Create step-by-step guides for the installation, configuration and usage of the IoT Agent in different scenarios.

Update RPM Creation tool

This IOTA's RPM creation script its quite old, and its not in sync with those used in the rest of IoT Agents, missing the possibility of defining the version and release of the RPM in the command line.

OMA Registry Mappingd

Some decisions were taken concerning OMA Registry mapping that may be subjected to further discussion:

• All the attributes are registered as passive.
• The resource name in the DDF is used as the attribute name (but the name should be changed to reflect the instance and possible the object itself).
• No resource is mapped for the third party objects that don't provide DDF.

IoT Agent doesn't check the supported characters in params

Currently, the IoT Agent does not check COAP query parameters have supported characters, and this cause late errors unconnected with the user input. Add an early check in the user input.

Remove <a name> anchors as they are not needed and slightly break readthedocs

Update documentation with the last API changes in the IoT Agent Library

The Device Provisioning API has suffered some modifications in the last versions of the agents, that should be reflected in the Getting Started howtos.

Revise the Command execution system

Current command execution features make the Context Provider calls to command attributes trigger "Execute" calls in the client successfully, but the IoTAgent is not correctly changing the Command status and result (nor it provides mechanisms for the client to do it).

Code debug

Hi,

I checked the last changes (inclusion of last version of iotagent-node-lib) using my custom static mapping configuration already working with previous versions.

When the IOTAgent receives the registration from the device, it raises the following error:

lvl=FATAL | corr=n/a | trans=n/a | op=n/a | msg=An unexpected exception has been raised. Ignoring: TypeError: callback is not a function

I wonder know if there is some way to obtain debug information from errors in a deeper way (stacktrace).

The error appears using both device provisioning or static configuration.

(using device provisioning and mongodb)

time=2017-11-22T08:23:41.581Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPRouter | msg=Handling request with method [POST] on url [/rd?lwm2m=1.0&ep=HOP240ac4045c86&b=U&lt=40] with messageId [49353]
time=2017-11-22T08:23:41.582Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Extracting query parameters from request
time=2017-11-22T08:23:41.582Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Processing query [lwm2m=1.0&ep=HOP240ac4045c86&b=U&lt=40]
time=2017-11-22T08:23:41.582Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Handling registration request
time=2017-11-22T08:23:41.582Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Checking for the existence of the following parameters [["ep"]]
time=2017-11-22T08:23:41.583Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Storing the following device in the db:
{
"name": "HOP240ac4045c86",
"lifetime": "40",
"address": "188.214.241.72",
"port": 62783,
"creationDate": "2017-11-22T08:23:41.583Z"
}
time=2017-11-22T08:23:41.583Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Registered device [HOP240ac4045c86] with type [SmartSpot]
time=2017-11-22T08:23:41.584Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Calling user handler for registration actions for device [HOP240ac4045c86]
time=2017-11-22T08:23:41.585Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.LWM2MHandlers | msg=Handling registration of the device
time=2017-11-22T08:23:41.585Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Looking for entity params ["resource","apikey"]
time=2017-11-22T08:23:41.598Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Device group for fields [["resource","apikey"]]not found.
time=2017-11-22T08:23:41.598Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBDeviceRegister | msg=Looking for entity with id [HOP240ac4045c86].
time=2017-11-22T08:23:41.605Z | lvl=FATAL | corr=n/a | trans=n/a | op=n/a | msg=An unexpected exception has been raised. Ignoring: TypeError: callback is not a function

(using static configuration and mongodb)

time=2017-11-22T08:26:05.265Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPRouter | msg=Handling request with method [POST] on url [/rd?lwm2m=1.0&ep=HOP240ac4045c86&b=U&lt=40] with messageId [49353]
time=2017-11-22T08:26:05.266Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Extracting query parameters from request
time=2017-11-22T08:26:05.266Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Processing query [lwm2m=1.0&ep=HOP240ac4045c86&b=U&lt=40]
time=2017-11-22T08:26:05.266Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Handling registration request
time=2017-11-22T08:26:05.267Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.COAPUtils | msg=Checking for the existence of the following parameters [["ep"]]
time=2017-11-22T08:26:05.268Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Storing the following device in the db:
{
"name": "HOP240ac4045c86",
"lifetime": "40",
"address": "188.214.241.72",
"port": 62917,
"creationDate": "2017-11-22T08:26:05.267Z"
}
time=2017-11-22T08:26:05.268Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Registered device [HOP240ac4045c86] with type [SmartSpot]
time=2017-11-22T08:26:05.268Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Calling user handler for registration actions for device [HOP240ac4045c86]
time=2017-11-22T08:26:05.269Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.LWM2MHandlers | msg=Handling registration of the device
time=2017-11-22T08:26:05.269Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Looking for entity params ["resource","apikey"]
time=2017-11-22T08:26:05.282Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Device group for fields [["resource","apikey"]]not found.
time=2017-11-22T08:26:05.282Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBDeviceRegister | msg=Looking for entity with id [HOP240ac4045c86].
time=2017-11-22T08:26:05.285Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBDeviceRegister | msg=Entity [HOP240ac4045c86] not found.
time=2017-11-22T08:26:05.286Z | lvl=FATAL | corr=n/a | trans=n/a | op=n/a | msg=An unexpected exception has been raised. Ignoring: TypeError: callback is not a function

I didn't test with memory storage.

First observable value is lost.

I have a button active in CoAP, of custom ObjectID. When I press the button, it increments a counter and then sends a Confirmable with value '1'. The agent confirms with ACK, but it never reaches the Context Broker. Its lost somewhere in the Agent.

When I press the button again, after increasing '1' to '2', it sends again a Confirmable. This message is also confirmed with ACK, but this and any packet after this reaches Context Broker.

If I watch Context Broker for changes, I see the value jumping from 0 to 2.

Should use the resource path as the path for device group mapping in Configuration API

Currently, the device group mapping is only performed via static configuration. Should change this to allow configuration via the resource in new Device Groups.

Device configurations are not loaded into the URL mappings while starting the agent

When the MongoDB registry is used, and the agent is restarted, the URL mapping table is not recreated, thus that information gets lost. The agent should load all the configurations and rewrite the table.

Registration error raised if device group is not present

In the cases where a device group is not present for a preprovisioned device, the registration fails as no API Key can be found (this behavior will happen for all '/' acceses). This seems to be a side-effect of the change of signature of some of the functions regarding groups. Tests based on the step-by-step guides should be added to check this end-2-end functionality, in order to avoid this problems.

develop branch should be removed

PRs are done against master
develop is no longer needed.

Integration with standard OMA LWM2M Tools

There are currently some problems integrating the Agent with other LWM2M libraries, due to the requirement of the single port for I/O. This involves some changes in the LWM2M Node library as well as in the COAP Library used (that have been already proposed but need further discussion).

communication between iot-agent and latest version of iot-discovery

can this prototype of the iot-agent communicates without any problem with the latest version of the iot-discovery?
The ContextBroker, in case of correct operation, uses this status statement:
....
"statusCode": {
"code": "200",
"reasonPhrase": "OK"
}
.....

instead the latest version of the iot-discovery response in case of correct operation is:

.....
errorCode: {
code: 200
reasonPhrase: "OK"
details: "Stored"
}
.....

is this difference a problem when the iot-agent communicates with the iot-discovery?

Best regards,
Lorenzo Trisolini

Cant retrieve an Attribute

Hi all

I cant recover an attribute that I include in my device.

First I create a service:

(curl localhost:4041/iot/services -s -S --header 'Content-Type: application/json' \
  --header 'Accept: application/json' --header 'fiware-service: Weather' --header 'fiware-servicepath: /baloons' \
  -d @- | python -mjson.tool) <<EOF
{
  "services": [
    {
      "resource": "/weatherBaloon",
      "apikey": "",
      "type": "WeatherBaloon",
      "commands": [],
      "lazy": [
        {
          "name": "Longitude",
          "type": "double"
        },
        {
          "name": "Latitude",
          "type": "double"
        },
        {
          "name": "Temperature Sensor",
          "type": "degrees"
        }
      ],
      "active": [
        {
          "name": "Power Control",
          "type": "Boolean"
        }
      ]
    }
  ]
}
EOF

The IotAgent response:

time=2017-08-24T09:49:47.295Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.LWM2MHandlers | msg=Device register not found. Creating new device.
time=2017-08-24T09:49:47.296Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.LWM2MHandlers | msg=Mapping device found to NGSI register
time=2017-08-24T09:49:47.299Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.DeviceService | msg=Registering device into NGSI Service:
{
    "id": "weather1",
    "name": "weather1:WeatherBaloon",
    "type": "WeatherBaloon",
    "service": "Weather",
    "subservice": "/baloons",
    "lazy": [
        {
            "type": "double",
            "name": "Longitude"
        },
        {
            "type": "double",
            "name": "Latitude"
        },
        {
            "type": "degrees",
            "name": "Temperature Sensor"
        },
        {
            "name": "Latitude",
            "type": "String",
            "operations": "R"
        },
        {
            "name": "Longitude",
            "type": "String",
            "operations": "R"
        },
        {
            "name": "Altitude",
            "type": "String",
            "operations": "R"
        },
        {
            "name": "Uncertainty",
            "type": "String",
            "operations": "R"
        },
        {
            "name": "Velocity",
            "type": "Opaque",
            "operations": "R"
        },
        {
            "name": "Timestamp",
            "type": "Time",
            "operations": "R"
        },
        {
            "name": "Temperature Sensor#0",
            "type": "string"
        },
        {
            "name": "Power Control#0",
            "type": "string"
        }
    ],
    "active": [
        {
            "type": "Boolean",
            "name": "Power Control"
        }
    ],
    "internalId": 3
}

Before that I created 3 objects like the guide said.

I want to read Latitude attribute so I did a POST request like that:

curl http://localhost:1026/v1/queryContext -s -S --header 'Content-Type: application/json' \
 --header 'Accept: application/json' --header 'fiware-service: Weather' --header 'fiware-servicepath: /baloons' \
 -d '
{
    "entities": [
        {
            "type": "WeatherBaloon",
            "isPattern": "false",
            "id": "weather1:WeatherBaloon"
        }
    ],
    "attributes" : [
        "Latitude"
    ]    
}' 

but the response is :

{
"errorCode" : {
"code" : "404",
"reasonPhrase" : "No context element found"
}
}

Need Help !

Thanks

Docker path

The Docker link is not correct.

Change with: "fiware/lightweightm2m-iotagent"

so the correct command is: "docker run --link orion:orion fiwareiotplatform/lightweightm2m-iotagent"

Should allow to provision LWM2M devices with an explicit internal mapping

The agent currently requires one of two situations to happen:

• Either the device type is configured and has an internal attribute LWM2M mapping.
• o the device has been provisioned and has the needed mapping.

If a device registration arrives which doesn't meet any of this conditions, the agent crashes. The agent should allow a third condition: the device hasn't any kind of mapping, but it can use the OMA Registry mapping. And, in any case, the Agent should return an error instead of crashing.

Deal with slow communication technologies

Hello,

Some of our devices communicates through GSM or BT, both have something in common, they are slow communication technologies.

Some Orion queries such as request for a device information is managed by the Lightweight IOTAgent as several queries to resources. We don't care if we are requesting resource by resource instead of a complete instance, since there are also queries to request for several resources.

To avoid possible bottlenecks and the coap re-send behaviour, we introduced a new way to request for several resources when the ngsi request is processed by the Lightweight IOTAgent. This is request serially for resources, waiting for the response before to send a new request.

Is an optinal feature which can appear or not in the config.js file:

config.lwm2m = {
...
 requestSerially: false,
...
}

If it appears and the value is true, then this new behaviour will be enabled.

BR,
David

Faulty test with last versions of the libraries

A faulty test was disabled in the last version, that should be checked and fixed. The test belongs to the registration-test.js file and has the following description: 'When a preprovisioned device registers to the the IoT Agent with an active attribute without internal mapping, but present in the OMA registry'.

Compose device name when not found using device provisioning

Hi,

I found a strange behaviour after the inclusion of the iotagent-node-lib upgrade.

Using device provisioning based on the example presented here, the following message appears after receive the provisioning:

time=2017-11-22T08:21:16.341Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.DeviceService | msg=Device name not found, falling back to deviceId:type [SmartSpot:HOP240ac4045c86]

• Seems library doesn't compose correctly the name using the deviceId and the device type.
• Provisioning API has been modified?

Execute resource without provide value

Hello,

We didn't find a way to execute a resource without provide an attribute value. Maybe someone knows how to carry out this?

We used a query such as the next one:

(curl http://127.0.0.1:1026/v1/updateContext -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'fiware-service: HOPBeacon' --header 'fiware-servicepath: /hopbeacon' \
 -d @- | python -mjson.tool) <<EOF
{
    "contextElements": [
        {
            "type": "HOPBeacon",
            "isPattern": "false",
            "id": "HOPe0e5cfb1f4df",
            "attributes": [
            {
                "name": "Temperature Reset Min and Max Measured Values",
                "type": "String",
                "value": ""
            }
            ]
        }
    ],
    "updateAction": "UPDATE"
}
EOF

When value field is empty or value field is not present, the next error is raised in the Lightweight-IOTAgent:

time=2016-11-16T17:53:00.330Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.DeviceManagement | msg=Writting a new value [function (err, v) {
                results[index] = v;
                callback(err);
            }] on resource /3303/0/5605 in device [1]
time=2016-11-16T17:55:25.859Z | lvl=FATAL | corr=n/a | trans=n/a | op=n/a | msg=An unexpected exception has been raised. Ignoring: TypeError: Invalid non-string/buffer chunk

And the query receives the next response:

{
    "errorCode": {
        "code": "404", 
        "details": "error forwarding update", 
        "reasonPhrase": "No context element found"
    }
}

This is what happen in our devices in case of specify a dummy value:

And the query receives the next response:

{
    "errorCode": {
        "code": "404", 
        "reasonPhrase": "No context element found"
    }
}

Not all the available LwM2M Clients are able to manage this extra value.

If there is no way to send an execute command without value, send a dummy value isn't a correct behaviour to execute some resources in LwM2M, most of the executable resources does not receive value, some example of executable resources without value are:

• Reset device (3/0/4)
• Factory reset (3/0/5)
• Registration Update Trigger (1/0/8)
• Reset Min/Max values for some sensor (some IPSO objects)
• ...

We expect to hear about other way to carry out this, but in the meantime we introduced a condition to allow execute commands without specify a value.

and the query result is ok:

{
    "contextResponses": [
        {
            "contextElement": {
                "attributes": [
                    {
                        "name": "Temperature Reset Min and Max Measured Values", 
                        "type": "String", 
                        "value": ""
                    }
                ], 
                "id": "HOPe0e5cfb1f4df", 
                "isPattern": "false", 
                "type": "HOPBeacon"
            }, 
            "statusCode": {
                "code": "200", 
                "reasonPhrase": "OK"
            }
        }
    ]
}

Tests are passed correctly.

BR,
David

Update iotagent-node-lib version

In package.json, the version defined currently is 0.10, which was released on October 6th,2016. It should be updated to current version available in https://github.com/Fiware/iot.IoTagent-node-lib as has happened with IoT Agent for the Ultralight 2.0 protocol

Add contibution guidelines to the repository

There are currently no guidelines to contribute to the repository, making it difficult for people to understand and follow the project lifecycle.

Custom configuration for OMA mapping

First of all, I'd like to point that this issue is directly connected with issue #14, although my idea is to focus on a few specific aspects.

Let's say you have a device which provides several instances of multiple types. Something like this (2x3-PhasePM):
</10242/0>,</10242/1>

In case you wanted to set all the attributes as active for all the instances, you should have to define multiple types (something like 3-PhasePM-0 for instance 0 and 3-PhasePM-1 for instance 1) in the config file, because each attribute in the "lwm2mResourceMapping" section is directly linked to a specific instance (that is, you have to set the "objectInstance" field), otherwise it won't send the observe requests. A way to solve this could be to establish the instance number dynamically, depending on the instances of this type that have been registered, without being forced to set it in the mapping.

IoT Agent LWM2M Memory exhaust and production environment

I'm working on a project which aims to connect +1k devices to a machine, which may probably autoscale. The problem is that with only 2 connected devices, the IDAS IoTAgent ends in a couple of days with a "Memory exhaust" message. It is running in a docker instance in a machine in AWS m3.medium with memory swap.

Is this a recomendable Generic Enabler for production? Has it been tested for a big amount of connected devices? Is this actually out of maintenance?

Thanks in advance

RPM And Services

Create the service files and RPM creation script for the Agent.

Faulty test - Registration-test.js

There is a little bug in the registration-test.js.

" Linting test/unit/registration-test.js ...ERROR [L401:C9] W117: 'xit' is not defined. xit('should return the registration information', function(done)... "

Easy solution: change 'xit' with 'it'.

OSX 10.11

First observe doesn't register the value.

I have an observation over a button, which first triggers:

time=2016-01-19T12:29:51.783Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.InformationReporting | msg=Got first piece of data on resource /60000/0/0 in device [2]
time=2016-01-19T12:29:51.784Z | lvl=DEBUG | corr=n/a | trans=n/a | op=n/a | msg=Observers created successfully.

And the second time (and further times) does:

time=2016-01-19T12:29:59.734Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.InformationReporting | msg=New data on resource /60000/0/0 in device [2]
time=2016-01-19T12:29:59.734Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.LWM2MHandlers | msg=Handling data from device [...]
time=2016-01-19T12:29:59.735Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Looking for entity params ["resource","apikey"]
time=2016-01-19T12:29:59.737Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBGroupRegister | msg=Device group for fields [["resource","apikey"]]not found.
time=2016-01-19T12:29:59.738Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NGSIService | msg=Updating device value in the Context Broker at [http://...:1026/v1/updateContext]
time=2016-01-19T12:29:59.738Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NGSIService | msg=Using the following request:
...

It should register the first value too into context broker.

Queries from Orion to IOTA: Error forwarding query

Hi,

I have been using the LWM2M with success with one physical device. In order to implement a more complex device, we have created a Leshan client (https://github.com/eclipse/leshan), with several resources to observe.

With the first device, the lazy attributes were observable doing the proper requests in Orion. With the device disconnected we can see that the query is forwarded from Orion to the IOTA:

(curl localhost:1026/v1/queryContext -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Fiware-service: smartGondor' --header 'Fiware-servicepath: /gardens' -d @- | python -mjson.tool) <<EOF
{
    "entities": [
        {
            "type": "Device",
            "isPattern": "false",
            "id": "00124b0004117a7f:Device"
        }
    ],
"attributes": [
        "temperature"
    ]

} 
EOF

Resultado en Orion & IOTA:

{
    "errorCode": {
        "code": "404",
        "reasonPhrase": "No context element found"
    }
}

time=2016-05-17T10:10:29.395Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Request for path [//queryContext] from [130.206.125.211:4042]
time=2016-05-17T10:10:29.396Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "entities": [
        {
            "type": "Device",
            "isPattern": "false",
            "id": "00124b0004117a7f:Device"
        }
    ],
    "attributes": [
        "temperature"
    ]
}

time=2016-05-17T10:10:29.396Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Handling query from [130.206.125.211:4042]
time=2016-05-17T10:10:29.397Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Handling received set of attributes: ["temperature"]
time=2016-05-17T10:10:29.397Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.NGSIHandlers | msg=Handling device data query from the northbound for device [00124b0004117a7f:Device] of type [Device]
time=2016-05-17T10:10:29.397Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IOTAgent.NGSIHandlers | msg=New attributes;
temperature
time=2016-05-17T10:10:29.398Z | lvl=DEBUG | corr=n/a | trans=n/a | op=n/a | msg=Selecting ID from the stored device data
time=2016-05-17T10:10:29.398Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=There was an error handling the query: [object Object].
time=2016-05-17T10:10:29.398Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [DEVICE_NOT_FOUND] handing request: The device with id: 00124b0004117a7f was not found
`

But when trying to do the same with the leshan client, the query is not forwarded to the IOTA:

`#!/bin/bash
(curl localhost:1026/v1/queryContext -s -S --header 'Content-Type: application/json' --header 'Accept: application/json' --header 'Fiware-service: smartGondor' --header 'Fiware-servicepath: /gardens' -d @- | python -mjson.tool) <<EOF
{
    "entities": [
        {
            "type": "Device",
            "isPattern": "false",
            "id": "orioncentos.novalocal:Device"
        }
    ],
"attributes": [
        "TFlowCH2"
    ]

} 
EOF

{
    "errorCode": {
        "code": "404",
        "details": "error forwarding query",
        "reasonPhrase": "No context element found"
    }
}
`

Finally, the preprovision of the devices are:

-Device OK

`{
    "devices":[
        {
            "device_id": "00124b0004117a7f",
            "entity_name": "00124b0004117a7f:Device",
            "entity_type": "Device",
            "attributes": [],
            "lazy": [
                {
                "name": "serial",
                "type": "integer"
                },
                {
                "name": "temperature",
                "type": "integer"
                }

            ],
            "commands": [],
            "internal_attributes": {
                "lwm2mResourceMapping": {
                    "serial" : {
                        "objectType": 7111,
                        "objectInstance": 0,
                        "objectResource": 0
                    },
                    "reboot" : {
                        "objectType": 7111,
                        "objectInstance": 0,
                        "objectResource": 1
                    },
                    "temperature" : {
                        "objectType": 7111,
                        "objectInstance": 0,
                        "objectResource": 2
                    }
                }
            }
        }
  ]

}`

-Device NOK

`{
    "devices":[
        {
            "device_id": "orioncentos.novalocal",
            "entity_name": "orioncentos.novalocal:Device",
            "entity_type": "Device",
            "attributes": [],
            "lazy": [

                {
                "name": "DHW Setpoint",
                "type": "integer"
                },{
                    "name":"TFlowCH2",
                    "type":"integer"
                }

            ],
            "commands": [],
            "internal_attributes": {
                "lwm2mResourceMapping": {
                    "TFlowCH2" : {
                        "objectType": 6000,
                        "objectInstance": 0,
                        "objectResource": 31
                    },
                    "DHW Setpoint":{
                        "objectType": 6000,
                        "objectInstance": 0,
                        "objectResource": 56
                    }
                }
            }
        }
  ]

}`

The only difference seems to be the ID of the device, and in the logs of the IOTA, it can be seen that the device ID of the leshan client is the one that I am using as device_id:

`time=2016-05-17T09:29:09.337Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Storing the following device in the db:
{
    "name": "orioncentos.novalocal",
    "lifetime": "30",
    "address": "130.206.125.211",
    "port": 45935,
    "creationDate": "2016-05-17T09:29:09.337Z"
}
time=2016-05-17T09:29:09.337Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.Registration | msg=Registered device [orioncentos.novalocal] with type [Device]`

Thanks in advance

Attribute name collision

In order to support some devices I need, I've registered a new ObjectID in LWM2M, which is available here:
http://technical.openmobilealliance.org/Technical/technical-information/omna/lightweight-m2m-lwm2m-object-registry (urn:oma:lwm2m:x:10242 3-PhasePM)

First of all, if I run the downloadOmaRegistry.js process I can get this ObjectID in the omaRegistry.json file. However, there are no attributes, even though the DDF file is available in this website. Apparently it has to do with the parsing method used to download the .xml files, because for X-ObjectID (Objects defined by vendors or individuals) there is an extra column named "Company", which makes the script think that the .xml file is not available, while in this case it is.

Second, there is a problem with omaInverseRegistry.json. Each element of this file is related to the name of an attribute, and in case two DDF files have an attribute in common one of them disappears. In this case /10242/x/0 is named "Manufacturer", exactly as /3/0/0. So, take a look at this output:

time=2016-03-23T11:39:55.180Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Handling received set of attributes: ["Manufacturer","Model Number","Serial Number","Description","Tension R","Current R","Active Power R","Reactive Power R","Inductive Reactive Power R","Capacitive Reactive Power R","Apparent Power R","Power Factor R","THD-V R","THD-A R","Tension S","Current S","Active Power S","Reactive Power S","Inductive Reactive Power S","Capacitive Reactive Power S","Apparent Power S","Power Factor S","THD-V S","THD-A S","Tension T","Current T","Active Power T","Reactive Power T","Inductive Reactive Power T","Capacitive Reactive Power T","Apparent Power T","Power Factor T","THD-V T","THD-A T","3-Phase Active Power","3-Phase Reactive Power","3-Phase Inductive Reactive Power","3-Phase Capacitive Reactive Power","3-Phase Apparent Power","3-Phase Power Factor","3-Phase phi cosine","Active Energy","Reactive Energy","Inductive Reactive Energy","Capacitive Reactive Energy","Apparent Energy","Tension R-S","Tension S-T","Tension T-R","Frequency"] {"op":"IOTAgent.NGSIHandlers","time":"2016-03-23T11:39:55.180Z","lvl":"DEBUG","msg":"Handling device data query from the northbound for device [1234:3-PhasePM] of type [3-PhasePM]"} {"op":"IOTAgent.NGSIHandlers","time":"2016-03-23T11:39:55.180Z","lvl":"DEBUG","msg":"New attributes;\nManufacturer,Model Number,Serial Number,Description,Tension R,Current R,Active Power R,Reactive Power R,Inductive Reactive Power R,Capacitive Reactive Power R,Apparent Power R,Power Factor R,THD-V R,THD-A R,Tension S,Current S,Active Power S,Reactive Power S,Inductive Reactive Power S,Capacitive Reactive Power S,Apparent Power S,Power Factor S,THD-V S,THD-A S,Tension T,Current T,Active Power T,Reactive Power T,Inductive Reactive Power T,Capacitive Reactive Power T,Apparent Power T,Power Factor T,THD-V T,THD-A T,3-Phase Active Power,3-Phase Reactive Power,3-Phase Inductive Reactive Power,3-Phase Capacitive Reactive Power,3-Phase Apparent Power,3-Phase Power Factor,3-Phase phi cosine,Active Energy,Reactive Energy,Inductive Reactive Energy,Capacitive Reactive Energy,Apparent Energy,Tension R-S,Tension S-T,Tension T-R,Frequency"} time=2016-03-23T11:39:55.180Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.MongoDBDeviceRegister | msg=Looking for entity with name [1234:3-PhasePM]. {"time":"2016-03-23T11:39:55.183Z","lvl":"DEBUG","msg":"Selecting ID from the stored device data"} time=2016-03-23T11:39:55.184Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.DeviceManagement | msg=Reading value from resource /3/0/0 in device [1] time=2016-03-23T11:39:55.184Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.DeviceManagement | msg=Reading value from resource /3/0/1 in device [1] time=2016-03-23T11:39:55.184Z | lvl=DEBUG | corr=n/a | trans=n/a | op=LWM2MLib.DeviceManagement | msg=Reading value from resource /3/0/2 in device [1] time=2016-03-23T11:39:55.184Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=There was an error handling the query: [object Object]. time=2016-03-23T11:39:55.185Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [OMA_MAPPING_NOT_FOUND] handing request: No OMA to NGSI Mapping found for this device or type

To summarize, I'm trying to register a 3-PhasePM (ObjectID 10242) with endPoint = 1234, and it receives a request for /3/0/0 (Device:Manufacturer), /3/0/1 (Device:Model Number) and /3/0/2 (Serial Number), which don't exist, as expected.

In case you intend to guarantee each attribute to be unique, you'll probably have to use a pair "Type:Attribute" in the file omaInverRegistry.json, because otherwise sooner or later you're bound to find the same attribute name in different types (or any similar syntax, as long as it guarantees the uniqueness). For this example you should have "Device:Manufacturer" and "3-PhasePM:Manufacturer" as different names.

Error in config.js

I've installed the LWM2M IoT Agent in the CentOS VM provided for the Context Broker. I obtain the following error after executing "bin/lwm2mAgent.js":

msg=No host found for MongoDB driver.

I checked the configuration in config.js file and I removed these lines

deviceRegistry: {
type: 'mongodb',
host: 'localhost'
},

and I replaced them with the following:

deviceRegistry: {
type: 'mongodb',
},
mongodb: {
host: 'localhost',
port: '27017',
db: 'iotlwm2m'
},

With these changes I could run the instance of the Agent. The problem is that the configuration in this file wasn't updated. I got these lines from the file config.js of the UL2.0 Agent.

This file: lwm2miotagent.iml probably should have been ignored

I'm not sure why this file ended up in the folder root, but it seems to be part of an IDE files or something similar. It should be removed and, if its commonly present in multiple deployments, added to the .gitignore file to avoid future adds.

Observe multiple active attributes

When an instance with multiple attributes is registered the agent is expected to observe all the active ones.

However, I've seen a 30-second delay between each observe request, which would be too much in case the device had loads of attributes. In my case there are 50 of them, which would take like 50 * 30 = 1500 seconds (25 minutes) to complete the observe registrations. Is it normal? Where can I change it?

In case the agent doesn't want to flood the device with too many observe requests, it might be interesting to have some delay, but 30 seconds is way too much. I good solution for me would be making this configurable, so that depending on the device type you could have a different delay. In the end it has to do with how much constrained the "thing" is, and whether it can handle several requests one right after the next.

Strange LwM2M registration response.

Hi all,

Currently the registration packet response is an ACK with code "Empty message" and don't contains the Location-path.

After receive this packet, the "correct" packet response is received (code 2.01 Created and Location-path) but the packet have a different MID, so can't be managed as response for the petition.

Information related with the registration process can be found in 8.2.3 section of the specification (specification link)

I'm trying to fix this problem and contribute, but I'm having some difficulties to find where this extra intermediate packet is being sent. (Note I don't have expertise in javascript programming/node-coap so maybe I'm missing something).

As far as I have come to understand, "lightweightm2m-iotagent" project is based on "lwm2m-node-lib" project, inside of "lwm2m-node-lib" project it's possible find the registration handler at the registration.js file.

The handler calls for 3 tasks and after that, calls the "endRegistration" function to send the registration response packet (the packet with 'Created' code and location-path but with different MID).

async.series([
async.apply(coapUtils.checkMandatoryQueryParams, ['ep'], queryParams),
async.apply(storeDevice, queryParams, req),
async.apply(applyHandler, queryParams, req.payload.toString(), handler)
], endRegistration(req, res));

At this point, I'm not sure if the extra-packet (empty message) is being sent by the user callback (applyHandler which calls to the registrationHandler function in the "lightweightm2m-iotagent" project) or internally by node-coap.

The MID problem can be fixed by introduce:

res.messageId = req.messageId;

at the 'endRegistration' function. But this will not solve the problem since the extra message is sent before the registration packet response (clients will discard the correct message as duplicated and will process the incorrect packet).

Seems no extra packet is being sent by the registrationHandler of "lightweight m2m-iot agent" project so I have my suspicions about the node-coap library.

Someone have idea about from where this extra message comes?

Best regards and sorry for the inconvenience

Any error in the server returns a 2.05 error code to the client

A different error code for each server error would be nice (or at least une unique global error code, but not a 2.xx but a 5.xx)

npm install fails

Just after having cloned the repository, if you execute npm install, the result is the following:

npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but npm-shrinkwrap.json was generated for lockfileVersion@0. I'll try to do my best with it!
npm WARN deprecated [email protected]: ReDoS vulnerability parsing Set-Cookie https://nodesecurity.io/advisories/130
npm WARN deprecated [email protected]: Update to 1.0.8 in node<6 or 2.0.0 for node>6. https://github.com/telefonicaid/logops/issues/36
npm ERR! code ETARGET
npm ERR! notarget No matching version found for [email protected]
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

I have tried this with two different versions of Node:

• The one defined in the package.json: v0.10.8
• v8.7.0

The most simple solution would be to update the Node engine to v4.8.4, as with IoT Agent for the Ultralight 2.0 protocol

Add RPM and linux service for RedHat

Create the RPM generation script and add the needed services and configuration files for the IOTA to work as a standard Linux service.

Integrate commands in the IoT Agent

Integrate the commands functionality from IoTAgent-nodelib into the LWM2M IoT Agent.

Release tags missing on official Fiware DockerHub

Looking at here: https://hub.docker.com/r/fiware/lightweightm2m-iotagent/tags/, it looks like release tags are missing from official Fiware DockerHub.

This prevents users to pull the specific (tested) release of the agent.