advancedtelematic / aktualizr

C++ SOTA Client

aktualizr

C++ implementation of Uptane OTA update client.

The client is intended to be installed on devices that wish to receive OTA updates from an Uptane-compatible OTA server such as HERE OTA Connect. It is most commonly built by using the meta-updater layer in a Yocto environment. You can use aktualizr as a stand-alone system tool or you can integrate libaktualizr into a larger project.

The client is responsible for:

• Communicating with the OTA server

• Authenticating using locally available device and user credentials

• Reporting current software and hardware configuration to the server

• Checking for any available updates for the device

• Downloading any available updates

• Installing the updates on the system, or notifying other services of the availability of the downloaded file

• Receiving or generating installation reports (success or failure) for attempts to install received software

• Submitting installation reports to the server

The client maintains the integrity and confidentiality of the OTA update in transit, communicating with the server over a TLS link. The client can run either as a system service, periodically checking for updates, or can by triggered by other system interactions (for example on user request, or on receipt of a wake-up message from the OTA server).

Table of Contents

• Security
• Installation
  • Dependencies
  • Building
  • Running tests
  • Tags
  • Building with Docker
• Usage
  • Configuration
  • Running a "fake" device
  • Provisioning
• Changelog
• Maintainers
• Contribute
• License

Security

This client is aligned with the Uptane security framework for software updates. Full details and documentation can be found on their site.

Installation

Dependencies

To install the minimal requirements on Debian/Ubuntu, run this:

sudo apt install asn1c build-essential cmake curl libarchive-dev libboost-dev libboost-filesystem-dev libboost-log-dev libboost-program-options-dev libcurl4-openssl-dev libpthread-stubs0-dev libsodium-dev libsqlite3-dev libssl-dev python3

The default versions packaged in recent Debian/Ubuntu releases are generally new enough to be compatible. If you are using older releases or a different variety of Linux, there are a few known minimum versions:

• cmake (>= 3.5)

• curl (>= 7.47)

• openssl (>= 1.0.2)

• libboost-* (>= 1.58.0)

• libcurl4-openssl-dev (>= 7.47)

• libpthread-stubs0-dev (>= 0.3)

Additional packages are used for non-essential components:

• To build the test suite, you will need net-tools python3-dev python3-openssl python3-venv sqlite3 valgrind.

• To run the linting tools, you will need clang clang-format-10 clang-tidy-10.

• To build additional documentation, you will need doxygen graphviz.

• To build with code coverage, you will need lcov.

Some features also require additional packages:

• For OSTree support, you will need libostree-dev (>= 2017.7).

• For PKCS#11 support, you will need libp11-3 libp11-dev.

• For fault injection, you will need fiu-utils libfiu-dev.

Building

This project uses git submodules. To checkout the code:

git clone --recursive https://github.com/advancedtelematic/aktualizr
cd aktualizr

If you had an old checkout, forgot to include --recursive or need to update the submodules, run:

git submodule update --init --recursive

aktualizr is built using CMake. To setup your build directory:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Debug ..

You can then build the project from the build directory using Make:

make

You can also create a debian package:

make package

To use CMake’s Ninja backend, add -G Ninja to the first CMake invocation. It has the advantage of running all targets in parallel by default and is recommended for local development.

Running tests

Before checking in code, it must pass the following tests (along with their corresponding build targets):

• compilation of the main targets and tests without warning: make and make build_tests

• validation against the project’s automatic formatting conventions: make check-format to run the check, make format to apply the transformation automatically

• absence of clang-tidy warning: make clang-tidy

• full test suite run: make check (test build included), make test (only run the tests)

The qa target includes all of these checks, including auto-formatting:

make qa

Note that, by default, the compilation and tests run in sequence and the output of failing tests is suppressed. To run in parallel, for example with eight threads, and print the output of failing tests, run this:

CTEST_OUTPUT_ON_FAILURE=1 CTEST_PARALLEL_LEVEL=8 make -j8 qa

Some tests require additional setups, such as code coverage, HSM emulation or provisioning credentials. The exact reference about these steps is the main test script used for CI. It is parametrized by a list of environment variables and is used by our CI environments. To use it, run it in the project’s root directory:

./scripts/test.sh

Note that it will run CMake itself in a dedicated build directory.

To get a list of the common environment variables and their corresponding system requirements, have a look at the Gitlab CI configuration and the project’s Dockerfiles.

Tags

Generate tags:

make tags

Building with Docker

Several Dockerfiles are provided to support building and testing the application without dependencies on your local environment.

If you have a working docker client and docker server running on your machine, you can build and run a docker image on the default environment with:

./scripts/run_docker_test.sh Dockerfile

It will start a shell session inside the container, running as the same UID/GID as on the host system, with the current directory mounted as a docker volume. Any local code changes are then immediately in effect inside the container and user/group permissions are compatible in the two environments.

Inside the container, the test suite with coverage can be run with:

TEST_WITH_COVERAGE=1 TEST_WITH_P11=1 TEST_WITH_STATICTESTS=1 ./scripts/test.sh

(see the content of ci/gitlab/.gitlab-ci.yml and scripts/test.sh for more testing options)

Alternatively, scripts/run_docker_test.sh can directly run the test script:

./scripts/run_docker_test.sh Dockerfile \
                            -eTEST_WITH_COVERAGE=1 \
                            -eTEST_WITH_P11=1 \
                            -eTEST_WITH_STATICTESTS=1 \
                            -- ./scripts/test.sh

Usage

Configuration

To run the aktualizr client, you will need to provide a toml-formatted configuration file using the command line option -c or --config:

aktualizr -c <path/configfile>

Additional command line options can be found in the code or by running aktualizr --help. More details on configuring aktualizr can be found in docs/ota-client-guide/modules/ROOT/pages/aktualizr-config-options.adoc. If you are using meta-updater, more information about configuring aktualizr in that environment can be found there.

Running a "fake" device

Aktualizr is generally intended to run on embedded devices, but you may find it convenient to run it on your local system for development or testing. To get a binary you can run locally, you can:

• follow the build instructions,

• install Ubuntu packages from the Releases page

Some more detailed instructions on how to configure a fake device can be found on the OTA Connect docs site.

Provisioning

If you intend to use aktualizr to authenticate with a server, you will need some form of provisioning. Aktualizr currently supports provisioning with shared credentials or with device credentials. Device credential provisioning supports using an HSM to store private keys. The differences and details are explained in docs/ota-client-guide/modules/ROOT/pages/client-provisioning-methods.adoc and docs/ota-client-guide/modules/ROOT/pages/enable-device-cred-provisioning.adoc. You can learn more about the credentials files used to support provisioning in docs/ota-client-guide/modules/ROOT/pages/provisioning-methods-and-credentialszip.adoc.

Changelog

The changelog is available in CHANGELOG.md.

Maintainers

This code is maintained by the OTA team at HERE Technologies. If you have questions about the project, please reach us through Github issues for this repository or email us at [email protected].

Contribute

Complete contribution guidelines can be found in CONTRIBUTING.md.

License

This code is licensed under the Mozilla Public License 2.0, a copy of which can be found in this repository. All code is copyright HERE Europe B.V., 2016-2020.

We require that contributors accept the terms of Linux Foundation’s Developer Certificate of Origin. Specific instructions can be found in CONTRIBUTING.md.

aktualizr's Issues

OTAC-FUN-14 Check for updates

Client should check for the availability of new software updates at a configured interval.

This requires (roughly):

• Fetching a OAuth2 token from the authorization endpoint (also known as 'Auth+' in some places in the documentation)
• GET /api/v1/device_updates//queued to poll for pending updates
• GET /api/v1/device_updates///download to fetch the updates
• POST /api/v1/device_updates// (yes, duplicated) to report the outcome of the installation
• When the token appears to be invalid (i.e. the update server starts returning 403 errors), fetch a new one.

The API documentation is here, and @taheris can confirm any questions about the API:

http://advancedtelematic.github.io/rvi_sota_server/dev/api.html

(The client also uses PUT /api/v1/device_updates//system_info and PUT /api/v1/device_updates//installed with list of installed packages, I'll put these in a separate ticket)

For the purposes of this ticket, saving the download to disk in /tmp or simply discarding it is OK.

Acceptance criteria:

• Device appears as 'connected' in ATS Garage UI
• A user uploads a dummy file as a package and queues it for installation on the device
• It gets fetched by sota_client and the UI reports that the installation completed successfully.

Download update file

For initial implementation, client should download the update as soon as it receives data that an update is available.

In the final client, the download will be triggered after interaction with the Software Installer

Doxygen Source Code Documentation

A Doxygen compliant comments are already part of the code we can add a Doxygen configuration and integrate Doxygen in the makefiles / build process.

Register with Coverity Scan

Arthur has registered with Coverity Scan, but the configuration has not been set up yet. We should set this up.

Automatic HTTP redirect using curl

The garage sota server provides update downloads via a HTTP 30X redirect to the amazon cloud. There is an option in curl that tells curl to automatically accept and follow such redirects. If this will be the desired behavior, the corresponding option should be enabled. Up to now the software checks the curl response for redirects and follows the "manually" allowing to detect, analyse and maybe reject redirects.
sota_server::servercon::download_update(void)
https://curl.haxx.se/libcurl/c/CURLOPT_FOLLOWLOCATION.html

JSON Event parsing

For each Event, implement serialization and deserialization to and from JSON.

The parsing format should be: {"variant":{EventName},"fields":[{arg1},{arg2},...,{argN}]}

For example, if the Event was UpdatesReceived the JSON representation would be:

{
  "variant": "UpdatesReceived",
  "fields": [
    [{
      "requestId": "04179cd3-a73b-4b87-82f7-c0c6616d5f0f",
      "status": "Pending",
      "packageId": {
        "name": "libats",
        "version": "1.0.0"
      },
      "installPos": 0,
      "createdAt": "2016-12-19T16:03:09.171Z"
    }]
  ]
}

In this case {arg1} is another list containing each UpdateRequest (which contains a requestId, status, packageId, installPos and createdAt). Whitespace was added in this example for clarity but should be excluded from the actual implementation.

OTAC-SYS-5 System must support multiple log levels

JSON processing for server communication

JSON encoded response data from the server should be encoded with an appropriate JSON parser. This simple C approach seems to be promising: https://github.com/udp/json-parser

JSON Command Parsing

For each Command, implement serialization and deserialization to and from JSON.

The parsing format should be: {"variant":{CommandName},"fields":[{arg1},{arg2},...,{argN}]}

For example, if the Command was SendInstalledPackages the JSON representation would be:

{
  "variant": "SendInstalledPackages",
  "fields": [
    [{
      "name": "libats",
      "version": "1.0.0"
    }]
  ]
}

In this case {arg1} is another list containing each Package (which contains a name and version). Whitespace was added in this example for clarity but should be excluded from the actual implementation.

Note that this does not follow the syntax in the Rust client of {CommandName} {arg1} {arg2} ... {argN}.

Create ATS Garage account for testing

Generate credentials and give to Moritz

Numerous issues with jsoncpp-1.7.5

jsoncpp-1.7.5 is default on Arch Linux. Some issues are due to immaturity of jsoncpp-1.7.5 itself (the fail to include to their config.h), some are because of incompatible type definitions and very strict compilation rules. Maybe worth specifying in the docs that we require jsoncpp-1.7.2 for now, and keep these issues in mind for the future.

Add coding guidelines to repo

Select a coding guideline and add it to the repo

Integrate support for GENIVI DLT as logging framework option

Diagnostic Log and Trace is a logging framework used in GENIVI-compliant systems (and other automotive projects). It would be great to add support for DLT as a logger for better integration with those systems:

https://github.com/GENIVI/dlt-daemon

Error parsing the result of /device_updates/<uuid>/queued

The data::UpdateRequest::fromJson function assumes that the 'status' enum is an integer, but the actual output of the SOTA Server is an enum containing text strings like "Pending".

Here is a failing test case, derived from an actual interaction with the SOTA Server:

BOOST_AUTO_TEST_CASE(UpdatesReceived_parse) {
  std::string json(
      "[ "
      "{\n"
      "   \"createdAt\" : \"2017-01-05T08:43:40.685Z\",\n"
      "		\"installPos\" : 0,\n"
      "		\"packageId\" : \n"
      "		{\n"
      "			\"name\" : \"treehub-ota-raspberrypi3\",\n"
      "			\"version\" : "
      "\"15f13e2582f29d2218f88adada52ff043d6e9596b7cea288321ed91e275a1768\"\n"
      "		},\n"
      "		\"requestId\" : \"06d64e46-cb25-4d76-b62e-4341b6944d07\",\n"
      "		\"status\" : \"Pending\",\n"
      "		\"updatedAt\" : \"2017-01-05T08:43:40.685Z\"\n"
      "	}\n"
      "]");

  data::UpdateRequest request(data::UpdateRequest::fromJson(json));
}

The value of 'status' can be either "Pending", "InFlight", "Canceled", "Failed" or "Finished". The server won't send all of these. I will check with @jerrytrieu which are possible.

The server-side definitions of this are here:

https://github.com/advancedtelematic/rvi_sota_server/blob/master/core/src/main/scala/org/genivi/sota/core/data/client/PendingUpdateRequest.scala

https://github.com/advancedtelematic/rvi_sota_server/blob/master/core/src/main/scala/org/genivi/sota/core/data/UpdateRequest.scala#L66

Improve log wrapper type checking

TODO from https://github.com/advancedtelematic/sota_client_cpp/blob/a85e2a0/src/main.cpp#L57-L59

Use the enumeration from the logger.hpp to enable automatic range checking
This requires to overload operators << and >> as shown in the boost header boost/log/trivial.hpp
Desired result: bpo::value<loggerLevels_t>

Switch configuration file format to toml

At the moment the configuration file format is yaml, but the configuration files generated by ATS Garage are in toml format. We should make sota_client_cpp read these files directly.

As an implementation note, .toml is thing from the rust world, but it is very close to the Windows INI file format, which boost PropertyTree can read, although the " around strings need to be stripped by hand.

Acceptance Criteria:

• The ota-plus-sdk-.toml files generated by ATS Garage can be passed to sota_client --config

Send ATS coding guidelines to Moritz

Improve dockerfile

Update the docker build system to leave the built executables in the local directory

OTAC-SYS-3 Write exceptions to stderr

Make sure that all exceptions / backtraces and failure messages get logged to stderr.

Add socket gateway

A socket gateway should be added that mirrors the existing Rust implementation. Namely:

• publish all events defined under config path network.socket_events_list (e.g. ["DownloadComplete", "DownloadFailed"]) to the socket at network.socket_events_path.
• accept (and interpret) any command sent to the path defined under network.socket_commands_path, waiting for a SHUT_WR signal to terminate writing and sending the resulting event back to the sender on the same socket.

std::cout undefined in main.cc

Error reported by gcc-5.3.0

Abort on startup

With a new release build, the following crash report is returned on startup:

root@b6b8f70db894:~/build/target# ./sota_client --config sota.toml
[2017-Jan-23 10:09:57.140174]: config read from sota.toml :
	Authtentification Server: https://auth-plus.atsgarage.com
	Sota Server: https://sota-core.atsgarage.com
	Pooling enabled : 0
	Pooling interval : 10
	Client: 432aece6-a001-4c9a-9941-ddb971c265ed
	Secret: YvbGH9pMRQ
	Device UUID: ba0dac05-da73-4add-a808-4db18e80a6ba
[2017-Jan-23 10:09:57.141997]: no commandline option 'loglevel' provided
[2017-Jan-23 10:09:57.142332]: http-gateway turned on, use it
process 7893: arguments to dbus_bus_request_name() were incorrect, assertion "connection != NULL" failed in file /source/third_party/dbus-1.9.0/dbus/dbus-bus.c line 1120.
This is normally a bug in some application using the D-Bus library.
  /source/libdbus_build/lib/libdbus-1.so(_dbus_print_backtrace+0x2e) [0x7f895951b45d]
  /source/libdbus_build/lib/libdbus-1.so(_dbus_abort+0xd) [0x7f8959516013]
  /source/libdbus_build/lib/libdbus-1.so(_dbus_warn_check_failed+0x111) [0x7f89595013f3]
  /source/libdbus_build/lib/libdbus-1.so(dbus_bus_request_name+0x8f) [0x7f89594bacad]
  /source/dbusapi_install/lib/libCommonAPI-DBus.so.3(_ZNK9CommonAPI4DBus14DBusConnection26requestServiceNameAndBlockERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xfa) [0x7f895c4534f6]
  /source/dbusapi_install/lib/libCommonAPI-DBus.so.3(_ZN9CommonAPI4DBus7Factory19registerStubAdapterESt10shared_ptrINS0_15DBusStubAdapterEE+0x3c8) [0x7f895c48d0fa]
  /source/dbusapi_install/lib/libCommonAPI-DBus.so.3(_ZN9CommonAPI4DBus7Factory12registerStubERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES9_S9_St10shared_ptrINS_8StubBaseEES9_+0x343) [0x7f895c48c2f3]
  /source/commonapi_install/lib/libCommonAPI.so.3(_ZN9CommonAPI7Runtime18registerStubHelperERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_St10shared_ptrINS_8StubBaseEES8_b+0x23d) [0x7f895c81019d]
  /source/commonapi_install/lib/libCommonAPI.so.3(_ZN9CommonAPI7Runtime12registerStubERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_S8_St10shared_ptrINS_8StubBaseEES8_+0x179) [0x7f895c80ec31]
  ./sota_client(_ZN17EventsInterpreter3runEv+0x37e) [0x4a63fe]
  /usr/lib/x86_64-linux-gnu/libboost_thread.so.1.58.0(+0x115d5) [0x7f895d5025d5]
  /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f895d9226ba]
  /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f895b53b82d]
Aborted

The sota.toml is as follows:

[auth]
server = "https://auth-plus.atsgarage.com"
client_id = [removed]
client_secret = [removed]

[core]
server = "https://sota-core.atsgarage.com"
polling = false
polling_sec = 10

[dbus]
name = "org.genivi.SotaClient"
path = "/org/genivi/SotaClient"
interface = "org.genivi.SotaClient"
software_manager = "org.genivi.SoftwareLoadingManager"
software_manager_path = "/org/genivi/SoftwareLoadingManager"
timeout = 60

[device]
uuid = "ba0dac05-da73-4add-a808-4db18e80a6ba"
packages_dir = "/tmp/"
package_manager = "off"
certificates_path = "/usr/local/etc/sota_certificates"
system_info = "sota_sysinfo.sh"

[gateway]
console = false
dbus = false
http = false
rvi = false
socket = true
websocket = false

[network]
http_server = "127.0.0.1:8888"
rvi_edge_server = "127.0.0.1:9080"
socket_commands_path = "/tmp/sota-commands.socket"
socket_events_path = "/tmp/sota-events.socket"
websocket_server = "127.0.0.1:3012"

[rvi]
client = "http://127.0.0.1:8901"
storage_dir = "/var/sota"
timeout = 20

Demo SWLM update from ATS Garage SOTA Server

This ticket covers the end to end functionality required to trigger a software update in ATS Garage and have it downloaded over HTTPS then handed off to the GENIVI Software Loading Manager (SWLM). SWLM is responsible for installing the update.

Acceptance Criteria:

• Upload one of the rpm_update.upd packages shipped with SWLM (e.g. rpm_update.upd) to ATS Garage
• Create a new device in ATS garage and download the sota.toml file for sota_client_cpp
• In the existing SOTA Server UI select the rpm_update.upd package and hit 'install'
• The package is downloaded over HTTPS and handed off to SWLM
• SWLM produces a bunch of debug information
  -Configuring SWLM to actually install a package is out of scope

There is an example command-line tool that talks the DBus API in the genivi_swm repository sota_client/sota_client.py

OTAC-SYS-6 User data and URLs should only appear at DEBUG level

INFO logging or higher should not reveal any security or privacy sensitive data

Establish HTTPS OAuth communication with server

Connect to configured server and retrieve Access Token

Binary sizes are very large

A recent bloaty build (with DBus) comes out as:

     VM SIZE                         FILE SIZE
 --------------                   --------------
   0.0%       0 .debug_str         2.51Mi  30.5%
   0.0%       0 .debug_info        1.97Mi  23.9%
   0.0%       0 .strtab             933Ki  11.1%
  42.1%   914Ki .dynstr             914Ki  10.8%
  28.8%   625Ki .text               625Ki   7.4%
   0.0%       0 .debug_line         255Ki   3.0%
   0.0%       0 .symtab             220Ki   2.6%
   9.5%   205Ki .dynsym             205Ki   2.4%
   8.5%   185Ki .eh_frame           185Ki   2.2%
   0.0%       0 .debug_ranges       112Ki   1.3%
   0.0%       0 .debug_aranges      105Ki   1.3%
   3.4%  73.1Ki .gnu.hash          73.1Ki   0.9%
   2.8%  60.7Ki .rodata            60.7Ki   0.7%
   0.0%       0 .debug_abbrev      55.3Ki   0.7%
   2.0%  44.2Ki .eh_frame_hdr      44.2Ki   0.5%
   0.9%  20.5Ki .gcc_except_table  20.5Ki   0.2%
   0.8%  17.2Ki .gnu.version       17.2Ki   0.2%
   0.4%  8.51Ki .rela.plt          8.51Ki   0.1%
   0.3%  5.57Ki [Other]            8.48Ki   0.1%
   0.3%  5.69Ki .plt               5.69Ki   0.1%
   0.1%  3.09Ki .bss                    0   0.0%
 100.0%  2.12Mi TOTAL              8.24Mi 100.0%

After stripping the binary:

     VM SIZE                         FILE SIZE
 --------------                   --------------
  42.1%   914Ki .dynstr             914Ki  42.2%
  28.8%   625Ki .text               625Ki  28.8%
   9.5%   205Ki .dynsym             205Ki   9.5%
   8.5%   185Ki .eh_frame           185Ki   8.5%
   3.4%  73.1Ki .gnu.hash          73.1Ki   3.4%
   2.8%  60.7Ki .rodata            60.7Ki   2.8%
   2.0%  44.2Ki .eh_frame_hdr      44.2Ki   2.0%
   0.9%  20.5Ki .gcc_except_table  20.5Ki   0.9%
   0.8%  17.2Ki .gnu.version       17.2Ki   0.8%
   0.4%  8.51Ki .rela.plt          8.51Ki   0.4%
   0.3%  5.69Ki .plt               5.69Ki   0.3%
   0.1%  3.09Ki .bss                    0   0.0%
   0.1%  2.86Ki .got.plt           2.86Ki   0.1%
   0.0%     568 [ELF Headers]      2.43Ki   0.1%
   0.0%     816 .rela.dyn             816   0.0%
   0.0%     672 .dynamic              672   0.0%
   0.0%     384 .gnu.version_r        384   0.0%
   0.0%       0 .shstrtab             270   0.0%
   0.0%     179 [Other]               231   0.0%
   0.0%      38 [Unmapped]            168   0.0%
   0.0%     120 .init_array           120   0.0%
 100.0%  2.12Mi TOTAL              2.12Mi 100.0%

2MB is on par with the current Rust version (which is too large as it stands). 42% of the size comes from .dynstr, which appears mainly to be link symbols from the Boost library.

Communicate with RVI Core

Using librvi, enable the C++ SOTA client to communicate with RVI.

The Rust client currently implements this by creating an Edge server (which is an HTTP server listening at the interface defined at network.rvi_edge_server). The Edge server connects to an RVI Client on startup and registers all the available services (notify, start, chunk, abort, finish and getpackages).

When the RVI client notifies the SOTA client of a new update, this will trigger a download in chunks which are then re-assembled into the final package.

Events are broadcast to the rest of the SOTA system as follows:

• notify -> UpdateAvailable
• finish -> DownloadComplete
• report (getpackages) -> InstalledSoftwareNeeded

Where librvi already implements features that were done manually in the Rust client, the librvi implementation should be used instead.

Documentation outlining the message formats and communication can be found here:

• http://advancedtelematic.github.io/rvi_sota_server/cli/client-implementation.html
• http://advancedtelematic.github.io/rvi_sota_server/cli/client-implementation-example-session.html

For testing, there is documentation on bringing up the rvi_sota_server with an RVI client and server here: http://advancedtelematic.github.io/rvi_sota_server/doc/deployment-with-dockercompose.html

Segfault on startup

Testing out a release build of the latest master segfaults on startup:

# ./sota_client --loglevel 0 --config sota.toml
[2017-Feb-01 16:39:52.639032]: no 'network.socket_events' option have been found in config file: temp, Falling back to default value
[2017-Feb-01 16:39:52.639242]: config read from temp :
	Authtentification Server: https://auth-plus-staging.atsgarage.com
	Sota Server: https://sota-core-staging.atsgarage.com
	Pooling enabled : 1
	Pooling interval : 10
	Client: e3a63586-37fa-4a98-8f80-011306bfffd1
	Secret: I5oNTaAtkI
	Device UUID: e4a36666-a434-4588-adf2-659e428fdcef
[2017-Feb-01 16:39:52.639380]: boost command line option: loglevel detected.
[2017-Feb-01 16:39:52.639451]: loglevel set to trace
[2017-Feb-01 16:39:52.639576]: http-gateway turned on, use it
Segmentation fault

The config is as follows:

[auth]
server = "https://auth-plus-staging.atsgarage.com"
client_id = [removed]
client_secret = [removed]

[core]
server = "https://sota-core-staging.atsgarage.com"
polling = true
polling_sec = 10

[device]
uuid = "e4a36666-a434-4588-adf2-659e428fdcef"
packages_dir = "/tmp/"
package_manager = "off"
certificates_path = "/sota_certificates"

[gateway]
socket = true

[network]
socket_commands_path = "/tmp/sota-commands.socket"
socket_events_path = "/tmp/sota-events.socket"

Add websocket gateway

Add a websocket gateway that supports the same operations as the Rust implementation. Namely:

• publish every event in the system to the each connected client.
• accept (and interpret) commands from any connected client and return the resulting event

The websocketpp library should be used. In addition, conditional compilation should be added such that the websocket gateway may be excluded (including the libraries) at compile time.

Add processing of YAML-format configuration file

Identify/Isolate ARM build issue

There is a problem running the sota_client_cpp in RVI/CommonAPI mode on ARM processors. We believe that this is caused by one of the libraries that (rvi_lib?) uses.

The aim of this ticket is to isolate this bug into a small, reproducible test case.

OTAC-SYS-1 Add usage information when --help is passed

Integrate logging framework

Integrate a logging framework with support for TRACE, DEBUG, INFO, WARN and ERROR logging.

Log level should be configurable by config file and overridable by command-line argument or environment variable

Add CII Badge to project

Add Core Infrastructure Initiative badge to project, and address any low-hanging fruit items.

Add linting tool

Would like to add a linting tool to the project to check for quality issues. @embmk if you have a tool that you normally use, please let me know the details and I'll get it integrated

OTAC-SYS-2 Exit with meaningful error if required configuration is absent

The client should quit with a useful error message if no config file is supplied

Add code coverage report

Would like to include a code coverage report. Am investigating online approaches (I'm obsessed with badges!)

Argument HttpClient::perform shadows a member

Tidy up CMakeLists.txt

The CMakeLists should be improved to:

• use find_package rather than explicitly naming link libraries like libboost_system.so
• Split it into separate top-level and project CMakeLists.txt. The top-level CMakeLists should find the packages, and the 'project' one should contain definitions of the executables.

Please feel free to change the existing directory structure. I would be happy with all the source code in a single directory for now. We can split it out when the project is big enough. Tests should be called oauthtoken_test.cc, and in the same directory as oauthtoken.cc. https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes

I'm no cmake expert, but the example at https://github.com/advancedtelematic/sota-tools/blob/master/CMakeLists.txt might be a good starting point. Feel free to copy/paste from that code.

HttpClient::authenticate leaks the curl_slist headers struct

HttpClient::authenticate leaks the curl_slist headers struct that it allocates:

From httpclient.cc line 77:

  token = json["access_token"].asString();
  token_type = json["token_type"].asString();

  std::string header = "Authorization: Bearer " + token;
  curl_slist* headers = NULL;
  headers = curl_slist_append(headers, header.c_str());
  headers = curl_slist_append(headers, "Accept: */*");
  headers = curl_slist_append(headers, "Content-Type: application/json");
  headers = curl_slist_append(headers, "charsets: utf-8");
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);

The curl_slist *headers struct is not managed by libcurl, and needs to be kept around for the lifetime of the curl handle, and cleaned up in ~HttpClient() See https://curl.haxx.se/libcurl/c/CURLOPT_HTTPHEADER.html for details.

OTAC-SYS-4 Log messages should be written to stdout

testing strategy for class servercon

If the http connection class managing communication to the sota server will be kept, the way of testing its methods has to be discussed. The current implementation shows two different approaches to substitute parts that rely on communication with the server.
For the oauthtoken-class the methods have been reimplemented to get a valid token when testing/calling parts that rely on a valid token.
For download_update an additional method is implemented and embedded in a hash-define that is activate when building the tests in order to test using a valid available-update-information.

Create Gateway abstraction

A Gateway has two purposes:

1. Forward Command messages to the main interpreter and receive the final Event response.
2. Subscribe to different kinds of Event messages (e.g. all DownloadComplete events).

Re-format code to Google Style Guide

In order to provide a solid base going forward, we should reformat and rearrange the codebase to match the Google C++ Style Guide:

https://google.github.io/styleguide/cppguide.html

For example:

• Fix up the names of classes, class members to be correct: class MyClass {int my_member_; }; https://google.github.io/styleguide/cppguide.html#Naming
• Re-order declarations (public members first) https://google.github.io/styleguide/cppguide.html#Declaration_Order
• Move the header files to be next to the implementation (https://google.github.io/styleguide/cppguide.html#Names_and_Order_of_Includes)

Also, please make sure that the resulting code has been run through clang-format to normalise the whitespace.

Socket command parsing behaviour

A couple of examples of unexpected behaviour in current socket parsing:

input: {}
expected: an error without crashing
actual output:

terminate called after throwing an instance of 'std::runtime_error'
  what():  wrong command variant = null
Aborted

input: []
expected: an error without crashing
actual output:

terminate called after throwing an instance of 'std::runtime_error'
  what():  is<object>()
Aborted

input: {"variant": "SendInstalledPackages", "fields": [[{"name": "libats", "version": "1.0.0"}]]}
expected: send a list of the installed packages
actual output: no action

File write callback for curl

Until now a callback that writes server responses into a string is used. When issue #36 was considered, the curl-handle that is used for the actual download should be configured to use a callback that writes directly into a file. The callback should be more or less like the string callback but use a ofstream or something similar.
Note that the curl-operation that actually downloads the file has to be determined as there may be HTTP 30X redirects and this could lead to writing the redirect URL into a file when using the file-write-callback at the wrong place.

Demo SWLM update from RVI SOTA Server

This ticket covers the end to end functionality required to trigger a software update in RVI SOTA Server and have it downloaded over RVI then handed off to the GENIVI Software Loading Manager (SWLM). SWLM is responsible for installing the update.

Acceptance Criteria:

• Upload one of the rpm_update.upd packages shipped with SWLM (e.g. rpm_update.upd) to a local install of RVI SOTA Server
• Connect sota_client_cpp to the SOTA Server using RVI
• In the existing SOTA Server UI select the rpm_update.upd package and hit 'install'
• The package is downloaded over RVI and handed off to SWLM
• SWLM produces a bunch of debug information
• Configuring SWLM to actually install a package is out of scope

systemd .service files

Create and test default systemd service files.

Acceptance Criteria

• Sensible settings for automatic restart behaviour
• The recipes the meta-updater Yocto layer use them
• Basic startup tests in garage-quickstart-rpi

Add D-Bus gateway

This should support the same operations that exist in the Rust implementation. Namely:

• publish the UpdateAvailable, DownloadComplete and InstalledSoftwareNeeded events to the software_manager{,_path} paths defined in the config.
• handle the initiateDownload, abortDownload and updateReport methods

A Franca IDL interface description of the methods can be found here: https://github.com/advancedtelematic/genivi_swm/blob/master/franca_idl/SotaClient.fidl.

The sd-bus library from systemd should be used. In addition, conditional compilation should be added such that the D-Bus gateway may be excluded (including the libraries) at compile time.

Make simple API request using token

Use the previously retrieved (or cached token) to make an API request