edgexfoundry / edgex-compose

EdgeX Foundry Docker Compose release compose files and tools for building EdgeX compose files

Edgex Docker Compose

This repository contains the docker compose files for EdgeX releases.

Note: Each release is now on it's own branch named after the release codename. You can view all branches here.

This branch contains the pre-release docker compose files that pull and run the EdgeX images from the Nexus3 docker registry that are tagged with master. These images are built from the Edgex CI Pipeline when PRs are merged into the master branch.

Note: Docker does not re-pull newer instances of these images. You must pull the new image instances. See the make pull command described below that will do this for you.

These pre-release docker compose files are generated from the multiple source compose files located in the compose-builder folder. See README there for details on regenerating these files after making changes to the source files.

Compose Tool

The Makefile in this folder expects the docker compose CLI command. The old stand-alone docker-compose tool is no longer supported. See https://docs.docker.com/compose/install/ for installation details for the latest docker compose CLI command.

Compose Files

This folder contains the following compose files:

Generated Compose files

**NOTES: **

• DO NOT EDIT the files directly for permanent changes. Make all permanent changes to the source compose files in the compose-builder folder and then regenerate these files
• Use make build from Compose Builder to regenerate all the following compose files.
• See each description for the convenience make commands that are provided to work with each of these compose files.

• docker-compose.yml Contains all the services required to run in secure configuration. Includes the Device Virtual & Device REST device services and the UI. Make Commands

  • Use make run <service(s)> and make down to start and stop the services using this compose file.

  • Use make pull <service(s)> to pull all or some images for the services in this compose file.

  • Use make get-token to generate a Kong access token for remote access of the services running from this compose file.

• docker-compose-arm64.yml Contains all the services required to run in secure configuration on ARM64 system. Includes the Device Virtual & Device REST device services and the UI. Make Commands

  • Use make run arm64 and make down to start and stop the services using this compose file.
  • Use make pull arm64 <service(s)> to pull all or some images for the services in this compose file.
  • Use make get-token arm64 to generate a Kong access token for remote access of the services running from this compose file.

• docker-compose-with-app-sample.yml Contains all the services required to run in secure configuration with Sample application service. Includes the Device Virtual, Device REST, UI & App Sample services. Use this version when using the UI to make changes to the configurable pipeline on the Sample application service. Make Commands

  • Use make run app-sample and make down to start and stop the services using this compose file.
  • Use make pull app-sample <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-with-app-sample-arm64.yml Contains all the services required to run in secure configuration with the Sample application service on ARM64 system . Includes the Device Virtual, Device REST, UI & App Sample services. Use this version when using the UI to make changes to the configurable pipeline on the Sample application service.

  Make Commands

  • Use make run no-secty app-sample arm64 and make down to start and stop the services using this compose file.
  • Use make pull no-secty ui app-sample <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-no-secty.yml Contains just the services needed to run in non-secure configuration. Includes the Device Virtual & Device REST device services and the UI. Make Commands

  • Use make run no-secty and make down to start and stop the services using this compose file.
  • Use make pull no-secty <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-no-secty-arm64.yml Contains just the services needed to run in non-secure configuration on ARM64 system. Includes the Device Virtual & Device REST device services and the UI.

  Make Commands

  • Use make run no-secty arm64 and make down to start and stop the services using this compose file.
  • Use make pull no-secty arm64 <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-no-secty-with-app-sample.yml Contains just the services needed to run in non-secure configuration with Sample application service. Includes the Device Virtual, Device REST, UI & App Sample services. Use this version when using the UI to make changes to the configurable pipeline on the Sample application service. Make Commands

  • Use make run no-secty app-sample and make down to start and stop the services using this compose file.
  • Use make pull no-secty app-sample <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-no-secty-with-app-sample-arm64.yml Contains just the services needed to run in non-secure configuration with the Sample application service on ARM64 system . Includes the Device Virtual, Device REST, UI & App Sample services. Use this version when using the UI to make changes to the configurable pipeline on the Sample application service.

  Make Commands

  • Use make run no-secty app-sample arm64 and make down to start and stop the services using this compose file.
  • Use make pull no-secty app-sample <service(s)> to pull all or some images for the services in this compose file.

• docker-compose-openziti.yml Contains the services needed to bring OpenZiti online, configure it, and enable consul to perform underlay-based health checking. Used in conjunction with make run (pull) zero-trust. This compose file should be started before starting the make run zero-trust compose file.

  Make Commands

  • Use make openziti and make openziti-down to start and stop the services using this compose file.
  • Use make openziti-clean to remove all stopped containers, all volumes and all networks used by the EdgeX stack. Use this command when needing to do a fresh restart. Note You must also run the corresponding make down zero-trust command to fully clean up.
  • Use make openziti-logs to follow the logs

TAF Compose files

The compose files under the taf subfolder are used for the automated TAF tests. These compose files are also generated from Compose Builder when the make build command is used.

Additional make commands

• make clean

  Runs down command and removes all stopped containers, all volumes and all networks used by the EdgeX stack. Use this command when needing to do a fresh restart.

• make get-token For secure mode only. Runs commands via docker to generate a new API Gateway token.

• make get-consul-acl-token For secure mode only. Runs commands via docker to retrieve a Consul Access token.

Additional compose files

• docker-compose-portainer.yml Stand-alone compose file for running Portainer which is a Docker container management tool. Visit here https://www.portainer.io/ for more details on Portianer. Use make portainerand make portainer-down to start and stop Portainer.

edgex-compose's Issues

Make Kuiper run as non-root in docker

Kuiper 1.1.2 has been release which supports running as non-root.

Compose files need to be update to use this version which runs Kuiper under the kuiper user.

[Secure Consul Ph. 2] Integrate Consul tokens with device services

Once device services are using the latest go-mod-bootstrap of useSecretProvider, the compose builder need to enhance and integrate device related services to use Consul tokens

Fail to start device-virtual with security enabled

It seems to lost command on taf-compose file.

level=INFO ts=2021-04-16T07:04:40.509233472Z app=edgex-security-bootstrapper source=command.go:119 msg="Security bootstrapper running waitFor"
level=INFO ts=2021-04-16T07:04:40.509291991Z app=edgex-security-bootstrapper source=command.go:144 msg="Waiting for: [tcp://edgex-security-bootstrapper:54329] with timeout: [1m0s]"
level=INFO ts=2021-04-16T07:04:40.51292993Z app=edgex-security-bootstrapper source=command.go:214 msg="Connected to tcp://edgex-security-bootstrapper:54329"
Fri Apr 16 07:04:40 UTC 2021 Starting --registry --confdir=/custom-config ...
/edgex-init/ready_to_run_wait_install.sh: exec: line 41: illegal option --

Needs to modify environment on app-service for the trigger configuration changing on taf compose file

Based on the app-service-configurable PR #203, needs to modify environment variable from MESSAGEBUS_SUBSCRIBEHOST_HOST
to TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST.

Update Proxy Setup env overrides for change to use service keys

Proxy Setup route configuration now uses the services' service keys as the configuration map key.

The env overrides must be updated to use the new names.

Clean up feature flag ENABLE_REGISTRY_ACL in docker-compose

once feature flag in edgex-go got removed, it also needs to be cleaned up from compose file.

Why using environment variable name with hyphen(-) symbole?

I found there are quite many environment vairable such as CLIENTS_EDGEX-CORE-COMMAND_HOST,
CLIENTS_EDGEX-CORE-DATA_HOST, which contain the hypen(-) symbol.

As we know, hyphen(-) symbol is invalid for an environment variable name, especially for bash shell.

Why using such strange naming convention? It would be unfriendly when we are trying the edgex service on native Linux running system.

Update compose files for Kong Admin lockdown

See relegated issue edgexfoundry/edgex-go#2537

Consolidate App Service Export services to just secure or just non-secure for TAF tests

Currently the TAF secure compose file has both the secure and non-secure App Service export services (http & mqtt)

edgex-compose/taf/docker-compose-taf.yml

Lines 174 to 318 in 6f9eebd

	app-service-mqtt-export:
	container_name: app-service-mqtt-export
	depends_on:
	- consul
	- data
	environment:
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	DATABASE_HOST: edgex-redis
	EDGEX_PROFILE: mqtt-export
	EDGEX_SECURITY_SECRET_STORE: "false"
	REGISTRY_HOST: edgex-core-consul
	SERVICE_HOST: app-service-mqtt-export
	SERVICE_PORT: 48103
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	WRITABLE_LOGLEVEL: INFO
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_BROKERADDRESS: tcp://MQTT_BROKER_ADDRESS_PLACE_HOLDER:1883
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_TOPIC: edgex-events
	hostname: app-service-mqtt-export
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48103:48103/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	app-service-rules:
	container_name: edgex-app-service-configurable-rules
	depends_on:
	- consul
	- data
	environment:
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	EDGEX_PROFILE: rules-engine
	EDGEX_SECURITY_SECRET_STORE: "false"
	REGISTRY_HOST: edgex-core-consul
	SERVICE_HOST: edgex-app-service-configurable-rules
	SERVICE_PORT: 48100
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	TRIGGER_PUBLISHTOPIC: events
	hostname: edgex-app-service-configurable-rules
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48100:48100/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	appservice-mqtt-export-secrets:
	command: /app-service-configurable -cp=consul.http://edgex-core-consul:8500 --registry
	--confdir=/res
	container_name: app-service-mqtt-export-secrets
	depends_on:
	- consul
	- data
	- security-bootstrapper
	entrypoint:
	- /edgex-init/ready_to_run_wait_install.sh
	environment:
	API_GATEWAY_HOST: kong
	API_GATEWAY_STATUS_PORT: '8001'
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	DATABASE_HOST: edgex-redis
	EDGEX_PROFILE: mqtt-export
	EDGEX_SECURITY_SECRET_STORE: "true"
	ENABLE_REGISTRY_ACL: "true"
	PROXY_SETUP_HOST: edgex-proxy-setup
	REGISTRY_HOST: edgex-core-consul
	SECRETSTORE_HOST: edgex-vault
	SECRETSTORE_PATH: /v1/secret/edgex/app-service-mqtt-export-secrets/
	SECRETSTORE_PORT: '8200'
	SECRETSTORE_TOKENFILE: /tmp/edgex/secrets/app-service-mqtt-export-secrets/secrets-token.json
	SERVICE_HOST: app-service-mqtt-export-secrets
	SERVICE_PORT: 48104
	STAGEGATE_BOOTSTRAPPER_HOST: edgex-security-bootstrapper
	STAGEGATE_BOOTSTRAPPER_STARTPORT: '54321'
	STAGEGATE_DATABASE_HOST: edgex-redis
	STAGEGATE_DATABASE_PORT: '6379'
	STAGEGATE_DATABASE_READYPORT: '6379'
	STAGEGATE_KONGDB_HOST: kong-db
	STAGEGATE_KONGDB_PORT: '5432'
	STAGEGATE_KONGDB_READYPORT: '54325'
	STAGEGATE_READY_TORUNPORT: '54329'
	STAGEGATE_REGISTRY_HOST: edgex-core-consul
	STAGEGATE_REGISTRY_PORT: '8500'
	STAGEGATE_REGISTRY_READYPORT: '54324'
	STAGEGATE_SECRETSTORESETUP_HOST: edgex-secretstore-setup
	STAGEGATE_SECRETSTORESETUP_TOKENS_READYPORT: '54322'
	STAGEGATE_WAITFOR_TIMEOUT: 60s
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	WRITABLE_INSECURESECRETS_MQTT_SECRETS_PASSWORD: PASSWORD_PLACE_HOLDER
	WRITABLE_INSECURESECRETS_MQTT_SECRETS_USERNAME: USERNAME_PLACEH_OLDER
	WRITABLE_LOGLEVEL: INFO
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_AUTHMODE: usernamepassword
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_BROKERADDRESS: tcp://MQTT_BROKER_ADDRESS_PLACE_HOLDER:1883
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_TOPIC: edgex-events
	hostname: app-service-mqtt-export-secrets
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48104:48104/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	volumes:
	- edgex-init:/edgex-init:ro,z
	- /tmp/edgex/secrets/app-service-mqtt-export-secrets:/tmp/edgex/secrets/app-service-mqtt-export-secrets:ro,z

edgex-compose/taf/docker-compose-taf.yml

Lines 174 to 318 in 6f9eebd

	app-service-mqtt-export:
	container_name: app-service-mqtt-export
	depends_on:
	- consul
	- data
	environment:
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	DATABASE_HOST: edgex-redis
	EDGEX_PROFILE: mqtt-export
	EDGEX_SECURITY_SECRET_STORE: "false"
	REGISTRY_HOST: edgex-core-consul
	SERVICE_HOST: app-service-mqtt-export
	SERVICE_PORT: 48103
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	WRITABLE_LOGLEVEL: INFO
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_BROKERADDRESS: tcp://MQTT_BROKER_ADDRESS_PLACE_HOLDER:1883
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_TOPIC: edgex-events
	hostname: app-service-mqtt-export
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48103:48103/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	app-service-rules:
	container_name: edgex-app-service-configurable-rules
	depends_on:
	- consul
	- data
	environment:
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	EDGEX_PROFILE: rules-engine
	EDGEX_SECURITY_SECRET_STORE: "false"
	REGISTRY_HOST: edgex-core-consul
	SERVICE_HOST: edgex-app-service-configurable-rules
	SERVICE_PORT: 48100
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	TRIGGER_PUBLISHTOPIC: events
	hostname: edgex-app-service-configurable-rules
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48100:48100/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	appservice-mqtt-export-secrets:
	command: /app-service-configurable -cp=consul.http://edgex-core-consul:8500 --registry
	--confdir=/res
	container_name: app-service-mqtt-export-secrets
	depends_on:
	- consul
	- data
	- security-bootstrapper
	entrypoint:
	- /edgex-init/ready_to_run_wait_install.sh
	environment:
	API_GATEWAY_HOST: kong
	API_GATEWAY_STATUS_PORT: '8001'
	CLIENTS_COMMAND_HOST: edgex-core-command
	CLIENTS_COREDATA_HOST: edgex-core-data
	CLIENTS_DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-COMMAND_HOST: edgex-core-command
	CLIENTS_EDGEX-CORE-DATA_HOST: edgex-core-data
	CLIENTS_EDGEX-CORE-METADATA_HOST: edgex-core-metadata
	CLIENTS_EDGEX-SUPPORT-NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_EDGEX-SUPPORT-SCHEDULER_HOST: edgex-support-scheduler
	CLIENTS_METADATA_HOST: edgex-core-metadata
	CLIENTS_NOTIFICATIONS_HOST: edgex-support-notifications
	CLIENTS_SCHEDULER_HOST: edgex-support-scheduler
	DATABASES_PRIMARY_HOST: edgex-redis
	DATABASE_HOST: edgex-redis
	EDGEX_PROFILE: mqtt-export
	EDGEX_SECURITY_SECRET_STORE: "true"
	ENABLE_REGISTRY_ACL: "true"
	PROXY_SETUP_HOST: edgex-proxy-setup
	REGISTRY_HOST: edgex-core-consul
	SECRETSTORE_HOST: edgex-vault
	SECRETSTORE_PATH: /v1/secret/edgex/app-service-mqtt-export-secrets/
	SECRETSTORE_PORT: '8200'
	SECRETSTORE_TOKENFILE: /tmp/edgex/secrets/app-service-mqtt-export-secrets/secrets-token.json
	SERVICE_HOST: app-service-mqtt-export-secrets
	SERVICE_PORT: 48104
	STAGEGATE_BOOTSTRAPPER_HOST: edgex-security-bootstrapper
	STAGEGATE_BOOTSTRAPPER_STARTPORT: '54321'
	STAGEGATE_DATABASE_HOST: edgex-redis
	STAGEGATE_DATABASE_PORT: '6379'
	STAGEGATE_DATABASE_READYPORT: '6379'
	STAGEGATE_KONGDB_HOST: kong-db
	STAGEGATE_KONGDB_PORT: '5432'
	STAGEGATE_KONGDB_READYPORT: '54325'
	STAGEGATE_READY_TORUNPORT: '54329'
	STAGEGATE_REGISTRY_HOST: edgex-core-consul
	STAGEGATE_REGISTRY_PORT: '8500'
	STAGEGATE_REGISTRY_READYPORT: '54324'
	STAGEGATE_SECRETSTORESETUP_HOST: edgex-secretstore-setup
	STAGEGATE_SECRETSTORESETUP_TOKENS_READYPORT: '54322'
	STAGEGATE_WAITFOR_TIMEOUT: 60s
	TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST: edgex-core-data
	WRITABLE_INSECURESECRETS_MQTT_SECRETS_PASSWORD: PASSWORD_PLACE_HOLDER
	WRITABLE_INSECURESECRETS_MQTT_SECRETS_USERNAME: USERNAME_PLACEH_OLDER
	WRITABLE_LOGLEVEL: INFO
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_AUTHMODE: usernamepassword
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_BROKERADDRESS: tcp://MQTT_BROKER_ADDRESS_PLACE_HOLDER:1883
	WRITABLE_PIPELINE_FUNCTIONS_MQTTEXPORT_PARAMETERS_TOPIC: edgex-events
	hostname: app-service-mqtt-export-secrets
	image: nexus3.edgexfoundry.org:10004/docker-app-service-configurable:master
	networks:
	edgex-network: {}
	ports:
	- 127.0.0.1:48104:48104/tcp
	read_only: true
	security_opt:
	- no-new-privileges:true
	user: 2002:2001
	volumes:
	- edgex-init:/edgex-init:ro,z
	- /tmp/edgex/secrets/app-service-mqtt-export-secrets:/tmp/edgex/secrets/app-service-mqtt-export-secrets:ro,z

With the new Secure Consul access tokens the app service have to be run as "Secure" so it doesn't make sense to have both versions. Secure compose should have just one app-service-export-http and just one app-service-export-mqtt

Add read-only flag to device-virtual

device-virtual should be included as a read-only container similar to the containers added from edgexfoundry/edgex-go#1921

Add Builder options for ds-coap, ds-gpio, ds-llrp & app-llrp

Device Coap, Device GPIO, Device LLRP and App LLRP Inventory are all new device and app services that need option to add them to a custom compose file.

Add Hanoi Branch

Add the Hanoi branch and copy over appropriate contents from hanoi branch on developer-scripts repo

Compose: Switch Host overrides for MessageBus Host to edgex-redis

Redis will be the default MessageBus for Ireland.

system(edgex-sys-mgmt-agent) needs to run as root user

sys-mgmt-agent is currently running as non-root user:
https://github.com/edgexfoundry/developer-scripts/blob/381dcb903ca9af6a562466d18c9bb6460235bb6f/releases/nightly-build/compose-files/docker-compose-nexus-no-secty.yml#L363

but some of its APIs require underlying docker call, if running as non-root user there will be error. For example:

$ curl http://localhost:48090/api/v1/metrics/edgex-core-data
[
    {
        "Success": false,
        "errorMessage": "exit status 1",
        "executor": "docker",
        "operation": "metrics",
        "service": "edgex-core-metadata"
    }
]

detailed logs if you try to call operation API:

[
    {
        "Success": false,
        "errorMessage": "Error stopping service: exit status 1 (Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/edgex-core-data/stop: dial unix /var/run/docker.sock: connect: permission denied )",
        "executor": "docker",
        "operation": "stop",
        "service": "edgex-core-data"
    }
]

Do we need to remove the specified user here or there will be API change in the future which requires no root privilege?

thanks

Add common app service env file

The file should contained the follow:

• DATABASE_HOST=edgex-redis
• TRIGGER_EDGEXMESSAGEBUS_SUBSCRIBEHOST_HOST=edgex-core-data

The latter should be removed from other asc env files.

[Secure Consul Ph. 2] Update docker-compose for adding consul token header on kong's route

After changes made from PR edgexfoundry/edgex-go#3391 merged into master, we need to update the docker compose files in order to make adding consul token header to Kong work.

Depends on issue edgexfoundry/edgex-go#3368

DBS failure: 4.1 - Ensure a user for the container has been created

DBS is logging the following findings against the EdgeX stack.

[WARN] 4.1  - Ensure a user for the container has been created
[WARN]      * Running as root: edgex-kuiper
[WARN]      * Running as root: edgex-core-consul
[WARN]      * Running as root: kong-db
[WARN]      * Running as root: edgex-secretstore-setup
[WARN]      * Running as root: edgex-vault
[WARN]      * Running as root: edgex-security-bootstrapper

What is actually happening here is that DBS is running a "docker inspect" on the running containers and looking at the config.user field to see if it is empty. If this field is empty, it means the container starts running as root. Not that this check does not ascertain whether or not any PROCESSES in the container are running as root or not. Just whether nor not THE USER IS EXPLICITY SPECIFIED either via the USER directive in the Dockerfile, or the "user" config in docker-compose, or via docker run command line.

Thus, for all containers that must start their execution as root, we should specify

user: 0:0

in the docker-compose and be explicit about it.

This will remove the above warnings.

Naturally, we want to go the extra mile and minimize the number of root PROCESSES in the container as well, especially eliminating any root processes that are listening on a network socket.

Update compose files for new port assignments

Per TSC 5/19/2021 decision the new ports assignments for Ireland will be:

North side services (app, rules, analytics, etc.): 597xx
Core and supporting services: 598xx (59880 for core data and 59881 for metadata for example)
South side services (device services): 599xx

Port assignments are as follows:

core-data: 59880
core-metadata: 59881
core-command: 59882
support-notifications: 59860
support-scheduler: 59861
sys-mgmt-agent: 58890

app-sample: 59700
app-rules-engine: 59701
app-push-to-core: 59702
app-mqtt-export: 59703
app-http-export: 59704
app-functional-tests: 59705

Kuiper: 59720

device-virtual: 59900
device-modbus: 59901
device-rest: 59986
device-camera: 59985
device-mqtt: 59982
device-random: 59988
device-snmp: 59993

device-grove: 59992
device-bacnet: 59980
device-coap: 59988

Update Setup Proxy Env Overrides for changes from Clients to Routes

`common.env' can be removed from Setup Proxy and the following added:

•   ROUTES_COREDATA_HOST: edgex-core-data
  

•   ROUTES_NOTIFICATIONS_HOST: edgex-support-notifications
  

•   ROUTES_METADATA_HOST: edgex-core-metadata
  

•   ROUTES_SCHEDULER_HOST: edgex-support-scheduler
  

•   ROUTES_RULESENGINE_HOST: edgex-kuiper
  

•   ROUTES_VIRTUALDEVICE_HOST: edgex-device-virtual
  

The following old override can be removed from common.env:

• CLIENTS_RULESENGINE_HOST=edgex-kuiper
• CLIENTS_VIRTUALDEVICE_HOST=edgex-device-virtual

Update docker-compose's common env

Since we modify the config.Clients to use service key, the docker-compose file needs to update the env to meet the changes.
see edgexfoundry/edgex-go#3196

Update TAF script for security modbus error

Connect to device-modbus through Kong got the error no Route matched with those values.

edgex started incorrectly

I just run edgex with docker-compose-no-secty.yml.

edgex-app-service-configurable-rules | level=ERROR ts=2021-05-10T07:49:09.095635923Z app=app-rules-engine source=config.go:172 msg="configuration provider is not available"
edgex-sys-mgmt-agent | level=ERROR ts=2021-05-10T07:49:09.101871994Z app=edgex-sys-mgmt-agent source=config.go:172 msg="configuration provider is not available"
edgex-support-notifications | level=ERROR ts=2021-05-10T07:49:09.980450792Z app=edgex-support-notifications source=config.go:172 msg="configuration provider is not available"
edgex-core-command | level=ERROR ts=2021-05-10T07:49:09.989219863Z app=edgex-core-command source=config.go:172 msg="configuration provider is not available"
edgex-support-scheduler | level=ERROR ts=2021-05-10T07:49:09.989764062Z app=edgex-support-scheduler source=config.go:172 msg="configuration provider is not available"
edgex-core-data | level=ERROR ts=2021-05-10T07:49:09.990050668Z app=edgex-core-data source=config.go:172 msg="configuration provider is not available"
edgex-core-metadata | level=ERROR ts=2021-05-10T07:49:09.992159592Z app=edgex-core-metadata source=config.go:172 msg="configuration provider is not available"
edgex-device-rest | level=ERROR ts=2021-05-10T07:49:10.083562861Z app=edgex-device-rest source=config.go:172 msg="configuration provider is not available"
edgex-device-virtual | level=ERROR ts=2021-05-10T07:49:10.088153312Z app=device-virtual source=config.go:172 msg="configuration provider is not available"

Restructure compose files to use Branches rather than Folders for each release

Currently releases folder contains the compose files for each EdgeX release. With the introduction of the Compose Builderwith source compose files inHanoiareleasebranch is required so user change use the appropriate releaseCompose Builder` to generate custom compose files. This create issue with having the release compose files in the folders and in each branch.

The repos needs to be restructure so that each release's compose files are on that release's branch and no other release's compose files are on that branch. The result will be new branches for all the past release prior to Hanoi, which already has a branch. The master branch will only contain the current work-in-progress files and generated pre-release compose files.

[Secure Consul Ph. 2] Make /tmp/edgex/secrets writable for consul container

Now Consul service container becomes the protagonist to generate the Consul tokens into the secrets folder and thus we need to make that folder to be writable.

make run error

got this error when i make run:

/bin/sh: 1: cannot create ../releases/hanoi/taf/docker-compose-taf.yml: Directory nonexistent

Make down is broken in Compose Builder

docker-compose-ui.yml no longer exists, so command fails.

Change authentication setting for mosquitto v2.0

The test with app-service-mqtt-export fails, because app-service-mqtt-export service can't connect to mqtt-broker .

It seems the default authentication setting of Mosquitto v2.0 different from v1.x. Base on the document https://mosquitto.org/documentation/migrating-to-2-0/.
The issue can be fixed by using mosquitto-no-auth.conf as default configuration.

Needs to add the following line on mqtt-broker service in TAF-related compose file.
command: ["/usr/sbin/mosquitto", "-c", "/mosquitto-no-auth.conf"]

Update Portainer image to portainer-ce

Per Neil Cresswell (CEO of Portainer.io) the Portainer 1.24.x (image Portainer/Portainer) was actually deprecated mid 2020 and has been replaced it with Portainer CE 2.0 (portainer/portainer-ce).

The docker-compose-portainer.yml need to be update to use the newer portainer-ce image

ASC Trigger configuration overrides not complete

Recent update of overrides to account for changes in Trigger configuration are not complete.

add-mqtt-messagebus.yml has a typo and two overrides still need changing.
docker-compose-base.yml has one missed for PublishTopic.

Add option for adding UI when building custom compose file.

Currently users have to run the UI separately with make run ui and stop the UI with make ui-down

To allow the UI to be included in a custom compose file the following changes are needed:

• Add make run-ui to run the UI separately as make run ui does now.
• Change the ui option to add the UI to the generated compose file. Similar to how the most other options like ds-virtual work.
• Building of the committed UI composes files has to be updated to use different option to indicate stand-alone UI compose file.

Enable TAF Smoke Tests

The TAF smoke test need to be enabled for PRs on this repo. Copy Jenkinsfile from developer-scripts

Update compose source for service key name changes for core/support/security services

Service keys core/support/security services no longer have the edgex- prefix

[Secure Consul Ph. 2] Add sys-management and app-service-rule-engine into security yml

We need to have both services system management and app-service-rule-engine use the security-bootstrapper's waiting mechanism so that consul token is ready for them before service is starting up.

Add README and LICENCE file for edgex-compose repo

Repo is missing README and LICENSE files.

Compose: Update ASC env overrides for function config changes

The function configuration for app service configurable has be consolidated. Any env overrides that change function configuration may need to be changed.

Remove Release from compose file names

Now that the releases have been moved to their own branches, there is no longer a need to have the release in the compose file name.

Note that the TAF scripts that pull the compose files will need to be update for the change if file name.

Add previous release compose files to edgex-compose repo

Add the previous release compose files under the releases folder.

Update ASC version when using app-dev option

The dev ASC docker image version has recently changed from "master" to "0.0.0" to be consistent with all other dev images.

The make file needs to be updated to set the ASC image version when the app-dev option is used.

Update docker-compose security for Consul ACL enable

Some environment overrides and volumes need to be changed for setting up Consul's ACL to be working

Update to latest version of 3rd party dependencies

VAULT_VERSION=1.5.3 --> 1.7.1
CONSUL_VERSION=1.9.1 --> 1.9.5
REDIS_VERSION=6.0.9-alpine --> 6.2.3-alpine
KONG_VERSION=2.3-alpine --> 2.4.0-alpine
MOSQUITTO_VERSION=1.6.3 --> 2.0.10

What's the username-password in the database edgex-redis?

I use make run to run the image in the docker-compose.yml, and I don't how to see the data in the edgex-redis.
I have tried the "root:root" to log in but it didn't work.
Here is the message:
root@ubuntu:/edgex-compose# redis-cli
127.0.0.1:6379> auth "root" "root"
(error) WRONGPASS invalid username-password pair or user is disabled.
Or maybe is there any other way to see the data in database?
Thanks in advance.

Populate with compose builder

Move the the compose builder over from developer-scripts and refactor as follows:

• Rename the nexus/nightly-build release to pre-release
• Drop the compose-files sub-folder
• Fix get-token in releases/pre-release/Makefile to call new script.

Update for recent change to Redis as the default MessageBus for all services

The Redis as the default MessageBus the following update and additions are needed.
Add common MessageQue.Host override so Device service also have the override
Update overrides for add-mqtt-messagebus appropriately
Add zmq-bus option, to swith to using ZMQ as the MessageBus

Fix out-of-date proxy-setup's environment overrides

In the service proxy-setup, the environment section, there are two out-of-date envs:

 environment:
      KONGURL_SERVER: kong
      SECRETSERVICE_SERVER: edgex-vault
      SECRETSERVICE_TOKENPATH: /tmp/edgex/secrets/edgex-security-proxy-setup/secrets-token.json

Change SECRETSERVICE_SERVER to SECRETSTORE_HOST and
SECRETSERVICE_TOKENPATH to SECRETSTORE_TOKENPATH

Update docker images names to new names

All the edgex docker image names have been changed to drop the docker- prefix and -go or -c postfix

Duplicate port 59703 in TAF compose files

Services scalability-test-mqtt-export and app-service-mqtt-export both used port 59703 in TAF compose files.

edgex-proxy-setup is not running in secure mode turned on

security service proxy-setup is running with environment variable EDGEX_SECURITY_SECRET_STORE: "false", in which it should be running with "true".

  proxy-setup:
    environment:
  .......
      EDGEX_SECURITY_SECRET_STORE: "false"
      KONGURL_SERVER: kong
      PROXY_SETUP_HOST: edgex-proxy-setup
  ........

Unclear instructions how to run hanoi on Raspberri B (Ubuntu 20.10)

I'm unable to start the services correctly, steps to reproduce:

• Using Raspberry Pi 3 B with Ubuntu 20.10 x64
• Docker version 19.03.13 build 4484c46
• Docker-compose version: 1.25.0
• I cloned this repo
• ran: git checkout -b hanoi
• updated .env file parameters:

    - RELEASE: hanoi
    - RELEASE_FOLDER: ../hanoi

-ran make gen no-secty arm64
-ran make build no-secty arm64

• navigated to $REPO/hanoi
• ran docker-compose -f docker-compose-no-secty-arm64.yaml --verbose up

the tail of the logs look something like:

Attaching to edgex-redis, edgex-core-consul, edgex-support-notifications, edgex-support-scheduler, edgex-core-metadata, edgex-core-command, edgex-core-data, edgex-sys-mgmt-agent, edgex-app-service-configurable-rules, edgex-device-virtual, edgex-device-rest, edgex-kuiper
�[33medgex-core-command   |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[36medgex-app-service-configurable-rules |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[32medgex-core-consul    |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[35medgex-core-data      |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[34medgex-core-metadata  |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[36;1medgex-device-rest    |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[33;1medgex-device-virtual |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[33medgex-core-command exited with code 1
�[0m�[32;1medgex-kuiper         |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[35;1medgex-redis          |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[34;1medgex-support-notifications |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[36medgex-app-service-configurable-rules exited with code 1
�[0m�[36medgex-support-scheduler |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[33medgex-sys-mgmt-agent |�[0m standard_init_linux.go:211: exec user process caused "operation not permitted"
�[32medgex-core-consul exited with code 1
�[0m�[35medgex-core-data exited with code 1
�[0m�[34medgex-core-metadata exited with code 1
�[0m�[36;1medgex-device-rest exited with code 1
�[0m�[36medgex-support-scheduler exited with code 1
�[0m�[35;1medgex-redis exited with code 1
�[0m�[33;1medgex-device-virtual exited with code 1
�[0m�[33medgex-sys-mgmt-agent exited with code 1
�[0m�[34;1medgex-support-notifications exited with code 1
�[0m�[32;1medgex-kuiper exited with code 1
�[0m

Am i doing something wrong?

thanks

Change overrides with hyphens to replace hyphens with underscores

Dependent on this PR merged and changes propagated to all Go services
edgexfoundry/go-mod-bootstrap#216

Incorrect hostname of EDGEX__DEFAULT__SERVICESERVER on rulesengine service

The value of EDGEX__DEFAULT__SERVICESERVER should be http://edgex-core-metadata:48081.

edgex-compose/taf/docker-compose-taf.yml

Line 1037 in 9121568

EDGEX__DEFAULT__SERVICESERVER: http://edgex-metadata-data:48081