The digitaltwin from industryfusion

The following interfaces should be integrated with a git repo for the purposes of distribution of content and semantic models:

IF1 EcoMan ATT Export - Format: Content ---- export to git
IF2 EcoMan Model Export - Format: Semantic ---- export to git
IF3 EcoMan ATT Import - Format: Content ---- import from git
IF4 FleetMan AS Export - Format: Content ---- export to git
IF5 FleetMan AS Import - Format: Content ---- import from git
IF7 FleetMan Model Export - Format: Semantic ---- export to git

See #42 for the original list and descriptions.

The git repo should be configurable and the authentication should be possible with ssh key (SSH) or basic auth (HTTPS).
Ideally a 3rd party library will be used for the git integration.

Git Authentication Credentials will be stored as application configuration (File system files or env vars)

Create dockerhub repo for DigitalTwin

Private Repo for Development
Enable Vulnerability scan

Adaptation of the fusion application to receive alerts from alerta

The fusion application to receive alerts from alerta and not OISP as up til now. The management of rules is out of scope:

Notifications tab on asset detail page
Other alerts (alerts on dashboards, global alerts page)

Containerize the Context Broker

Build docker images for the Context Broker components and push to docker registry (private repo to be provided)
Define helm charts/manifest files for the Context Broker
Deploy the Context Broker components on existing Kubernetes setup
#54

Adaptation of the fusion application EM module to allow peer relationships

The Ecosystem Manager Module of the fusion application will be adapted to envisage the management of inter asset relationships

Define subsystems also in EM
Allow for the management of peer relationships
- A peer relationship is created in the context of an ATT referencing another ATT (or theoretically itself) and
- A allows for the naming of the relationship which will be prefilled with the name of the referenced ATT and
- a cardinality for the relationship (..1 or ..N)
Import / Export of peer relationships in the #42 Interfaces (Content und Model)

Describe Multi-Tenancy concept and analyze state of Sorpio

Validate & test RBAC functionality in Alerta. Extend/test its integration with Keycloak.

Fix these Scorpio services to run with non-root user - AtContext Server, Storage Manager & Subscription Manager

Semantic Aggregation of objects relevant for use-cases

Fully containerized Platform services on Kubernetes

Install Strimzi operator on K8s cluster.

Install kafka connect for PostgreSQL using Strimzi operator to integrate Kafka and PostgreSQL

Implementation of RBAC functionality in Context Broker.

Document Aggregation concept and test with Zeppelin

Define Semantic serialization format for IFF Ecosystem-Manager

Define and document input and output formats (in the semantic web context) of

Ecosystem Manager
Fleet Manager
Factory Manager

Formats for 3 specific use-cases are required:

For the serialization (import and export) of content data in the fusion application
For the export of semantic models out of the fusion application towards the digital twin side of the ecosystem
For the definition of the actual asset in the digital twin itself (NGSI-LD)

It should also be decided and documented whether 1 and 2 above complement each other (i.e. 80% + 20%) or overlap (i.e. 100% + 80%). We could use 2 for the validation of the NGSI-LD of the asset.

Adaptation of the fusion application to receive live sensor data from the NGSI-LD broker

The fusion application will be adapted to receive live sensor data from the NGSI-LD broker and not OISP as up to this point. The following 2 screens will be adapted:

Performance tab live object view on asset detail page
Dashboard: Equipment Efficiency

Perform end to end integration tests with the platform services on Kubernetes

Use helmfile for managing dependencies between charts

Add Strimzi operator and integrate with Scorpio

#16
Analyze the existing Scorpio code base (configuration files, topic creation, etc) to integrate with Strimzi operator.

Integrate Keycloak with Context Broker APIs & Extend Context Broker APIs

Helm chart/Operator design improvement

The current Helm chart design with the two step installation is a problematic design. It does not use the original operators, rather it picks a subset of operators and adds them to the helm charts.

This makes updates of operators very hard, since we create our own package with operators. A proper separation of Operators (CRDs) and use of operators (CR) needs to be done.
The current helm chart cannot be deinstalled cleanly. There are always remainings, not deterministically (due to the fact that CR and CRDs are handled jointly and deleting CR and CRDs jointy creates unpredictable side effects.
The splitting into two helm charts would not be needed if operators are separated from the custom resource deployment.

Therefore, the design will have to be reworked, using a clear separation.

Develop API definitions for the Context Broker APIs in Swagger.

Integrate Rust static analysis/linting tool as part of the github workflow

https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/.github/workflows/quality.yaml

Device simulation for tests

Simulating device and send values on Kafka
- Plasmacutter
- Filter
Commandline tool in Rust
Press buttons to send data for demos

Implementation / adaptation of asset on-boarding in the fusion application

The current asset-on-boarding implementation will be adapted to use the import / export interfaces created in this phase of the project.

The interface numbers IF# are from #42

FleetMan AS Content Export (IF4) + FleetMan Asset Content Export (IF7) to package
FleetMan AS Content Import (IF6) + FabMan Asset Content Import (IF9) from package

The on-boarding process will also incorporate the creation of the asset in the NGSI-LD Content Broker.

Adaptation of the fusion application to call NGSI-LD Broker object creation (IF10)

Further requirements

The broker endpoint should be configurable.
Authentication details should be configurable.
Check the current implementation of "Onboard new asset" in Factory Manager

Build docker image for Debezium Adapater service, write helm chart & deploy on Kubernetes.

Implementation of import and export endpoints in the fusion application

2 Formats were decided upon:

A fusion specific serialisation format for transport (import / export) of fusion semantic data between Instances of the fusion application = Content
A semantic model that will be used in the rest of the ecosystem = Semantic

Following endpoints will be implemented and confirmed to conform to the formats defined in #4

IF1 EcoMan ATT Export - Type: Content, Format: JSON
IF2 EcoMan ATT Model Export - Type: Semantic, Format OWL
IF3 EcoMan ATT Import - Type: Content, Format: JSON
IF4 FleetMan AS Export - Type: Content, Format: JSON
IF5 FleetMan AS Model Export - Type: Semantic, Format OWL
IF6 FleetMan AS Import - Type: Content, Format: JSON
IF7 FleetMan Asset Export - Type: Content, Format: JSON
IF8 FleetMan Asset Model Export - Type: Semantic, Format OWL
IF9 FabMan Asset Import - Type: Content, Format: JSON

The Semantic Model Exports (IF2, IF7 and IF8) will contain the following imformation

Asset Properties (Fields) their optionality (mandatory / optional) and any applicable / defined thresholds
Asset Relationships, their cardinality, their type (peer/parent-child) and in the case of peer relationships, the relationship name

The following was removed as it is not required, The Asset Export is in NGSI-LD format and is handled in #46

IF# FabMan Asset Export - Format: Content

Provide sample payload to test the Debezium adapter

{
    "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
    "id": "urn:oisp:9be8df6c-0611-4ccc-8fec-dde3e11bdc7b:device:plasmacutter",
    "type": "https://industry-fusion.com/types/v0.9/plasmacutter",
    "https://industry-fusion.com/types/v0.9/state": {
      "type": "Property",
      "value": "https://industry-fusion.com/types/v0.9/state#on"
    },
    "https://industry-fusion.com/types/v0.9/workpiece": {
      "type": "Relationship",
      "object": "urn:ngsi-ld:Workpiece:workpiece002"
    },
    "refFilter": {
      "type": "Relationship",
      "object": "urn:oisp:9be8df6c-0611-4ccc-8fec-dde3e11bdc7b:device:filter"
    }
  }

 {
    "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
    "id": "urn:oisp:9be8df6c-0611-4ccc-8fec-dde3e11bdc7b:device:filter",
    "type": "https://industry-fusion.com/types/v0.9/filter",
    "https://industry-fusion.com/types/v0.9/refCartridge": {
      "type": "Relationship",
      "object": "urn:ngsi-ld:FilterCartridge:fc002"
    },
    "https://industry-fusion.com/types/v0.9/state": {
      "type": "Property",
      "value": "https://industry-fusion.com/types/v0.9/state#on"
    },
    "https://industry-fusion.com/types/v0.9/strength": {
      "type": "Property",
      "value": "0.75"
    }
  }
  {
    "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
    "id": "urn:ngsi-ld:FilterCartridge:fc001",
    "type": "https://industry-fusion.com/types/v0.9/filterCartridge"
  },
  {
    "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
    "id": "urn:ngsi-ld:FilterCartridge:fc002",
    "type": "https://industry-fusion.com/types/v0.9/filterCartridge",
    "https://industry-fusion.com/types/v0.9/inUseFrom": {
      "type": "Property",
      "value": "2021-10-25 13:51:51.0"
    },
    "https://industry-fusion.com/types/v0.9/inUseUntil": {
      "type": "Property",
      "value": "2021-10-25 13:54:55.4"
    },
    "https://industry-fusion.com/types/v0.9/wasteClass": {
      "type": "Property",
      "value": "WC2"
    }
  }
  {
    "@context": "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
    "id": "urn:ngsi-ld:Workpiece:workpiece002",
    "type": "https://industry-fusion.com/types/v0.9/workpiece",
    "https://industry-fusion.com/types/v0.9/steelGrade": {
      "type": "Property",
      "value": {
        "@type": "iff-pre:steelGrade/",
        "@value": "1.4301"
      }
    },
    "name": {
      "type": "Property",
      "value": "workpiece002"
    },
    "https://uri.fiware.org/ns/data-models#depth": {
      "type": "Property",
      "value": "0.1"
    },
    "https://uri.fiware.org/ns/data-models#height": {
      "type": "Property",
      "value": "100"
    },
    "https://uri.fiware.org/ns/data-models#weight": {
      "type": "Property",
      "value": "10"
    },
    "https://uri.fiware.org/ns/data-models#width": {
      "type": "Property",
      "value": "100"
    }
  }

Integrate Zalando with Scorpio Broker

NGSI-LD - Authorization concept

Fiware defines multi tenancy headers
Can it be abstracted only based on keycloak?
#13
task 1
task 2

Extend the feature of Apache Flink to accept SQL query/scripts using REST APIs (REST Specs TBD)
#55

Setup code scanning tool

A static code scanning tool must analyze every pull request

Create roles in Keycloak and develop code in Context Broker to integrate the keycloak roles to Context Broker, commit to Git

Add Flink & related components to helm charts

Flink helm charts for jobmanager and taskmanager
SQL Gateway
minio operator and deployment

Deploy the platform services on existing Kubernetes setup

Implementation of asset on-boarding in backend towards gateway

The topic of Asset ID across the whole ecosystem should be handled here
- Document the usage / existence of Asset IDs per System in the Ecosystem and what ID is passed from this system to the next over Interfaces
Required changes to the application.yaml of the Gateway services are implemented.
Functionality to generate the asset package that will be used in the future to on-board assets will be implemented. This package will hold the following for 1 asset as a zip file:
- the application.yaml configuration file
- the NGSI-LD initial payload (IF10)
- the asset content export (IF7)

See https://github.com/IndustryFusion/DigitalTwin-WIP/blob/main/Import_Export.md for the IF# Interface descriptions.

Develop Helm charts/use existing operator pattern for the platform services - Apache Flink, Apache Kafka, Postgresql, RocksDB

Project documentation concept

Create Operator which deploys DT-use-cases as StreamingSQL on Flink

Use-cases:

Plasmacutter/Filter
Conditional monitoring
OEE aggregation
Maintenance aggregation

Deploy Apache Flink with checkpointing, required for fault tolerance

Helm chart improvement

#91
Add Flink Services with SQL Gateway
Add All-In-One deployment from Scorpio for low resource testing
Add profiles for test and production deployment
Extend documentation for deployment

industryfusion / digitaltwin Goto Github PK

digitaltwin's People

Stargazers

Watchers

Forkers

digitaltwin's Issues

Recommend Projects

Recommend Topics

Recommend Org