Coder Social home page Coder Social logo

application-connector-manager's People

Contributors

ajinkyapatil8190 avatar akgalwas avatar dependabot[bot] avatar disper avatar grego952 avatar koala7659 avatar kyma-bot avatar m00g3n avatar mmitoraj avatar mvshao avatar nataliasitko avatar pbochynski avatar void404 avatar

Watchers

avatar avatar avatar avatar

Forkers

grego952 m00g3n nataliasitko koala7659 mvshao disper mmitoraj void404 akgalwas halamix2

application-connector-manager's Issues

App Connector Module - Add Compass Runtime Agent to the module

Description
The module must contain the Compass Runtime Agent

Acceptance criteria

  • Application Connector Module contains the Compass Runtime Agent
  • The Compass Runtime Agent's deployment is scaled to zero when the secret with the configuration doesn't exist

Update the AC module documentation to the current state of the module

Description

This issue is a follow-up to #6. The documentation of the AC module must be updated to the current state of the module.

Area

  • Application Connector module

Reasons
Currently, not all features are available in the module. As soon as they are, they must be documented.

AC

  • describe the new ApplicationConnector CR
  • Remove **NOTE:** At this stage of development, the ApplicationConnector Custom Resource Definition (CRD) contains only one parameter for testing. from README
  • Update the 01-10-installation.md doc
  • Remove mentions of the disableLegacyConnectivity flag
  • Update Configurable parameters in the 05-10-application-connector-chart.md doc
  • Provide missing descriptions for CompassConnection CRD

Assignees

@grego952

Attachments
#22
#6

Fix GH Actions to be required for merge

At the moment the GH Action running integration tests k3d can fail on PR and it does not prevents the code merge.

  • Fix GH Action to be required for merge
  • Review other GH Actions if they are setup correctly

[Test Case] - Test the integrations with SAP systems

Description

Implement tests integrating external Java application with Kyma.

We do not have an automated test for testing such integrations and we want to track such scenarios.

Reasons

There are some integrations written in Java that are working with Kyma. The main reason is we want to make sure that we do not break any existing integrations.

App Connector Module - Implement host name setup for Istio Gateway with Kyma cluster domain

Description

We need to setup Application Connector Istio Gateway with correct cluster domain name. In existing POC the gateway hostname is statically set to some example value.

      hosts:
        - "gateway.kyma.example.com"

This will require modify yaml object installed by the operator.

Acceptance criteria

  • Investigation
    • Investigation is required how the domain name will be provided. Possible approaches might be:
      • In Application Connector CR
      • Read from the Kyma Cluster
      • Provided by control plane
  • Implement according to specification

Suggestion:

For the host name use value passed in Module CR, when missing use value from Istio kyma-gateway

App Connector Module - [Optional task] Ensure that Runtime Agent is enabled even if Istio is missing

This functionality should be implemented if we decide to enable the module with using old reconciler.

That means that Compass Manager was not ready on control plane to create Compass Connection token for enabled module at the checkpoint time.

In such a case we want to install only the Operator that will install all services for the module to avoid dependency to Istio module.

When Istio is missing - we cannot install Application Connector module but at the same time we must ensure that Runtime Agent service is enabled to used one time token before it expires.

We should make sure that operator is able to detect missing Istio install only Runtime Agent service from the module.

Ensure that central-application-connectivity-validator service is scaling correctly under high load of events

Description

I observed following problem with central-application-connectivity-validator service after hpa scales it under high load of events - pods are created correctly but new replicas cannot get any http requests. When initially scaled - all replicas get requests. We need to investigate and fix it.

This issue might be happening because of multiple reasons:

  • Testing setup
  • Gardener cluster configuration
  • Kubernetes configuration
  • Bad autoscaler used

This issue can be done as part of the migration of validator into Kyma module

Implement Application Connector module logic

Description

We need to implement Application Connector Operator responsible for deploying Application Connector components.

Acceptance criteria:

  • The operator must install the following components:
    • Application Gateway
    • Connectivity Validator
    • Compass Runtime Agent
  • Application Connector module configuration CRD must contain all parameters defined in Application Connector's charts
  • Application Connector component must not be installed if Istio CRDs such as Istio Gateway or Istio Virtual Service are not available in the cluster
  • Application Connector components can be installed even if configuration for Compass Runtime Agent is not available yet
  • Module Template CR must contain accurate status information.

Reasons

This issue is a part of this epic.

Add caching layer to central-application-gateway to improve performance

Description
Application Gateway must cache Kubernetes API server calls to improve performance. The following objects data must be cached:

  • Application CRD objects
  • Credentials

Reasons
Some experiments were performed to understand the performance characteristics of the Application Gateway. Mock target API deployed inside Kubernetes cluster was used to measure operations performed by the Application Gateway. The experiments were performed with ddosify tool.

Although Application Gateway uses Horizontal Autoscaling, the experiments revealed that the maximal number of request per second was much smaller than expected. Analysis shown that the problem is directly caused by long response time of Kubernetes API server calls.

Depends on
#118 - if load test shows we are performing well, adding a caching layer could become obsolete

App Connector Module - Add Runtime Agent [EPIC]

Description

After Enabling Compass Manager on KCP we are finally ready to make Runtime Agent the part of Application Connector module.

This task will include:

Required for first release:

  • Insert manifests for Runtime Agent to .yaml file managed by application-connector-manager
  • Add code to enable and disable Runtime Agent deployment based on the existence of a secret compass-agent-configuration (the Compass Runtime Agent's deployment is scaled to zero when the secret with the configuration doesn't exist)
  • fixes of tests #17 (@koala7659 , @m00g3n )
  • (can be done in parallel)

Rollout of ACM:

  • #212
  • Cleanup
    • Remove Compass Runtime Agent helm chart from installed Kyma Resources (make sure they are not installed an any more)
      • PR-208 - Move ACM CRDs from kyma repository
      • PR-18606 - Remove compass-runtime-agent from kyma resources
      • code-review
    • #195

Default Application Connector gateway TLS adjustments

Since Kyma 2.15 ECDHE-RSA-AES256-SHA and ECDHE-RSA-AES128-SHA cipher suites are deprecated because of Kyma security team recommendations.
In Kyma 2.19 they were removed from kyma-gateway.

We should follow that change and adjust Application Connector Istio gateway to remove deprecated cipher suites from it .

For more details:
Kyma 2.19 release notes - https://github.com/kyma-project/kyma/releases/tag/2.19.0
Issue for Kyma Gateway - kyma-project/kyma#17616
Pull request for Kyma Gateway - kyma-project/kyma#18085

App Connector Module - Application Connector migration plan

At this point we think there is no need to do anything. The migration is not needed as:

  • the only thing that changes is the tool that applies AC components (Reconciler->AC module Operator)
  • the contents of the chart used by the Reconciler is identical with the yaml applied by the AC module Operator

sidecar.istio.io/inject is set to false by default in Central-Application-Gateway

Description

Components in kyma-system must have sidecar.istio.io/inject set to true by default to work correctly. After installing Central-Application-Gateway on the cluster, we can observe that the deployment (on the cluster) in the labels section has the entry sidecar.istio.io/inject: "false", and in the annotations section has sidecar.istio.io/inject: "true". The deployments.yaml file for Central-Application-Gateway contains only an entry for Istio sidecar in the annotations section set to true. This condition makes Central-Application-Gateway have sidecar disabled by default.

Central-Application-Gateway deployment.yaml

    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
      labels:
        app: {{ .Chart.Name }}
        release: {{ .Release.Name }}

Actual deployment on cluster

    metadata:
      creationTimestamp: null
      labels:
        app: central-application-gateway
        release: application-connector
        sidecar.istio.io/inject: 'false'
      annotations:
        sidecar.istio.io/inject: 'true'

After investigating, I discovered that in Busola, in the deployment edit window, there is a switch that can change the value of the sidecar.istio.io/inject entry from true to false. However, it adds/modifies the Istio entry only in the label section.

Kyma version: 2.7.0
Kubernetes version: v1.22.12

Expected result

sidecar.istio.io/inject is set to true by default

Actual result

sidecar.istio.io/inject is set to false by coincidences

Steps to reproduce

Deploy a Kyma on the fresh cluster, and check if the Istio sidecar is properly configured and enabled on Central-Application-Gateway

Remove deprecated fields from Application CRD

Description

There is a bunch of fields in Application CRD that were introduced before Kyma 2.x and are no longer used (e.g. fields previously used by Application Broker). Those fields must be removed and Application CRD client regenerated.

Reasons

We must clean up Application CRD to reduce maintenance effort

App Connector Module - Implement webhook to validate domain name from CR

Description

We need to create validation logic for the cluster domain provided in App Connector Custom resource.

It is a good idea to build such a logic as validation webhook inside application-connector-manager service with endpoint

If there is no provided domain value, or provided domain cannot be parsed, we replace it with cluster domain taken from the kubernetes cluster.

Reasons

We need to provide fallback mechanism for setting the correct cluster domain if user does not provide any.

Attachments

It is follow up story of #23

App Connector Module - Connect security tools to the module

Reason
We want to be secure and compliant with the release process

Description
Connect Application Connector Manager to security scanners

Acceptance Criteria

  • Application Connector Manager is scanned regularly with other modules and vulnerabilities are visible on security dashboards
  • All obligatory security tools are supported (probably Mend/Blackduck/Checkmarx)

Prepare Application Connector benchmarks

Description

We must introduce benchmarks that exercise Application Connector including all authorisation methods. The benchmarks should have similar structure and test cases to the Application Connector tests.

Reasons

In order to be able to spot performance regression we need to automate benchmarking. Some part of the existing tests can be used, such as mock services.

Related issues: #150 .

App Connector Module - Add custom linter configuration for golangci-lint and adjust the code

Description

At the moment application-connector-manager does not use any custom linter configuration for checking the code on GH Action CI. As a result GH Action golangci-lint job is running with default settings. We should create our own configuration and put it into golangci.yml file in the root of the project.

As a base we can use existing configs from lifecycle-manager , nats-manager or compass-manager as a base.

We should make sure that all linter checks are passed, and adjust the code accordingly.

Reasons

Custom linter configuration will allow us to maintain higher standards of the code quality.

App Connector Module - Implement getting metrics and scaling app-connector pods with HPA

Description

In POC of the Application connector module metrics are not correctly collected by Horizontal Pod Autoscaler objects. Following errors are logged in HPA object status section:

Warning FailedComputeMetricsReplicas 2s horizontal-pod-autoscaler invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics AP

Acceptance criteria

Metrics should be correctly collected and deployments should be scaled according to HPA configuration.

Create Application Connector module [EPIC]

Description

Application Connector needs to be moved to the modular architecture according to this concept.

Reasons

Application Connector should be more independent so that it must be possible to install, upgrade and operate it separately without dependencies on other Kyma modules.

Acceptance criteria

Release 1.0.0:

  • Application Connector module must have self-healing properties.
  • Operator's configuration must be defined dedicated CRD
  • Module must contain legacy Application Connector Components : Application Gateway and Connectivity Validator.

Release 1.1.0:

  • Module must contain all Application Connector Components : Application Gateway, Connectivity Validator, and Compass Runtime Agent.

Prerequisites:

Workplan:

Until the end of September, Release MVP:

Release required tasks, until 25 October: Milestone 1.0.0

Release hardening, until mid of November: Milestone 1.1.0

Until the end of November: Milestone 1.2.0

Stretch:

Consider to close as non-relevant:

Additional information

README improvement

Description
There are some small documentation bugs in the README file that could be improved.

Area

Reasons

Assignees

@grego952

Attachments

Troubleshooting guide and on call guides

Description

To be ready for the go-live, we have to create an on-call guide for the Application Connector Manager. This is also a pre-requisite for the Microdelivery of the Application Connector Manager.

Possible location for the on-call guide: https://github.tools.sap/kyma/documentation/tree/main/kyma-internal/on-call-guides/mps

AC:

  • Document the common use-cases / possible incidents we have to expect when the Application Connector Manager runs in a productive context

Area

  • Application Connector Manager

Reasons

Mandatory pre-requisite before we can go-live and part of the SAP Product Standards.

Assignees

@kyma-project/technical-writers

Attachments

  • How to disable reconciliation
  • Consider if it would be possible to see if Kymas are healthy (e.g. see if there are issues with other modules)

Prepare customer-facing documentation for Application Connector Module

Description
For the new modularization approach, we need documentation that is common for all the modules, e.g. general description (landing page), getting started, module configuration, etc.

AC

  • Your audience is the end-user so the documentation must be written for them (not for the team that builds this module, or other SAP teams that wish to build a module, not for SRE, etc). Remember about the context. How much user knows by now (probably just high level knowledge about Kyma as a whole and that there are modules).
  • The documentation must be consumable by the kyma-project.io home page (split into several information types/md files)
  • Create a docs folder following the standard structure proposed in -> this issue
  • In each module's documentation include:
    • General description of the functionality (if the module is mandatory, mention it!)
    • Getting Started - instructions on how to use the module. The module docs must mention the prerequisite that the module must be enabled. They must also include instructions on how to install/enable a module.
    • Module configuration
    • Instructional documents (currently called "dev tutorials" and "operational guides") - must be isolated, i.e. no dependencies on other tutorials or modules. An example.
    • If needed, troubleshooting
    • Instructions on how to give feedback

Reasons

Assignees

@kyma-project/technical-writers

Attachments
kyma-project/kyma#16421

App Connector Module - Prepare Module bugfix release process

PR

We need to document the release process for the Application Connector module,

Acceptance criteria

  • Must be reliable

  • Must be documented

  • Must be quick - be able to release in 2h since the fix is prepared

  • Must be finally discussed and approved with @zhoujing2022

  • Technical review

  • Language review (in progress)

  • adapt to review remarks

  • @zhoujing2022 review (in progress)

  • adapt to review remark

  • Process verification

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.