opendevstack / ods-provisioning-app

Provisioning app, which triggers project and component provisions (including Jira / Confluence / BitBucket and OCP resource creation)

OpenDevStack Provisioning Application

Introduction

This application creates new OpenDevStack projects. It is the central entrypoint to get started with a new project / or provision new components based on quickstarters. It delegates the tasks to create / update resources to several services such as Jira, Confluence, Bitbucket and Jenkins.

Documentation

The sources are located in the antora folder at https://github.com/opendevstack/ods-provisioning-app/tree/master/docs/.

Development

Open the cloned provision application in your favorite IDE.

If you run the application locally, you will have to provide some addional information.

In case you want to use a local Nexus, you will have to create a gradle.properties file in the root to provide the Nexus credentials.

nexus_url=http://nexus-cd.192.168.56.101.nip.io
nexus_user=developer
nexus_pw=developer

You also have to ensure the Nexus certificate is integrated in the keystore of the JDK the IDE uses.

If you don’t want to use the internal Nexus, you will have to provide a gradle.properties file with the following content:

no_nexus=true

After startup the application is available at http://localhost:8080/.

You can login in with the Crowd admin user.

ods-provisioning-app's Issues

Raise Permissions on Confluence Groups when special permission set = true

We've detected the permissions assignation on the 3 different GROUPS for CONFLUENCE SPACES that have some permission-attributes DISABLED.

MANAGER -
STAKEHOLDER -
TEAM -

Upgrade provision app to spring boot 2 and spring sec 5

1.5.9 is horribly outdated

jira project provisioning does not link special permission set - when project key upper&lowercase

2018-10-30 12:27:37.954 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Creating new jira project

  | 2018-10-30 12:27:37.986 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/project 🆕false
  | 2018-10-30 12:27:38.711 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"self":"x/rest/api/latest/project/10105","id":10105,"key":"ITTESTPROV "}
  | 2018-10-30 12:27:38.729 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Created project: FullJiraProject{self=https://jx/rest/api/latest/project/10105 , key=ITTESTPROV, name=Test Provisioning, description=Test Provisioning, lead=null, uri=null, components=null, issueTypes=null, versions=null}
  | 2018-10-30 12:27:38.735 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Found permissionsets: 1
  | 2018-10-30 12:27:38.743 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Update permissionScheme ITTestprov PERMISSION SCHEME location: jira.permission.all.txt
  | 2018-10-30 12:27:38.746 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/permissionscheme 🆕true
  | 2018-10-30 12:27:39.833 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"expand":"permissions,user,group,projectRole,field,all","id":10104,"self":"x/rest/api/2/permissionscheme/10104","name":"ITTestprov PERMISSION SCHEME","description":"Test Provisioning"}
  | 2018-10-30 12:27:39.835 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/project/ITTestprov/permissionscheme 🆕true
  | 2018-10-30 12:27:40.571 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"errorMessages":["No project could be found with key 'ITTestprov'."],"errors":{}}
  | 2018-10-30 12:27:40.571 ERROR 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Could not update permissionset: ITTestprov
  | Exception: 404: Could not PUT > org.opendevstack.provision.model.jira.FullJiraProject: {"errorMessages":["No project could be found with key 'ITTestprov'."],"errors":{}}

This is a bug in JiraAdapter:107ff - where the key is uppercased on time - but then the original project key is used in :createPermissions .. which fails as the project was created with uppercase keys

Quick fix: JiraAdapter:220
String.format("%s%s/project/%s/permissionscheme", jiraUri, jiraApiPath, project.key);

Add Update guide

Add section to documentation how to update between releases or update to HEAD

Provision app -bitbucket webhook creation fails against bitbucket 5.13 while working against 5.11

Prov app code not compatible with 5.13 bitbucket :(
BitbucketAdapter:227ff (createWebHooksForRepository)

String url = String.format("%s/plugins/servlet/webhooks/repository/%s/%s/settings", bitbucketUri, project.key, repo.getSlug());
works against 5.11 - fails against 5.13

1. v5.11 uri: plugins/servlet/webhooks/repository/OPENDEVSTACK/ods-core/settings
2. v5.13 uri: plugins/servlet/webhooks/projects/OPENDEVSTACK/repos/ods-core

@tjaeschke @rattermeyer fyi

prov-cd pvc expects storage class name -> param name mismatch

https://github.com/opendevstack/ods-provisioning-app/blob/master/ocp-config/prov-cd/pvc.yml#L36
requires STORAGE_CLASS_NAME

while the param is called STORAGE_CLASS_DATA
https://github.com/opendevstack/ods-provisioning-app/blob/master/ocp-config/prov-cd/pvc.yml#L39

the latter STORAGE_CLASS_DATA is correct according to
https://github.com/opendevstack/ods-configuration-sample/blob/master/ods-provisioning-app/ocp-config/prov-cd/pvc.env.sample#L9

Provision app created jira project does see rights to add an attachment to an issue

I've just noticed than the default permissions on a jira project after the provisioning app, does not have the ability to add an attachment to an issue, could be the possibility to do this by default, -manager and -team

Provisioning App should set the Project Permissions only on group basis. No user references

When creating a new project, the project permissions for administrator/owner is set on a user Basis. they should be group based and optional user based. This is especially True for confluence and for bitbucket

Project returned from provision API contains jira project API REST url instead of browser url

"jiraUrl":"https://jira..../rest/api/latest/project/10308"

should instead contain
https://jira..../browse/project-key

so people can click on it and get directly to the JIRA project thru their browser

Provisioning App JS UI does not work with IE

With IE when an error is reported back from the API (eg lacking rights) the modal status/updatw dialog is not shown - leading to massive confusion. Same thing works well with google chrome

Create service framework to allow integration to other bugtrackers/scm & idmgt

Today integration to jira/confluence and bitbucket is hardwired.. also the dependency to crowd for Identitymgmt - we should provide a Plug and Play Framework so other Tools can be plugged in..

Add Multi cluster /environment support

Today the Provision App is tied to One target environment (thru One rundeck connection). in Case of multiple ones - it needs to be deployed multiple times - eg on each env

Harmonize log format so people can search easier for errors in the logs that happened during specific Project creation/update

Idea - prefix with Project key and where available quickstarter name

@michaelsauter thoughts?

Sporadic error from mail rendering

We have seen the following a few times now:

Can not update project, error Could not prepare mail; nested exception is
org.thymeleaf.exceptions.TemplateProcessingException:
Link base "jenkins-foo-cd.22ad.bar.openshiftapps.com" cannot be context relative (/) or
page relative unless you implement the org.thymeleaf.context.IWebContext interface
(context is of class: org.thymeleaf.context.Context) (mailTemplate:39)

It does not happen all the time, so not sure what the cause is.

Web UI / CURL gets 504 Gateway timeout after 30s during provisioning of quickstarters / projects ..

reproduces via API as well

19), continuing anyway.

• STATE: PROTOCONNECT => DO handle 0x66f160; line 1596 (connection #0)
  } [5 bytes data]

POST /api/v1/project HTTP/1.1
Host: prov-app-test.....
User-Agent: curl/7.58.0
Accept: /
Cookie: JSESSIONID=..; crowd.token_key=....
Content-Type: application/json; charset=utf-8
Content-Length: 352

} [352 bytes data]

• upload completely sent off: 352 out of 352 bytes
• STATE: DO => DO_DONE handle 0x66f160; line 1658 (connection #0)
• STATE: DO_DONE => WAITPERFORM handle 0x66f160; line 1783 (connection #0)
• STATE: WAITPERFORM => PERFORM handle 0x66f160; line 1799 (connection #0)
  100 352 0 0 100 352 0 11 0:00:32 0:00:30 0:00:02 0{ [5 bytes data]
• HTTP 1.0, assume close after body
• Marked for [closure]: HTTP/1.0 close after body
  < HTTP/1.0 504 Gateway Time-out
  < Cache-Control: no-cache
  < Connection: close

Spike: Permissions schemes in Jira - project roles / ...

Permissions schemes in Jira should type permissions to project roles and not directly to permission groups

Creation of Bitbucket project fails in case of long description (~480 chars) - causing downstream NPE

2018-11-28 10:47:19 loggerFileName=BRASS [http-nio-8080-exec-4] DEBUG o.o.p.services.BitbucketAdapter - https://...../rest/api/1.0/projects/null/permissions/groups?permission=PROJECT_WRITE&name=BI-dDevstack-Users - 404>{"errors":[{"context":null,"message":"Project null does not exist.","exceptionName":"com.atlassian.bitbucket.project.NoSuchProjectException"}]}
2018-11-28 10:47:20 loggerFileName=BRASS [http-nio-8080-exec-4] ERROR o.o.p.c.ProjectApiController - An error occured while provisioning project: {}
java.lang.NullPointerException: null
at org.opendevstack.provision.services.BitbucketAdapter.createBitbucketProjectsForProject(BitbucketAdapter.java:126)
at org.opendevstack.provision.controller.ProjectApiController.createDeliveryChain(ProjectApiController.java:286)
at org.opendevstack.provision.controller.ProjectApiController.addProject(ProjectApiController.java:143)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

allow to disable mail send from provision app

currently provision app tries to send an email upon successfull provisioning. When configured wrong it will just dump the error, and still return success.

We need a way to disable email where not needed, e.g. if API is used and embedded in another app

Errors during project creation are not exposed

During project creation, things can go wrong. For example, it might not be possible to create a PVC because there is not enough disk quota left. This can be seen in the Rundeck logs, but it is not immediately visible as the job is marked as successful. To notice the problem, one needs to figure out that the failing Jenkins pod is due to a missing PVC, and then to look into Rundeck to see the actual error.

Could we somehow make this more obvious?

cd_user not added when provisioning new bitbucket repo (thru quickstarter)

bug in the codebase - createRepositoryPermissions - tries to set cd_user as group .. this is wrong.

wrong (lean) project permissions in Bitbucket, when tight permission set configured

There is a default permission after the provisioning app is executed on BITBUCKET.

First is adding the user to a group called xxxxxxxdevstack-users, and, after that the same group is added into the $NEW-CREATED-PROJECT.

What is the problem?
Problem is that this group xxxxxxxdevstack-users has access to $ALL-PROJECTS and can write into them.

Could be the possibility to remove this default Group Access for the provisioning app?

mismatch in bc and 0.1jenkins shared lib

prov app production latest fails ...

[prov-cd-prov-app-dev] Running shell script
+ oc patch bc prov-app --patch '
   spec:
     output:
       to:
         kind: ImageStreamTag
         name: prov-app:3-8d0fa827
     runPolicy: Serial
     source:
       type: Binary
     strategy:
       type: Docker
       dockerstrategy: {}
    ' -n prov-dev
The BuildConfig "prov-app" is invalid: 
* spec.source.git: Invalid value: "": may not be set when binary is also set
* spec.source.binary: Invalid value: "": may not be set when git is also set

I have hit the same issue on quickstarters as well - fix is with @michaelsauter update bc of prov app

bc mismatch in provisioning app configuration

Currently there seems to be a template mismatch. After importing the provisioning app with tailor I get the following error as soon as a jenkins build has been triggered in the build console:

[prov-cd-prov-app-dev] Running shell script
+ oc patch bc prov-app --patch '
   spec:
     output:
       to:
         kind: ImageStreamTag
         name: prov-app:3-8d0fa827
     runPolicy: Serial
     source:
       type: Binary
     strategy:
       type: Docker
       dockerstrategy: {}
    ' -n prov-dev
The BuildConfig "prov-app" is invalid:
* spec.source.git: Invalid value: "": may not be set when binary is also set
* spec.source.binary: Invalid value: "": may not be set when git is also set

The pipeline uses the shared-lib latest

Provide hook/interface to create access groups used for atlassian during provisioning

Most users of Open Dev stack will have crowd and an id mgmt behind such as azure ad.. de need to allow them to create groups on the fly that can be be assigned as access groups to the newly created space/projects

Seed production pipeline

Now that we have a webhook proxy, we usually do not create Jenkins pipelines in OpenShift. This works well for repositories where we assume that development happens. The provisioning app however is likely to be just deployed, and never developed by users. For that case, it is best to seed one pipeline in the getting started guide.

Openshift projects are created with current user as admin, even in case of special permission set

Today we pass down the current (logged in) principal to rundeck and subsequently to openshift (as project role admin) example below

tech_integration admin -> the currently logged in user

rather than taking the provided project admin (in case of special permission set = true)

https://github.com/opendevstack/ods-provisioning-app/blob/master/src/main/java/org/opendevstack/provision/services/RundeckAdapter.java#L185

Provision app fails with 401 not authenticated in case crowd's sso cookie name was changed - make configurable

On crowd you can change the sso cookie name, like below

the codebase assumes crowd.token_key to be the standard name - and hence sso will not work if one changed this name in the crowd server config

https://github.com/opendevstack/ods-provisioning-app/blob/master/src/main/java/org/opendevstack/provision/util/CrowdCookieJar.java#L78

Project specific cd_user

We had some issues with a global cd_user (e.g. if password is changed, there are lots of places that need to be adjusted). This would be easier if each project had its own cd_user.

Just raising this question / issue here - not sure how that would work and what implications this has.

FYI @clemensutschig @gerardcl @stitakis @rattermeyer

Provision app does NOT set admin group into project / repo permissions

Extreme cornercase - API is used with a technical user - now a user logs with admin rights logs in - sees the project and provision will fail, as the repository cannot be created (as he lacks rights).

Provisioning app does not return error if rundeck adapter fails

provisioning app should support lower level container based projects

The provisionig app should also support lower level deployments based on simple and complex Dockerfile container definitions.

git url created from Bitbucket 5.13.0+ does not follow conventions

Normally, we expect in the downstream jobs (e.g. when creating the pipline build configuration), that the http git url contains the technical user.
The logic sits in BitbucketAdapter. It expects that the clone url returned by bitbucket contains the username of the currently logged in user.
This behavior seems to have changed from Bitbucket 5.13.0+ on. The clone url does no longer contain a username.
Thus the URL is not correctly created for downstream jobs.

Refactor prov app services & fix sonarqube findings

Plenty of code duplication.. especially on the http post/put side jira/confluence/bitbucket-adapter

Sonarqube coverage below 50÷

Add default created -GROUPS to roles by default scheme on JIRA

There could be the option to add the:
a) $project-TEAM group on the Developers role

b) $project-MANAGER group on the **Administrators role ;

c) $project-STAKEHOLDER group on the **Stakeholders role ;

In to the software default scheme on Jira after the provisioning app is executed?

Role for -TEAM into Developers
Role for -MANAGER into Administrators
Role for -STAKEHOLDER into Stakeholders

Since Default permissions scheme are modified, anyone is inside the permissions scheme, and the $ProjectLeader has to add manually one by one for first time, could be much better if this is fully automated.

Cheers,
Borja

Provisioniong app should show rundeck job reference(s) in summary

Today one needs to look into rundeck - and find the job that prov app triggered,.. we have the execution in the prov app already .. e.g.

{"id":34,"href":"https://rundeck.../api/24/execution/34","permalink":"https://rundeck.....com/project/Quickstarters/execution/show/34","status":"running","project":"Quickstarters" ...... }

Provision app created jira project does seed rights to read a project for -readonly groups

I've just noticed than the default permissions on a jira project after the provisioning app, does not have the ability to read a project for -stakeholders group.

Wrong openshift console / project url generated

Example:

• Openshift Development Project: https://inh-ocdev.eu.boehringer.com/consolepos-dev
• Openshift Test Project: https://inh-ocdev.eu.boehringer.com/consolepos-test

Should be
https://inh-ocdev.eu.boehringer.com/console/project/pos-dev

CD user secrets are missing

add jenkins / openshift / bitbucket project URLs to jira space as shortcuts

today the jira space created only contains confluence links .. it should contain ALL necessary links

1. bitbucket project
2. openshift envs
3. jenkins link

Unify prov-dev and prov-test OCP config

Between the environments prov-dev and prov-test, the YAML templates should be exactly the same. Those two environments should only differ in parameters.

Split / Separate "tracking projects/documentation" from source control

Hi All This in regards to the Atlassian suite w.r.t to BITBUCKET as this is a code repository would recommend to
segregate the usage from JIRA and Confluence. As the user/project requirement in some cases be different for all three applications.
And i would like to recommend to provision all three application separately

Cheers

Missing comment permissions in created JIRA permission set

A Member of the *-MANAGER or *-STAKEHOLDER Team cannot comment on an issue.
The respective project permission scheme should be adjusted like this:

BITBUCKET_HOST is missing in ocp-config

Missing role bindings?

@michaelsauter I have a question regarding the import of the provisioning app via tailor. I had to add the system:image-puller role for the cd namespace to the system:serviceaccount:prov-cd:default manually, after I had imported the app via tailor and started to use Jenkins in the prov-cd project. Before this no Jenkins slaves could be spawned because of the missing role.
Are we missing something in the ocp-config of the provisioning app or is it even possible to export and import the necessary role bindings?

Upgrade prov App to latest jenkins Shared lib

Ready for v1 release.

Upgrade provision app to spring boot v.1x latest (1.5.15)

fix for ODS v1 - fup from #39

Provision app should support different (jira/confluence) project types

today - the template used for jira & confluence is hardcoded (thru a config file/map) and for jira in the source

#Data for confluence space creation confluence.blueprint.key=com.atlassian.confluence.plugins.confluence-software-project:sp-space-blueprint

This should be at least configurable

No Default Notification Scheme assigned into Jira projects upon creation

Hi
There is no Default Notification Scheme assigned after a Jira project is created and or the provisioning app is executed, and, has to be added each time manually.

Could be possible to add this feature for mail Notifications for any action on Jira?

Refactor http Call code in various Adapters

4 times the roughly same code to do a rest Call to jira / confluence / bitbucket and rundeck. This is just Bad legacy code.

jenkins build of provision app does not surface junit results / failed test outputs

Already fixed in the boilerplates (be-spring-boot) - just uptake into prov-app
https://github.com/opendevstack/ods-project-quickstarters/blob/master/boilerplates/be-springboot/Jenkinsfile#L40

vs
https://github.com/opendevstack/ods-provisioning-app/blob/master/Jenkinsfile#L47