opendevstack / ods-provisioning-app Goto Github PK
View Code? Open in Web Editor NEWProvisioning app, which triggers project and component provisions (including Jira / Confluence / BitBucket and OCP resource creation)
License: Apache License 2.0
Provisioning app, which triggers project and component provisions (including Jira / Confluence / BitBucket and OCP resource creation)
License: Apache License 2.0
Between the environments prov-dev
and prov-test
, the YAML templates should be exactly the same. Those two environments should only differ in parameters.
There could be the option to add the:
a) $project-TEAM group on the Developers role
b) $project-MANAGER group on the **Administrators role ;
c) $project-STAKEHOLDER group on the **Stakeholders role ;
In to the software default scheme on Jira after the provisioning app is executed?
Role for -TEAM into Developers
Role for -MANAGER into Administrators
Role for -STAKEHOLDER into Stakeholders
Since Default permissions scheme are modified, anyone is inside the permissions scheme, and the $ProjectLeader has to add manually one by one for first time, could be much better if this is fully automated.
Cheers,
Borja
4 times the roughly same code to do a rest Call to jira / confluence / bitbucket and rundeck. This is just Bad legacy code.
1.5.9 is horribly outdated
Today one needs to look into rundeck - and find the job that prov app triggered,.. we have the execution in the prov app already .. e.g.
{"id":34,"href":"https://rundeck.../api/24/execution/34","permalink":"https://rundeck.....com/project/Quickstarters/execution/show/34","status":"running","project":"Quickstarters" ...... }
I've just noticed than the default permissions on a jira project after the provisioning app, does not have the ability to add an attachment to an issue, could be the possibility to do this by default, -manager and -team
Most users of Open Dev stack will have crowd and an id mgmt behind such as azure ad.. de need to allow them to create groups on the fly that can be be assigned as access groups to the newly created space/projects
With IE when an error is reported back from the API (eg lacking rights) the modal status/updatw dialog is not shown - leading to massive confusion. Same thing works well with google chrome
Hi All This in regards to the Atlassian suite w.r.t to BITBUCKET as this is a code repository would recommend to
segregate the usage from JIRA and Confluence. As the user/project requirement in some cases be different for all three applications.
And i would like to recommend to provision all three application separately
Cheers
bug in the codebase - createRepositoryPermissions - tries to set cd_user as group .. this is wrong.
Ready for v1 release.
currently provision app tries to send an email upon successfull provisioning. When configured wrong it will just dump the error, and still return success.
We need a way to disable email where not needed, e.g. if API is used and embedded in another app
Prov app code not compatible with 5.13 bitbucket :(
BitbucketAdapter:227ff (createWebHooksForRepository)
String url = String.format("%s/plugins/servlet/webhooks/repository/%s/%s/settings", bitbucketUri, project.key, repo.getSlug());
works against 5.11 - fails against 5.13
Today integration to jira/confluence and bitbucket is hardwired.. also the dependency to crowd for Identitymgmt - we should provide a Plug and Play Framework so other Tools can be plugged in..
Currently there seems to be a template mismatch. After importing the provisioning app with tailor I get the following error as soon as a jenkins build has been triggered in the build console:
[prov-cd-prov-app-dev] Running shell script
+ oc patch bc prov-app --patch '
spec:
output:
to:
kind: ImageStreamTag
name: prov-app:3-8d0fa827
runPolicy: Serial
source:
type: Binary
strategy:
type: Docker
dockerstrategy: {}
' -n prov-dev
The BuildConfig "prov-app" is invalid:
* spec.source.git: Invalid value: "": may not be set when binary is also set
* spec.source.binary: Invalid value: "": may not be set when git is also set
The pipeline uses the shared-lib latest
During project creation, things can go wrong. For example, it might not be possible to create a PVC because there is not enough disk quota left. This can be seen in the Rundeck logs, but it is not immediately visible as the job is marked as successful. To notice the problem, one needs to figure out that the failing Jenkins pod is due to a missing PVC, and then to look into Rundeck to see the actual error.
Could we somehow make this more obvious?
| 2018-10-30 12:27:37.986 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/project 🆕false
| 2018-10-30 12:27:38.711 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"self":"x/rest/api/latest/project/10105","id":10105,"key":"ITTESTPROV "}
| 2018-10-30 12:27:38.729 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Created project: FullJiraProject{self=https://jx/rest/api/latest/project/10105 , key=ITTESTPROV, name=Test Provisioning, description=Test Provisioning, lead=null, uri=null, components=null, issueTypes=null, versions=null}
| 2018-10-30 12:27:38.735 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Found permissionsets: 1
| 2018-10-30 12:27:38.743 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Update permissionScheme ITTestprov PERMISSION SCHEME location: jira.permission.all.txt
| 2018-10-30 12:27:38.746 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/permissionscheme 🆕true
| 2018-10-30 12:27:39.833 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"expand":"permissions,user,group,projectRole,field,all","id":10104,"self":"x/rest/api/2/permissionscheme/10104","name":"ITTestprov PERMISSION SCHEME","description":"Test Provisioning"}
| 2018-10-30 12:27:39.835 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Call to: https://x/rest/api/latest/project/ITTestprov/permissionscheme 🆕true
| 2018-10-30 12:27:40.571 DEBUG 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : {"errorMessages":["No project could be found with key 'ITTestprov'."],"errors":{}}
| 2018-10-30 12:27:40.571 ERROR 1 --- [nio-8080-exec-3] o.o.provision.services.JiraAdapter : Could not update permissionset: ITTestprov
| Exception: 404: Could not PUT > org.opendevstack.provision.model.jira.FullJiraProject: {"errorMessages":["No project could be found with key 'ITTestprov'."],"errors":{}}
This is a bug in JiraAdapter:107ff - where the key is uppercased on time - but then the original project key is used in :createPermissions .. which fails as the project was created with uppercase keys
Quick fix: JiraAdapter:220
String.format("%s%s/project/%s/permissionscheme", jiraUri, jiraApiPath, project.key);
reproduces via API as well
19), continuing anyway.
POST /api/v1/project HTTP/1.1
Host: prov-app-test.....
User-Agent: curl/7.58.0
Accept: /
Cookie: JSESSIONID=..; crowd.token_key=....
Content-Type: application/json; charset=utf-8
Content-Length: 352
} [352 bytes data]
fix for ODS v1 - fup from #39
@michaelsauter I have a question regarding the import of the provisioning app via tailor. I had to add the system:image-puller role for the cd namespace to the system:serviceaccount:prov-cd:default manually, after I had imported the app via tailor and started to use Jenkins in the prov-cd project. Before this no Jenkins slaves could be spawned because of the missing role.
Are we missing something in the ocp-config of the provisioning app or is it even possible to export and import the necessary role bindings?
We have seen the following a few times now:
Can not update project, error Could not prepare mail; nested exception is
org.thymeleaf.exceptions.TemplateProcessingException:
Link base "jenkins-foo-cd.22ad.bar.openshiftapps.com" cannot be context relative (/) or
page relative unless you implement the org.thymeleaf.context.IWebContext interface
(context is of class: org.thymeleaf.context.Context) (mailTemplate:39)
It does not happen all the time, so not sure what the cause is.
Idea - prefix with Project key and where available quickstarter name
@michaelsauter thoughts?
When creating a new project, the project permissions for administrator/owner is set on a user Basis. they should be group based and optional user based. This is especially True for confluence and for bitbucket
Now that we have a webhook proxy, we usually do not create Jenkins pipelines in OpenShift. This works well for repositories where we assume that development happens. The provisioning app however is likely to be just deployed, and never developed by users. For that case, it is best to seed one pipeline in the getting started guide.
2018-11-28 10:47:19 loggerFileName=BRASS [http-nio-8080-exec-4] DEBUG o.o.p.services.BitbucketAdapter - https://...../rest/api/1.0/projects/null/permissions/groups?permission=PROJECT_WRITE&name=BI-dDevstack-Users - 404>{"errors":[{"context":null,"message":"Project null does not exist.","exceptionName":"com.atlassian.bitbucket.project.NoSuchProjectException"}]}
2018-11-28 10:47:20 loggerFileName=BRASS [http-nio-8080-exec-4] ERROR o.o.p.c.ProjectApiController - An error occured while provisioning project: {}
java.lang.NullPointerException: null
at org.opendevstack.provision.services.BitbucketAdapter.createBitbucketProjectsForProject(BitbucketAdapter.java:126)
at org.opendevstack.provision.controller.ProjectApiController.createDeliveryChain(ProjectApiController.java:286)
at org.opendevstack.provision.controller.ProjectApiController.addProject(ProjectApiController.java:143)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
We had some issues with a global cd_user
(e.g. if password is changed, there are lots of places that need to be adjusted). This would be easier if each project had its own cd_user
.
Just raising this question / issue here - not sure how that would work and what implications this has.
Normally, we expect in the downstream jobs (e.g. when creating the pipline build configuration), that the http git url contains the technical user.
The logic sits in BitbucketAdapter. It expects that the clone url returned by bitbucket contains the username of the currently logged in user.
This behavior seems to have changed from Bitbucket 5.13.0+ on. The clone url does no longer contain a username.
Thus the URL is not correctly created for downstream jobs.
Example:
• Openshift Development Project: https://inh-ocdev.eu.boehringer.com/consolepos-dev
• Openshift Test Project: https://inh-ocdev.eu.boehringer.com/consolepos-test
Should be
https://inh-ocdev.eu.boehringer.com/console/project/pos-dev
There is a default permission after the provisioning app is executed on BITBUCKET.
First is adding the user to a group called xxxxxxxdevstack-users, and, after that the same group is added into the $NEW-CREATED-PROJECT.
What is the problem?
Problem is that this group xxxxxxxdevstack-users has access to $ALL-PROJECTS and can write into them.
Could be the possibility to remove this default Group Access for the provisioning app?
https://github.com/opendevstack/ods-provisioning-app/blob/master/ocp-config/prov-cd/pvc.yml#L36
requires STORAGE_CLASS_NAME
while the param is called STORAGE_CLASS_DATA
https://github.com/opendevstack/ods-provisioning-app/blob/master/ocp-config/prov-cd/pvc.yml#L39
the latter STORAGE_CLASS_DATA
is correct according to
https://github.com/opendevstack/ods-configuration-sample/blob/master/ods-provisioning-app/ocp-config/prov-cd/pvc.env.sample#L9
Plenty of code duplication.. especially on the http post/put side jira/confluence/bitbucket-adapter
Sonarqube coverage below 50÷
"jiraUrl":"https://jira..../rest/api/latest/project/10308"
should instead contain
https://jira..../browse/project-key
so people can click on it and get directly to the JIRA project thru their browser
today - the template used for jira & confluence is hardcoded (thru a config file/map) and for jira in the source
#Data for confluence space creation confluence.blueprint.key=com.atlassian.confluence.plugins.confluence-software-project:sp-space-blueprint
This should be at least configurable
today the jira space created only contains confluence links .. it should contain ALL necessary links
Permissions schemes in Jira should type permissions to project roles and not directly to permission groups
Today we pass down the current (logged in) principal to rundeck and subsequently to openshift (as project role admin) example below
tech_integration admin -> the currently logged in user
rather than taking the provided project admin (in case of special permission set = true)
Hi
There is no Default Notification Scheme assigned after a Jira project is created and or the provisioning app is executed, and, has to be added each time manually.
Could be possible to add this feature for mail Notifications for any action on Jira?
On crowd you can change the sso cookie name, like below
the codebase assumes crowd.token_key
to be the standard name - and hence sso will not work if one changed this name in the crowd server config
Today the Provision App is tied to One target environment (thru One rundeck connection). in Case of multiple ones - it needs to be deployed multiple times - eg on each env
The provisionig app should also support lower level deployments based on simple and complex Dockerfile container definitions.
prov app production latest fails ...
[prov-cd-prov-app-dev] Running shell script
+ oc patch bc prov-app --patch '
spec:
output:
to:
kind: ImageStreamTag
name: prov-app:3-8d0fa827
runPolicy: Serial
source:
type: Binary
strategy:
type: Docker
dockerstrategy: {}
' -n prov-dev
The BuildConfig "prov-app" is invalid:
* spec.source.git: Invalid value: "": may not be set when binary is also set
* spec.source.binary: Invalid value: "": may not be set when git is also set
I have hit the same issue on quickstarters as well - fix is with @michaelsauter update bc of prov app
Already fixed in the boilerplates (be-spring-boot) - just uptake into prov-app
https://github.com/opendevstack/ods-project-quickstarters/blob/master/boilerplates/be-springboot/Jenkinsfile#L40
vs
https://github.com/opendevstack/ods-provisioning-app/blob/master/Jenkinsfile#L47
Add section to documentation how to update between releases or update to HEAD
Extreme cornercase - API is used with a technical user - now a user logs with admin rights logs in - sees the project and provision will fail, as the repository cannot be created (as he lacks rights).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.