Comments (15)
Hi @sbose78 ,
I first added the private builder image
and different service account
test cases in our CI/CD first. If we have a common solution for the private resources, we will contribute them back to build v2.
And @qu1queee is helping add private github test case.
Last week, we tried to onboard the build v2 on our dev env. But unluckily....ALL build strategies don't work on our dev environment because of the restrict PSP config.
Such:
-
Buildah needs privileged permission:
https://github.com/redhat-developer/build/blob/master/samples/buildstrategy/buildah/buildstrategy_buildah_cr.yaml#L12 -
Buildpack needs to run as root:
https://github.com/redhat-developer/build/blob/master/samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml#L62
Without them, the buildrun/taskrun will report failures like:
- Operation not permitted
- Privileged containers are not allowed
I am not sure do you have the related PSP in the project
of OpenShift.
But in our multi-tenant environments. We have to fix or workaround them ONE by ONE.
We have to focus on these blocking issues in this week first.... :(
Or do you have any idea about these permission issues?
from build.
private builder image and different service account
These should work, and yes, it would be good to ensure good test coverage for these 👍
from build.
@gabemontero , any pointers/thoughts on the issues above?
from build.
On the buildah example
I'm missing the context on why you need
volumeMounts:
- name: buildah-images
mountPath: /var/lib/containers/storage
Without clarification, at first blush, seems like those could be removed.
On buildpaks, yeah running as root a la https://github.com/redhat-developer/build/blob/master/samples/buildstrategy/buildpacks-v3/buildstrategy_buildpacks-v3_cr.yaml#L62 is a non starter on OpenShift for user data.
To draw a parallel to builds v1, any operation like that is encapsulated in the controller and its construction of the build pod, so as to remove the user from Pod create/edit anc controlling what is done within the escalated pod.
In theory then, build v2 could mimic that pattern, but then you are moving away from your plug in any image build tool seamlessly advantage. But that might be the compromise you have to make for these build tools that require root.
Or you expand the list of folks to ask ;-)
Lastly, if @zhangtbj is allowed to share it with us, the precise specifics of the "restricted PSP config" @zhangtbj mentions might shed some light.
from build.
I think buildah requires this:
volumeMounts:
- name: buildah-images
mountPath: /var/lib/containers/storage
to pass the build
artifacts to the push
step. Without it buildah will report error:
error pushing image "us.icr.io/source-to-image-build/buildah-taxi" to "docker://us.icr.io/source-to-image-build/buildah-taxi": error locating image with name "us.icr.io/source-to-image-build/buildah-taxi" ([us.icr.io/source-to-image-build/buildah-taxi]): image not known
from build.
That's right / totally forgot !! ... Thanks for the test run @zhangtbj
As it turns out I see the same thing in the upstream tekton examples: https://github.com/tektoncd/catalog/blob/master/buildah/buildah.yaml#L39-L43
Also, I incorrectly mixed the two. The volumeMounts
I believe in and of itself does not need privileged (it is a ephemeral volume if I remember correctly). There are other reasons buildah needs privileged.
Fixing it is beyond to scope of build v2 alone. And there is no short path for this.
There are a series of Jira's opened to track the requirement.
- the core runtimes epics to augment buildah itself: start with https://issues.redhat.com/browse/RUN-608 and follow the chain of effort to date
- Adam, Ben, and myself had a call with the buildah folks last week on it ... they are trying to ramp up, but bottom line, the ball is in their court
- build v1 has https://issues.redhat.com/browse/DEVEXP-332 ... as it notes, ultimately, this could be tied to re-establishing a node level cache
- Siamak has a couple of openshift pipeline jiras including https://issues.redhat.com/browse/SRVKP-95
from build.
Hi @gabemontero ,
Thanks for the info and it is great that you and buildah team already plan on that.
Today I tried ALL 4 buildstrategies.
The buildpacks and kaniko can run normally without privileged permission.
The S2I requires to use buildah
and buildah
requires privileged.
So S2I and buildah cannot build normally in our multi-tenant env.
I opened an issue to ask for help:
https://github.ibm.com/coligo/source-to-image/issues/196
And I saw there are some people also have this problem from Google search.
I cannot access your https://issues.redhat.com. Do you have any issue/doc or plan to track and do you know when they plan to fix this issue?
Thanks!
from build.
Hi @gabemontero ,
Thanks for the info and it is great that you and buildah team already plan on that.
Today I tried ALL 4 buildstrategies.
The buildpacks and kaniko can run normally without privileged permission.
The S2I requires to use
buildah
andbuildah
requires privileged.So S2I and buildah cannot build normally in our multi-tenant env.
I opened an issue to ask for help:
https://github.ibm.com/coligo/source-to-image/issues/196And I saw there are some people also have this problem from Google search.
I cannot access your https://issues.redhat.com. Do you have any issue/doc or plan to track and do you know when they plan to fix this issue?
Don't have dates from the buildah team for the current set of dependencies.
I'm going to direct you @zhangtbj to @siamaksade , our product manager for build v1, build v2, and tekton / openshift pipelines ... so he product manages for all the players here but buildah. And he can coordinate with his peer from the buildah team.
Thanks!
from build.
And as I surface this to the other players here on my end, got a good tip on how to do buildah non-privileged for build v1 (though it results in slower performance).
See https://docs.openshift.com/container-platform/4.3/builds/custom-builds-buildah.html
To map this to build v2 @sbose78 and @zhangtbj you'll need to reverse engineer:
- create the build v1 example there
- look at the resulting build pod's init containers and containers
- map those to tekton tasks / taskrun steps
- etc. etc.
The Dockerfile machinations in that example are the secret sauce here.
from build.
It is also possible perhaps to map the Dockerfile machinations in that example to tekton tasks/steps
from build.
Thanks for the info! We need to track this issue.
emm..... I am not sure if it is worth using the custom buildah in build v2 before buildah team support the official unprivileged mode.
Or how long we can reverse or support it. Do you have any idea about it? @sbose78 .
And can kaniko
replace buildah
at the first stage for Dockerfile build?
from build.
If I understood you correctly, you are good with
- Kaniko
- Buildpacks
...and
- buildah doesn't work
- s2i doesn't work because of buildah
And can kaniko replace buildah at the first stage for Dockerfile build?
Conceptually, yeah. I don't see why it wouldn't work :-) .. it's about building from a Dockerfile
anyway right? Do you want to give it a try? If you'd like to pair program over a call with @otaviof and try out an "s2i with kaniko", drop me an email and I will connect you both :)
from build.
Hi @sbose78 ,
yes, Kaniko
and Buildpacks
works. But the other two don't work.
I can co-work with you to have a try the "s2i with kaniko", but can we open an issue and prioritize it in our feature list?
After this privileged problem, I still need to investigate another performance blocking issue. Because the build which is executed under tenant namespace is very slower than executed by cluster admin.
- Cluster admin just take about 40-60 seconds to build
- Tenant user needs 5 - 10 mins to build.
It is terrible... :(
If kaniko can replace the buildah, I prefer we open an issue to track "s2i with kaniko". And maybe in the middle of this month (April), we can work together to see the proposal of "s2i with kaniko".
At the same time, I also would like to know if it is possible that we can workaround or fix the buildah privileged issue with buildah team....
Any idea? :)
from build.
I can co-work with you to have a try the "s2i with kaniko", but can we open an issue and prioritize it in our feature list?
Absolutely!
from build.
@zhangtbj I believe this issue can be closed, please close if possible.
from build.
Related Issues (20)
- [FEATURE] Move Readme Try It Section to BETA HOT 5
- [FEATURE] v0.13 Bump Tekton and Kubernetes dependencies
- [FEATURE] Document usage of OCIArtifact source type HOT 2
- [BUG] Endless reconcile of build when strategy kind is unknown
- [SHIP-0038] Use Release Branches for Releasing HOT 4
- [tech-debt] Release Workflow Action No Longer Maintained
- [BUG] BuildStrategy: Cannot Use Context Dir as Working Directory HOT 4
- [FEATURE] Conditional step HOT 1
- [FEATURE] Build Strategy from Tekton Task/Pipeline HOT 1
- [BUG] Automatic release README update needs to be fixed HOT 4
- [FEATURE] Provide Storage Version Migration from v1alpha1 to v1beta1 HOT 1
- [BUG] git clone issue on newest Git version HOT 3
- [FEATURE] Windows Containers/Node Support HOT 2
- [FEATURE] Support another technology to define and orchestrate the image creation inside a pod HOT 2
- [BUG] Unit tests run fine with `go test`, but fail when used with Ginkgo CLI HOT 1
- SHIP-0039: Allow node selector on `Build` and `BuildRun` to be set HOT 7
- SHIP-0039: Allow tolerations on `Build` and `BuildRun` to be set
- SHIP-0039: Allow custom scheduler to be used for `Build` and `BuildRun`
- [FEATURE] Establish a release cadence based on Tekton LTS releases HOT 2
- [BUG] The container name for shipwright build webhook doesn't match the environment variable HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from build.