Kaniko builds are slow with non-admin user about build

[zhangtbj]

Hi @otaviof and @sbose78 ,

I think I found the root cause on our tenant env.

Because we have limtRanges setting for each tenant namespace like this:

spec:
  limits:
  - default:
      cpu: 100m
      memory: 1Gi
    defaultRequest:
      cpu: 100m
      memory: 1Gi
    max:
      cpu: "8"
      memory: 32Gi
    min:
      cpu: 10m
      memory: 128Mi
    type: Container

So the build container cannot get too much resources to build then slower than before by using cluster admin.

So I have a question, how about the build v1 in your openshift before?

• Do you have any resource limitRanges or what CPU and memory sizes are appropriate?
• Is there any best practice about the related setting?

Thanks!

[zhangtbj]

This issue is fixed by PR: #92

We can close it.

Thanks!

[otaviof]

@zhangtbj hello! would you provide more details about your setup? It sounds rather strange that cluster-admin would provide different performance.

[otaviof]

Good to hear you've figured it out.

So I have a question, how about the build v1 in your openshift before?
* Do you have any resource limitRanges or what CPU and memory sizes are appropriate?

Currently the BuildStrategy (and also ClusterBuildStrategy) are defining core-v1 Container, and we are not setting any resource limits on them.

Furthermore, to tweak the resource limits a user would have to do so in the BuildStrategy level. Should we consider exposing the resource limits from the Build object itself?

* Is there any best practice about the related setting?

I think a good practice is to have resource limits set, and accordingly to the workload you expect on them.

Thanks for bringing this up, that's a good subject to discuss. :-)

[zhangtbj]

For us, maybe the resource limits on BuildStrategy level is not enough :), because we have the limitation on namespace level first and we need to follow that.

I did some deep test today, and I found the limitation on our side is spec.limits.default.cpu, here is my result:

spec.limits.default.cpu = 100m, buildpacks runs 3m30s
spec.limits.default.cpu = 500m, buildpacks runs 57s
spec.limits.default.cpu = 1G,   buildpacks runs 43s
spec.limits.default.cpu = 2G,   buildpacks runs 41s

spec.limits.default.cpu = 100m, kaniko runs 7m
spec.limits.default.cpu = 500m, kaniko runs 108s
spec.limits.default.cpu = 1G,   kaniko runs 68s
spec.limits.default.cpu = 2G,   kaniko runs 70s

spec.limits.default.cpu = 100m, buildah runs 10m
spec.limits.default.cpu = 500m, buildah runs 2m24s
spec.limits.default.cpu = 1G,   buildah runs 83s
spec.limits.default.cpu = 2G,   buildah runs 59s

From the result, I think 500m - 1G spec.limits.default.cpu setting is enough for us.

FYI, I think it should be also useful for you :)