payback159 / tenama

Tenama provides a simple REST API that allows non-cluster administrators in a shared Kubernetes environment to create temporary namespaces. tenama handles the creation, management, and cleanup of the temporary namespaces.

Hi there, my name is Alex 👋

I had an early interest in Linux and open source software in general, but it was Kubernetes and my entry into the cloud native world that sparked my interest in learning the different ways to deploy a platform. I've been working in Kubernetes cluster administration, development and architecture for ~4 years (12.02.22) and find it as exciting as my first kubectl command on the first cluster I installed myself.

Mainly I use Bash, Golang, Python, Terrafom and Ansible in my daily work.

Since then I'm always on the track to learn new things to be able to provide even better platform services.

---

---

---

Feel free to contact me, I am always happy to exchange ideas with like-minded people.

---

Made with the help of https://github.com/anuraghazra/github-readme-stats

tenama's Issues

Should it be possible to configure shorter namespace lifetimes than the cleanup loop checks?

Describe the bug

If the namespace lifetime is smaller than cleanupInterval in the tenama configuration, the namespace may still exist longer than desired.

To Reproduce
Steps to reproduce the behavior:

1. set cleanupInterval in the tenama configuration for example to 15m
2. create namespace with a lifetime of 5m
3. wait for 5m if the loop isn't random synced with the namespace lifetime the namespace will exist after 5m

Expected behavior
I would expect that the namespace will be cleared after it's configured lifetime.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context

I think at the moment the problem exists because the cleanup logic is based on a timely based loop which is execute in a configureable time (in the example all 15m). Maybe we can implemented an api-watcher which get's notified when the lifetime of a namespace is reached and cleans it on a more event-based way.

Alternative we can implement that the namespace lifetime can't be smaller than the cleanupInterval but I think that would be not the best solution.

Improvement DELETE endpoint

• The endpoints ask for a namespace string (i.e. I also need to know the postfix, which might not be the case).
• So it would be good if e.g. 'Find namespace by name' at least accepts a regex, or the same parameters as 'Create a new namespace' - infix and suffix, that would be cleanest.
• Alternatively 'Deletes a namespace' could have those input parameters and act idempotent, then I wouldn't need the 'Find namespace by name' endpoint.

Tenama throws an internal server error if not all resource limits/requests are configured.

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

1. Configure the tenama resources.requests and resources.limits and omit a value e.g. limits.cpu
2. create a namespace via api call
3. get following internal server error in the logs

[PANIC RECOVER] cannot parse '': quantities must match the regular expression '^([+-]?[0-9.]+)([eEinumkKMGTP]*[-+]?[0-9]*)$' goroutine 70 [running]:
github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1.1()
	/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/recover.go:93 +0x14e
panic({0x13bb5c0, 0xc000480e20})
	/usr/local/go/src/runtime/panic.go:884 +0x212
k8s.io/apimachinery/pkg/api/resource.MustParse({0x0, 0x0})
	/go/pkg/mod/k8s.io/[email protected]/pkg/api/resource/quantity.go:139 +0x1ce
github.com/Payback159/tenama/handlers.(*Container).craftNamespaceQuotaSpecification(0xc0003e7eb0, {0xc00040d4a0, 0x1e})
	/build/handlers/api_namespaces.go:395 +0x70
github.com/Payback159/tenama/handlers.(*Container).CreateNamespace(0xc0003e7eb0, {0x17a9cb8, 0xc000424960})
	/build/handlers/api_namespaces.go:80 +0x467
github.com/labstack/echo/v4/middleware.BasicAuthWithConfig.func1.1({0x17a9cb8, 0xc000424960})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/basic_auth.go:93 +0x42a
github.com/labstack/echo/v4.(*Echo).add.func1({0x17a9cb8, 0xc000424960})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:536 +0x51
github.com/labstack/echo/v4/middleware.RecoverWithConfig.func1.1({0x17a9cb8, 0xc000424960})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/recover.go:119 +0xfe
github.com/labstack/echo/v4/middleware.LoggerWithConfig.func2.1({0x17a9cb8, 0xc000424960})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/middleware/logger.go:119 +0xe2
github.com/labstack/echo/v4.(*Echo).ServeHTTP(0xc0003738c0, {0x179a138?, 0xc0001aa380}, 0xc00019a300)
	/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:646 +0x3d1
net/http.serverHandler.ServeHTTP({0x178e440?}, {0x179a138, 0xc0001aa380}, 0xc00019a300)
	/usr/local/go/src/net/http/server.go:2947 +0x30c
net/http.(*conn).serve(0xc000424a00, {0x179aba0, 0xc000436330})
	/usr/local/go/src/net/http/server.go:1991 +0x607
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3102 +0x4db

goroutine 1 [IO wait, 1 minutes]:
internal/poll.runtime_pollWait(0x7f4f4682f218, 0x72)
	/usr/local/go/src/runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc000434200?, 0x6?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Accept(0xc000434200)
	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x234
net.(*netFD).accept(0xc000434200)
	/usr/local/go/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc0003e8690)
	/usr/local/go/src/net/tcpsock_posix.go:142 +0x28
net.(*TCPListener).AcceptTCP(0xc0003e8690)
	/usr/local/go/src/net/tcpsock.go:275 +0x3d
github.com/labstack/echo/v4.tcpKeepAliveListener.Accept({0x440a40?})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:957 +0x1d
net/http.(*Server).Serve(0xc00042a000, {0x178f9f8, 0xc00031b520})
	/usr/local/go/src/net/http/server.go:3070 +0x385
github.com/labstack/echo/v4.(*Echo).Start(0xc0003738c0, {0x157a2a4, 0x5})
	/go/pkg/mod/github.com/labstack/echo/[email protected]/echo.go:663 +0xce
main.main()
	/build/main.go:193 +0xb49

goroutine 20 [sleep, 1 minutes]:
time.Sleep(0x1a3185c5000)
	/usr/local/go/src/runtime/time.go:195 +0x135
main.cleanupNamespaces(0xc000366a80, {0xc0003795d0, 0x6}, {0xc000379580?, 0x0?})
	/build/main.go:93 +0x6b
created by main.main
	/build/main.go:186 +0x9cd

goroutine 25 [IO wait]:
internal/poll.runtime_pollWait(0x7f4f4682f038, 0x72)
	/usr/local/go/src/runtime/netpoll.go:305 +0x89
internal/poll.(*pollDesc).wait(0xc000198380?, 0xc000436eb1?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000198380, {0xc000436eb1, 0x1, 0x1})
	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc000198380, {0xc000436eb1?, 0x0?, 0x0?})
	/usr/local/go/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc00031a040, {0xc000436eb1?, 0x0?, 0x0?})
	/usr/local/go/src/net/net.go:183 +0x45
net/http.(*connReader).backgroundRead(0xc000436ea0)
	/usr/local/go/src/net/http/server.go:678 +0x3f
created by net/http.(*connReader).startBackgroundRead
	/u

Expected behavior

That Tenama creates a namespace as expected and does not set/ignore the non-existent limits or requests value.

Additional context

Currently it works only if all values are set. So

requests.cpu, memory, storage and limits.cpu and limits.memory

cleanup logic enhancement

all namespaces which are created by tenama do have a label created-bywith the value tenama. At the moment the cleanup logic only checks if the namespace has the configured prefix in the name.

In this story the cleanup logic should be extended to check,

• if the namespace has the label created-by
• if the label created-byhas the value tenama

If both conditions are met, the namespace should be checked for it's duration.

Duration isnt set correctly

Reproducable with

curl -X 'POST' \
'https://tenama/namespace' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"duration": "60",
"infix": "test",
"suffix": "111",
"users": "[email protected]"
}'

Tenama instance exit code 1 if the generated namespace name is greater than 63 characters long.

Tenama instance exit code 1 if the generated namespace name is greater than 63 characters long.

invalid: metadata.name: Invalid value: "tenama-prepare-namespace-dev-preview-test-d49309b6-f7bd-4220-b66b-d4b58b7e98cf": must be no more than 63 characters

What happens

Service throws an exit code 1 and is restarted by kubernetes.

What do I expect

Error should be passed to the client and the service should not terminate but continue running.

reduce build time for faster CI-Feedback

As of today, the build-and-publish pipeline takes ~13min to build and publish everything. For a tag release I see this less critical but currently we use the main tag for the latest developer state and e.g. my test environment always gets the image from the main tag.

So I would like to optimize the pipeline, as most of the time is left in the multi-arch build from Docker.
Maybe there is a way to separate the 2 builds and parallelize them.

Input is desired :)

Introduction of a global resource limit

Is your feature request related to a problem? Please describe.
Tenama already supports limiting the maximum namespace size that can be provisioned via API call. However, cluster resources could be consumed by creating many namespaces.

Since from my point of view the use-cases of Tenama are the access and provisioning of temporary namespaces in a namespace-based shared cluster environment, this behavior could also affect other services on the cluster.

Describe the solution you'd like
Therefore, Tenama should also offer the possibility to configure a global limit over all created namespaces in the future.

I would define the resources the same way on a requests and limits basis and extend namespace creation logic so that tenama checks in advance if the sum of the resource quotas of all namespaces created by tenama that currently exist in the cluster + the new namespace exceed the global quota.

Probably the simplest variant would be to reject the call with a meaningful message and motivate the client to retry at a later time.

Describe alternatives you've considered

Alternatively, we could include a volatile internal queue in Tenama that attempts to create via FIFO the namespaces that would currently exceed the global quota. Of course, this could lead to very long queues if the user sets the lifetime of the namespaces to a "long" time.
Additionally, the client would have to be notified somehow afterwards about the successful creation, otherwise we would end up with long synchronous calls and I don't think that's good.

Additional context
Add any other context or screenshots about the feature request here.

Users wont get access to created namespace

Reproducable with:

curl -X 'POST' \  
'https://tenama/namespace' \  
-H 'accept: application/json' \  
-H 'Content-Type: application/json' \  
-d '{  
"duration": "60",  
"infix": "test",  
"suffix": "111",  
"users": "[email protected]"  
}'