scalability concern about funktion

[jstrachan]

Thanks @xpepermint its a great question. Scaling (both in terms of infrastructure, teams, versioning, CI / CD) is a large and complex field with many aspects to consider. Its worth remembering though that those virtualization limits are per individual kubernetes cluster; and you can have many kubernetes clusters federated together. Some of those limits are really the etcd / apiserver overhead issues; its not that containers don't scale to crazy numbers; just individual local clusters in a data centre don't yet. e.g. google handles many many more than those numbers by having lots of clusters (and each of their data centre has lots of clusters and they have lots of data centres too).

We've tried to keep funktion as flexible as possible so folks can scratch whatever performance/integration/scaling itch they have. e.g. a Runtime can really do whatever it wants to take a function and make some kind of kubernetes resource to host it. Right now we've gone down the 1 function -> 1 Deployment -> N pods approach as its the simplest thing that could possibly work and folks get independence of functions deployment and scaling.

If folks are writing lots of, say, nodejs functions with the same underlying package.json then we could absolutely co-locate those into the same pods. It just makes things a little more complex as its then harder to do separate scaling rules (e.g. reusing the kubernetes autoscaling of pods based on http traffic to scale up/down containers would be scaling all functions in the pod - rather than those that really need scaling) - if all the functions have no memory overhead thats not a biggie - but could cause memory issues if some rarely-used functions were pre-loading and caching lots of data.

So the simplest thing for now would be keep Deployments modular (at the per function level unless there's a clear grouping of functions together into a deployment) and then just use lots of kubernetes clusters - aligning them with teams - and you can scale to crazy numbers. Its unlikely you'll have 1 million functions with 1 person owning them all; its more typical that you'll have lots of groups/teams working on lots of functions so if the number starts getting large, try splitting the teams onto separate kubernetes clusters?

But careful co-location within deployments/pods/containers would be a nice option to reduce the number of kubernetes resources when it makes sense to do so. It'd be nice if funktion could auto-detect when co-location would make sense ;)

[xpepermint]

Thanks @jstrachan for your detailed reply. Multiple k8s cluster sounds like a reasonable way. It's also true that 1 mio is a lot and I'll have less function at the end. Let's consider it as resolved for now :).