Comments (9)
Ok, I figured it out, I was using kustomize wrong.
For posterity, the way I got it to work is
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- podinfo
- minecraft
and then I created 2 dirs in cluser apps
directory - podinfo and minecraft with their respective files and kustomizations.
@kingdonb sorry for thread-hijacking and thank you for your detailed explanation!
from flux2-kustomize-helm-example.
I stumbled upon this post as I had the same use case and I have pushed the multi-app approach to https://github.com/darioblanco/gitops
It might help somebody who wants to achieve the same.
from flux2-kustomize-helm-example.
I think you really want to maintain them together with the app, follow the Flux components as example:
If apps are being developed to run on Kubernetes, they should be distributed in each release version with manifests to apply. Following the flux controllers example can't go wrong. If you are installing third-party apps, then it's a choice of whether to vendor manifests or refer as remote bases, similar to what Flux v2's installation kustomization above linked does mapping the constellation of Flux controllers together.
If your third party apps are all managed through Helm, you could create a base for helm installations
Could put everything in a folder for itself in apps
, could put more remote app bases in the cluster/apps/base directory next to Podinfo from this example.
Could put those folders under each environment, if you maintain all those apps in staging as well as production. Maybe your cluster repo doesn't integrate staging and production, then you can do something else different.
The answers are going to depend on a lot of things. Namely, how many apps are both yours and in development, and perhaps also whether they are in a constellation together, or just neighbors. More complex examples might not just use Kustomize, but also enable some kind of elaborate (build-time) manifest generation like Jsonnet or kubecfg, as is being proposed in some new use case documentation I've written for fluxcd/flux2#1200.
How badly do you want to keep your configuration from repeating itself, and how many different environments do you need to manage within those parameters? Kustomize itself is some help here, but perhaps not the best tool for this job by itself.
The idea is that maybe some heavy lifting needed to keep your Kubernetes configs expressive should be done in a separate layer, in CI, and accordingly so also at release time.
from flux2-kustomize-helm-example.
I am not sure if my question is the same or just similar, but I need to install multiple copies of the same app in some of the clusters. So some clusters will have only one copy of the app, while some others two or three of them. Can I do that with the approach demonstrated in this repo?
from flux2-kustomize-helm-example.
@kingdonb I have the same question, and I think maybe you misunderstood @lmakarov here - there's an example how to add 'podinfo' to base apps, and then to kustomize it for different clusters. What is the workflow to add another app here?
For example - I want to add minecraft server. Where would we add minecraft
directory and how do we add it's release, kustomizations and such?
from flux2-kustomize-helm-example.
I'm not sure there's enough information here about your repo structure for me to provide a definitive answer.
If you had more than a single app to deploy into a cluster, where would you put the manifests?
There are effectively two places you can put things in the basic Flux cluster after bootstrap according to the getting started guide: inside of the clusters/[name]
directory, or outside of it.
If you are aiming for multiple deploys across different clusters, then you're going to need to add something to each cluster somehow either directly, or indirectly. One pattern is to add a new base directory in the cluster repository /apps/
and add a base app there for each app you wish to deploy. Then, in each cluster directory, add a reference to the app and customize it with things that are unique to that environment using a patch (like for example, ingress hostname, or number of replicas).
Flux is not prescriptive about repository structure, but the Repository Structure guide tries to provide a comprehensive workable guidance. If it seems complex to understand, it's because it builds on topics from earlier guides. In the first successive example after the Getting Started guide:
https://github.com/fluxcd/flux2-kustomize-helm-example
It's assumed that Production and Staging will both be running on a cluster, but flexibility is granted as to which cluster. The scenario is for multiple clusters and so, production
and staging
are instantiated from a base
directory, so they can avoid repeating common sections.
So, in the apps
directory, there are two Kustomize bases staging
and production
, and in the base
directory, you find one app (podinfo
) which is instantiated in both environments with a kustomization.yaml
that refers to the app in the base, and a patch to apply that updates some values in the associated HelmRelease.
If you don't have apps that need to be repeated with minor variations for multiple environments, then a simpler configuration is possible, but that was the question I think was being asked: how to spread one release across multiple environments efficiently. It's covered by this tutorial guide. (Please read the README in that repository for a fuller understanding.)
If you needed to add minecraft
then it would also go in base
, and you would include it in your environments in a similar way to how podinfo is included in production from the base and how the apps are all included at once in a production cluster. If you only need one minecraft
then you can skip the base, and put it directly in production
directory, but even then I'd consider using a base, as you might think you need only one today but in the future, it might be useful to have the configuration separated into "bare minimum to run this app" and "config that is specific to my instance of the app" which is essentially what's provided for by this pattern.
from flux2-kustomize-helm-example.
@kingdonb thank you so much for a detailed answer! We're getting closer! One more question, that's making me go nuts!
f you needed to add minecraft then it would also go in base, and you would include it in your environments in a similar way to how podinfo is included in production from the base and how the apps are all included at once in a production cluster.
I'm struggling with the part - how podinfo is included in production from the base - how do I add another app here?
I tried doing this, and for some reason, flux ignores it completely :/
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: podinfo
resources:
- ../base/podinfo
patchesStrategicMerge:
- podinfo-values.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: minecraft
resources:
- ../base/minecraft
patchesStrategicMerge:
- minecraft-values.yaml
Is there some other way I can specify two apps, each with it's own patch? Only the first section works. If I swap them around, then minecraft works, but podinfo disappears 😿
from flux2-kustomize-helm-example.
This would work for not very complex cases:
apps/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../base/podinfo
- ../base/gitlab-agent
patches:
- path: podinfo-values.yaml
target:
kind: HelmRelease
name: podinfo
namespace: podinfo
- path: gitlab-agent-values.yaml
target:
kind: HelmRelease
name: gitlab-agent
namespace: gitlab-agent
from flux2-kustomize-helm-example.
This has been very useful as I'm learning Flux and adopting it for my homelab. Thank you for sharing all the links.
from flux2-kustomize-helm-example.
Related Issues (20)
- [Question] How do you suggest using the e2e.yaml workflow with more complex environments?
- apps/production/kustomization.yaml: namespace left out by intention? HOT 1
- Noob Question: Function of kustomization.yaml in infrastructure
- Describe clusters/production/infrastructure.yaml HOT 3
- HelmController missing key 'values.yaml' in ConfigMap when using values file HOT 4
- e2e - Verify cluster reconciliation failed - invalid 'ssh' auth option: 'identity' is required HOT 13
- Azure: source.toolkit.fluxcd.io/v1beta2 broke Arc HOT 7
- validate.sh job does not pass boolen postBuild configuration with string HOT 1
- Why I got error context "staging" does not exist? HOT 1
- What is the purpose of clusters/[cluster]/infrastructure.yaml
- How to access/expose the service without the kubectl proxy command? HOT 3
- Can flux kustomize file patch? HOT 1
- Select which pre-release I want to install HOT 6
- Missing property "interval" + Missing property "sourceRef" for apiVersion: helm.toolkit.fluxcd.io/v2beta1 HOT 2
- cannot curl podinfo service - http 503 HOT 1
- Updating a Deployment from a Helm chart HOT 1
- Request for an example that uses one single Flux instance to deploy workloads on remote clusters with a kubeconfig HOT 8
- question: namespace in kustomization.yaml file necessary HOT 3
- Managing apps on multiple clusters & namespaces HOT 1
- Bootstrap fails with 2 health checks. Also Issues with Strimizi Operator HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from flux2-kustomize-helm-example.