wso2 / k8s-wso2is-operator

Identity Server Kubernetes Controller

WSO2 Identity Server - K8S Operator

The following CRD operator can be used to deploy WSO2 IS on your Kubernates Cluster. If you want to deploy the Identity Server via Helm operator please refer to the given link: https://github.com/wso2/kubernetes-is

Key benefits

• Auto healing
• Ability to make a test clusters
• Ability to provision multiple ISs on same cluster
• Custom Keystore addition
• Ability to mount custom deployment TOML files
• Seameless updates

Prerequisites (Development)

• Access to a Kubernetes v1.11.3+ cluster (v1.16.0+ if using apiextensions.k8s.io/v1 CRDs).
• User logged with admin permission. See how to grant yourself cluster-admin privileges or be logged in as admin
• Homebrew installed
• Git command line installed and configured
• GoLang installed and correctly configured, including system paths

System Architecture

External Database Setup

Please follow the instructions given in the documentation to setup the external MySQL databases

• https://is.docs.wso2.com/en/5.11.0/setup/changing-to-mysql/
• https://is.docs.wso2.com/en/5.11.0/setup/changing-datasource-bpsds/
• https://is.docs.wso2.com/en/5.11.0/setup/changing-datasource-consent-management/

Databases to be created

• WSO2_IDENTITY_DB
• WSO2_SHARED_DB
• WSO2_CONSENT_DB (Optional)
• WSO2_BPS_DB (Optional)

Development Environment Setup

Please follow the following instructions to install Operator-SDK in your development environment.

brew install operator-sdk

Clone the repository by running the following command

git clone https://github.com/wso2/k8s-wso2is-operator.git

Navigate to the project directory

cd k8s-wso2is-operator

Run the following command to install dependancies

make install

Apply the CRDs by running the following command

kubectl apply -f config/crd/bases/iam.wso2.com_wso2is.yaml
kubectl apply -f config/crd/bases/iam.wso2.com_userstores.yaml

Feel free to change any configurations at config/samples/wso2_v1_wso2is.yaml Once you do the config changes apply the config by running

kubectl apply -f config/samples/wso2_v1_wso2is.yaml

Finally run the following command to run the operator in your cluster

make run

Installation

It is possible to deploy a stand alone version of the IS Operator in your cluster as well. You many follow the given steps in order to setup correctly.

Prerequisites

1. Kubernetes Cluster
2. Required databases configured, and it should be accessible by all pods
   • Please read the documentation at: https://is.docs.wso2.com/en/5.11.0/setup/working-with-databases/
   • The following databases are required for the standard IS deployment: Identity database, Shared database
3. A persistence volume has be configured with ReadWriteMany permission
   • For AWS users, you can refer to Elastic File System (EFS) docs and learn about the configurations: https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html
   • Microsoft Azure users can use AzureFile as the persistent storage
   • Google Cloud users may use GCEPersistentDisk
4. Also you will need to have an Ingress ready to route requests from your endpoint to service, your ingress can vary from cloud provider to provider.

Run the given command within your cluster

kubectl apply -f https://raw.githubusercontent.com/wso2/k8s-wso2is-operator/main/artifacts/operator.yaml

For step by step instructions on setting up, please refer to the two articles that we've published

• Part 01: Deploying WSO2 Identity Server 5.11.0 on Kubernetes with all new K8s Operator
• Part 02: Deploying WSO2 Identity Server 5.11.0 on Kubernetes with all new K8s Operator

Finally you may apply your own configurations by refering to the formats given in samples https://github.com/wso2/k8s-wso2is-operator/tree/main/config/samples

k8s-wso2is-operator's Issues

[Bug] Changes to a .yaml config of kind Wso2Is are not reflected on the cluster's ConfigMap when re-apply it

Description:

Suggested Labels:
Bug

Suggested Assignees:
@RukshanJS

Steps to reproduce:

• On a working k8s cluster, apply a .yaml config of Wso2Is kind with the spec.tomlConfig field set.
• Do some change to the spec.tomlConfig field and re-apply it.
• The changes are NOT reflected on the cluster, unlike expected.

Handling Wso2Is yaml-files containing sensitive information in Git

Description:
I'm looking into using the wso2is-operator as described here, but have run in to some administrative issues.

In the environment I am working currently, all the k8s configurations must be committed to a Git repository. The ArgoCD operator will then pull the repo for new yaml configs and apply them to the cluster.

Secret information such as certificates, keys, credentials and similar can be committed to the Git repo safely by using sealed secrets.

The wso2is-operator introduces the new custom resource definition:

apiVersion: iam.wso2.com/v1beta1
kind: Wso2Is
...

Since this is not a secret, it can not be sealed as a sealed secret, and since the Wso2Is object will contain various passwords and other secret data, it can not be added to the Git repository as is.

Is there any suggested solution to this problem?

Suggested Labels:

Git, WSO2, IS, WSO2-IS, secret, security

Affected Product Version:
5.11.0

Invalid id token when try to login on My Account or Console

Description:

When the IS is deployed using the operator, cannot log in into My Account and the new Console (Can only log in into the old carbon console)

Suggested Labels:

bug

Affected Product Version:

Tested with IS 5.11.0 and possibly affect versions 6.x.x. too

OS, DB, other environment details and versions:

• Ubuntu 22.04.2 LTS
• minikube v1.29.0 on Ubuntu 22.04
• Kubernetes v1.26.1 on Docker 20.10.23

Steps to reproduce:

1. Deploy the IS using the operator
2. Go to https://<hostname>/myaccount page and enter credentials of a user to login
3. Receive error in browser console saying Uncaught (in promise) Invalid id_token in the token response:....

[Bug] User Creation Bug: Userstores Not Shared/Synced Across Pods

Description:
I encountered a bug where user creation in one pod instance does not make the user available in another pod instance. As a result, the userstores are not shared or synced across pods.

Suggested Labels:
bug, user management, synchronization

Suggested Assignees:
@RukshanJS

Affected Product Version:
6.1.0

OS, DB, other environment details and versions:
Operating System: Ubuntu 22.04.2 LTS
minikube version: 1.29.0
Docker version: 20.10.23

Steps to Reproduce:

• Set up a Kubernetes cluster using minikube with the specified configurations.
• Deploy multiple instances of the pod that hosts the user creation functionality.
• Create a new user in one of the pod instances.
• Attempt to access the newly created user from another pod instance.

Expected Behavior:
The user created in one pod instance should be accessible and synchronized across all pod instances, ensuring that the userstores are shared.

Actual Behavior:
The user created in one pod instance is not available or synchronized with other pod instances. The userstores are not shared across pods.

Ldap error when running the operator with IS 6.x.x

Description:
Since the newer 6.x versions of the wso2is removed the ldap userstore connection by default, if you try to run the operator versions of the operator with newer IS 6.x versions, errors similar to the following will occur.

[2023-05-19 14:08:10,228] [] ERROR {org.wso2.carbon.user.core.ldap.LDAPConnectionContext} - Error obtaining connection. Cannot parse url: ldap://localhost:-1 javax.naming.NamingException: Cannot parse url: ldap://localhost:-1 [Root exception is java.net.MalformedURLException: unsupported authority: localhost:-1]
	at java.naming/com.sun.jndi.ldap.LdapURL.<init>(LdapURL.java:115)
	at java.naming/com.sun.jndi.ldap.DefaultLdapDnsProvider.lookupEndpoints(DefaultLdapDnsProvider.java:47)

This issue is already fixed with #19 but added here for tracking purposes

make install fails with missing boilerplate

Description:

When running make install it fails with this output:

[user@fedora k8s-wso2is-operator]$ make install
which: no controller-gen in (/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/user/.local/bin:/home/user/bin)
Makefile:125: warning: overriding recipe for target 'test'
Makefile:30: warning: ignoring old recipe for target 'test'
go: creating new go.mod: module tmp
go: added github.com/fatih/color v1.7.0
go: added github.com/gobuffalo/flect v0.2.0
go: added github.com/gogo/protobuf v1.3.1
go: added github.com/google/gofuzz v1.1.0
go: added github.com/inconshreveable/mousetrap v1.0.0
go: added github.com/json-iterator/go v1.1.8
go: added github.com/mattn/go-colorable v0.1.2
go: added github.com/mattn/go-isatty v0.0.8
go: added github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
go: added github.com/modern-go/reflect2 v1.0.1
go: added github.com/spf13/cobra v0.0.5
go: added github.com/spf13/pflag v1.0.5
go: added golang.org/x/net v0.0.0-20191004110552-13f9640d40b9
go: added golang.org/x/sys v0.0.0-20191022100944-742c48ecaeb7
go: added golang.org/x/text v0.3.2
go: added golang.org/x/tools v0.0.0-20190920225731-5eefd052ad72
go: added gopkg.in/inf.v0 v0.9.1
go: added gopkg.in/yaml.v2 v2.2.8
go: added gopkg.in/yaml.v3 v3.0.0-20190905181640-827449938966
go: added k8s.io/api v0.18.2
go: added k8s.io/apiextensions-apiserver v0.18.2
go: added k8s.io/apimachinery v0.18.2
go: added k8s.io/klog v1.0.0
go: added k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89
go: added sigs.k8s.io/controller-tools v0.3.0
go: added sigs.k8s.io/structured-merge-diff/v3 v3.0.0
go: added sigs.k8s.io/yaml v1.2.0
/home/user/go/bin/controller-gen "crd:trivialVersions=true,crdVersions=v1" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
F1114 11:55:02.461048   19333 packages.go:59] Failed loading boilerplate: open k8s.io/kubernetes/hack/boilerplate/boilerplate.go.txt: no such file or directory
make: *** [Makefile:55: manifests] Error 255

The missing boilerplate is in the folder hacks/ but make just ignores it.

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:
Fedora 38

Steps to reproduce:
git clone on Fedora, cd in folder, make install

Related Issues: