bbertka / vmware-tmc-eks-demo

Quick guide to depoly a TMC managed EKS cluster with AWS ALB and starter application

VMware Tanzu Mission Control Demo (EKS w/ALB)

About

This is a guick guide to depoly a TMC managed EKS cluster with AWS ALB and starter application. This guide will walk through the process of setting up the reauired VPC and AWS credentials TMC needs to deploy a three node EKS cluster. AWS Application Load Balancer will be enabled and a common sample app deployed to the Kubernetes cluster. Various verification and troubleshooting steps will be performed throughtou this tutorial as well.

Assumptions

• You have access to AWS, and your organizization has provided you access to Tanzu Mission Control (TMC).

• If you are currently already developing with EKS and Kubernetes, feel free to begin Part 1. Otherwise, visit Setting up your EKS Dev Environment for instructions on how to set up your machine with the proper tools needed to continue this lab.

Part 1 - Tanzu CLI

1. Generate a Tanzu API key and install the Tanzu CLI by logging into your organization’s TMC instance, then navigate to the lower left panel and select "Automation Center". Follow the instructions there to Get an API token, Initialize the CLI and log in, and Try out the CLI.

2. Next, follow the Tanzu quickstart guide here to create a TMC context, and sync your TMC plugins. Follow all the steps through "Verify your updated plug-ins". You have completed this step properly if you can generate the following output with the command:

   $ tanzu plugin list
   Standalone Plugins
     NAME                DESCRIPTION                                                        TARGET      DISCOVERY  VERSION  STATUS
     isolated-cluster    isolated-cluster operations                                                    default    v0.28.0  installed
     login               Login to the platform                                                          default    v0.28.0  installed
     pinniped-auth       Pinniped authentication operations (usually not directly invoked)              default    v0.28.0  installed
     management-cluster  Kubernetes management-cluster operations                           kubernetes  default    v0.28.0  installed
     package             Tanzu package management                                           kubernetes  default    v0.28.0  installed
     secret              Tanzu secret management                                            kubernetes  default    v0.28.0  installed
     telemetry           Configure cluster-wide telemetry settings                          kubernetes  default    v0.28.0  installed
   
   Plugins from Context:  south
     NAME                DESCRIPTION                                                     TARGET           VERSION  STATUS
     account             Account for tmc resources                                       mission-control  v0.0.1   installed
     agentartifacts      helm for tmc resources                                          mission-control  v0.0.1   installed
     apply               Create/Update a resource with a resource file                   mission-control  v0.0.1   installed
     audit               Run an audit request on an org                                  mission-control  v0.0.1   installed
     cluster                                                                             mission-control  v0.0.1   installed
     clustergroup        A group of Kubernetes clusters                                  mission-control  v0.0.1   installed
     continuousdelivery  Continuousdelivery for tmc resources                            mission-control  v0.0.1   installed
     data-protection     Data protection for tmc resources                               mission-control  v0.0.1   installed
     ekscluster                                                                          mission-control  v0.0.1   installed
     events              Events for any meaningful user activity or system state change  mission-control  v0.0.1   installed
     helm                helm for tmc resources                                          mission-control  v0.0.1   installed
     iam                 IAM Policies for tmc resources                                  mission-control  v0.0.1   installed
     inspection          Inspection for tmc resources                                    mission-control  v0.0.1   installed
     integration         Get available integrations and their information from registry  mission-control  v0.0.1   installed
     management-cluster  A TMC registered Management cluster                             mission-control  v0.0.1   installed
     policy              Policy for tmc resources                                        mission-control  v0.0.1   installed
     secret              secret for tmc resources                                        mission-control  v0.0.1   installed
     setting             Setting plugin for tmc resources                                mission-control  v0.0.1   installed
     tanzupackage        Tanzupackage for tmc resources                                  mission-control  v0.0.1   installed
     workspace           A group of Kubernetes namespaces                                mission-control  v0.0.1   installed

3. Follow the instructions here to create a VPC with the proper subnets

4. Finally, now that your IaaS is paved and machine is ready for working with Tanzu, navigate back to the TMC GUI to create your first cluster.

Part 2 - Creating an EKS Cluster from TMC

1. In order to operate on AWS, TMC needs to be configured with a Credential for AWS. Please navigate to the TMC GUI lower left side "Administration" and then click "CREATE CREDENTIAL", choosing "AWS EKS" form the dropdown. Follow the product led instructions to generate your credential via a Cloud Formation Stack. Wait for the stack to finish completing successfully.

2. In TMC navigate to "Cluster Group" and then select "CREATE CLUSTER GROUP", creating a group per your preferences.

3. Now to create a cluster. Navigate to "Clusters" and then select "ADD CLUSTER", choosing "Create EKS Cluster" from the dropdown. You will be selecting your new credential, and VPC during the process. TODO: Show some screenshot defaults

Part 3 - Configuring your Cluster kube-system ConfigMap

This step ensures that TMC is able to be operated on your behalf given it has already initially provisioned your cluster. Follow along the AWS add-user-role guide here, or simply run the following command to edit your config map in real time:

kubectl edit -n kube-system configmap/aws-auth

#Add the following to the configmap, using your own userarn and username who is authenticated at the commmand line already for
the AWS CLI.

  mapUsers: |
    - groups:
      - system:masters
      userarn: arn:aws:iam::5027541610797:user/tanzuadmin
      username: tanzuadmin

Part 4 - Create an IAM policy on AWS

1. Download the default IAM policy from AWS here:

   $ curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.7/docs/install/iam_policy.json

2. Create the IAM policy

   $ aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json
   {
       "Policy": {
           "PolicyName": "AWSLoadBalancerControllerIAMPolicy",
           "PolicyId": "XXXXXXXXXXXXXXXXXXXXXX",
           "Arn": "arn:aws:iam::XXXXXXXXXXXXX:policy/AWSLoadBalancerControllerIAMPolicy",
           "Path": "/",
           "DefaultVersionId": "v1",
           "AttachmentCount": 0,
           "PermissionsBoundaryUsageCount": 0,
           "IsAttachable": true,
           "CreateDate": "2023-06-30T19:20:37+00:00",
           "UpdateDate": "2023-06-30T19:20:37+00:00"
       }
   }

Part 5 - Associate IAM OIDC Provider and create IAM service account

1. Associating IAM OIDC provider

   $ eksctl utils associate-iam-oidc-provider \
       --region us-east-2 \
       --cluster XXXXX \
       --approve

2. Create IAM service account

   $ eksctl create iamserviceaccount \
     --cluster=XXXX \
     --namespace=kube-system \
     --name=aws-load-balancer-controller \
     --role-name AmazonEKSLoadBalancerControllerRole \
     --attach-policy-arn=arn:aws:iam::XXXXXXXXXXXX:policy/AWSLoadBalancerControllerIAMPolicy \
     --approve

Part 6 - HELM

1. Add the eks-charts Helm repo and install the AWS load balancer controller

   $ helm repo add eks https://aws.github.io/eks-charts
   
   $ helm repo update eks
   
   $ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
     -n kube-system \
     --set clusterName=XXXX \
     --set serviceAccount.create=false \
     --set serviceAccount.name=aws-load-balancer-controller

Part 7 - Deploy App

1. Deploy an AWS sample app with ingress

   $ kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.7/docs/examples/2048/2048_full.yaml

2. Check the ingress

   $ kubectl get ingress/ingress-2048 -n game-2048

References

https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html