etsangsplk / aad-pod-identity

Assign Azure Active Directory Identities to Kubernetes applications.

AAD Pod Identity

AAD Pod Identity enables Kubernetes applications to access cloud resources securely with Azure Active Directory (AAD).

Using Kubernetes primitives, administrators configure identities and bindings to match pods. Then without any code modifications, your containerized applications can leverage any resource in the cloud that depends on AAD as an identity provider.

---

Contents

• Features
• Components
  • Managed Identity Controller
  • Node Managed Identity
• Demo
  • 1. Deploy aad-pod-identity
  • 2. Create an identity on Azure
  • 3. Deploy AzureIdentity
  • 4. (Optional) Match pods in the namespace
  • 5. Deploy AzureIdentityBinding
  • 6. Deployment and Validation
• Uninstall Notes
• What To Do Next?
• Code of Conduct

Getting Started

It is recommended to get familiar with the AAD Pod Identity ecosystem before diving into the demo. It consists of the Managed Identity Controller (MIC) deployment, the Node Managed Identity (NMI) DaemonSet, and several standard and custom resources.

Components

AAD Pod Identity has two components: the Managed Identity Controller (MIC) and the Node Managed Identity (NMI).

Managed Identity Controller

The Managed Identity Controller (MIC) is a Kubernetes custom resource that watches for changes to pods, identities, and bindings through the Kubernetes API server. When it detects a relevant change, the MIC adds or deletes assigned identities as needed.

Specifically, when a pod is scheduled, the MIC assigns an identity to the underlying VM/VMSS during the creation phase. When the pod is deleted, it removes the assigned identity from the underlying VM/VMSS. The MIC takes similar actions when AzureIdentity or AzureIdentityBinding are created or deleted.

Node Managed Identity

The authorization request to fetch a Service Principal Token from an MSI endpoint is sent to Azure Instance Metadata Service (IMDS) endpoint (169.254.169.254), which is redirected to the NMI pod. The redirection is accomplished by adding rules to redirect POD CIDR traffic with IMDS endpoint on port 80 to the NMI endpoint. The NMI server identifies the pod based on the remote address of the request and then queries Kubernetes (through MIC) for a matching Azure identity. NMI then makes an Azure Active Directory Authentication Library (ADAL) request to get the token for the client ID and returns it as a response. If the request had client ID as part of the query, it is validated against the admin-configured client ID.

Here is an example cURL command that will fetch an access token to access ARM within a pod identified by an AAD-Pod-Identity selector:

curl 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F' -H Metadata:true -s

For different ways to acquire an access token within a pod, please refer to this documentation.

Similarly, a host can make an authorization request to fetch Service Principal Token for a resource directly from the NMI host endpoint (http://127.0.0.1:2579/host/token/). The request must include the pod namespace podns and the pod name podname in the request header and the resource endpoint of the resource requesting the token. The NMI server identifies the pod based on the podns and podname in the request header and then queries k8s (through MIC) for a matching azure identity. Then NMI makes an ADAL request to get a token for the resource in the request, returning the token and the clientid as a response.

Here is an example cURL command:

curl http://127.0.0.1:2579/host/token/?resource=https://vault.azure.net -H "podname: nginx-flex-kv-int" -H "podns: default"

For more information, please refer to the design documentation.

Demo

You will need Azure CLI installed and a Kubernetes cluster running on Azure, either managed by AKS or provisioned with AKS Engine.

If running MSI-enabled AKS cluster, please make sure to follow the MSI specific documentation before getting started.

Set the following Azure-related environment variables:

export SUBSCRIPTION_ID="<SubscriptionId>"
export RESOURCE_GROUP="<ResourceGroup>"
export IDENTITY_NAME="demo"

1. Deploy aad-pod-identity

Deploy aad-pod-identity components to an RBAC-enabled cluster:

kubectl apply -f https://raw.githubusercontent.com/Azure/aad-pod-identity/master/deploy/infra/deployment-rbac.yaml

Deploy aad-pod-identity components to a non-RBAC cluster:

kubectl apply -f https://raw.githubusercontent.com/Azure/aad-pod-identity/master/deploy/infra/deployment.yaml

Important: For AKS clusters with limited egress-traffic, Please install pod-identity in kube-system namespace using the helm charts.

2. Create an identity on Azure

Create an identity on Azure and store the client ID and resource ID of the identity as environment variables:

az identity create -g $RESOURCE_GROUP -n $IDENTITY_NAME
export IDENTITY_CLIENT_ID="$(az identity show -g $RESOURCE_GROUP -n $IDENTITY_NAME --query clientId -otsv)"
export IDENTITY_RESOURCE_ID="$(az identity show -g $RESOURCE_GROUP -n $IDENTITY_NAME --query id -otsv)"

Assign the role "Reader" to the identity so it has read access to the resource group. At the same time, store the identity assignment ID as an environment variable.

export IDENTITY_ASSIGNMENT_ID="$(az role assignment create --role Reader --assignee $IDENTITY_CLIENT_ID --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP --query id -otsv)"

3. Deploy AzureIdentity

Create an AzureIdentity in your cluster that references the identity you created above:

cat <<EOF | kubectl apply -f -
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentity
metadata:
  name: $IDENTITY_NAME
spec:
  type: 0
  resourceID: $IDENTITY_RESOURCE_ID
  clientID: $IDENTITY_CLIENT_ID
EOF

Set type: 0 for user-assigned MSI or type: 1 for Service Principal.

4. (Optional) Match pods in the namespace

For matching pods in the namespace, please refer to namespaced README.

5. Deploy AzureIdentityBinding

Create an AzureIdentityBinding that reference the AzureIdentity you created above:

cat <<EOF | kubectl apply -f -
apiVersion: "aadpodidentity.k8s.io/v1"
kind: AzureIdentityBinding
metadata:
  name: $IDENTITY_NAME-binding
spec:
  azureIdentity: $IDENTITY_NAME
  selector: $IDENTITY_NAME
EOF

6. Deployment and Validation

For a pod to match an identity binding, it needs a label with the key aadpodidbinding whose value is that of the Selector: field in the AzureIdentityBinding. Deploy a pod that validates the functionality:

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: demo
  labels:
    aadpodidbinding: $IDENTITY_NAME
spec:
  containers:
  - name: demo
    image: mcr.microsoft.com/k8s/aad-pod-identity/demo:1.2
    args:
      - --subscriptionid=$SUBSCRIPTION_ID
      - --clientid=$IDENTITY_CLIENT_ID
      - --resourcegroup=$RESOURCE_GROUP
    env:
      - name: MY_POD_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.name
      - name: MY_POD_NAMESPACE
        valueFrom:
          fieldRef:
            fieldPath: metadata.namespace
      - name: MY_POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP
  nodeSelector:
    kubernetes.io/os: linux
EOF

mcr.microsoft.com/k8s/aad-pod-identity/demo is an image that demostrates the use of AAD pod identity. The source code can be found here.

To verify that the pod is indeed using the identity correctly:

kubectl logs demo

If successful, the log output would be similar to the following output:

...
succesfully doARMOperations vm count 1
succesfully acquired a token using the MSI, msiEndpoint(http://169.254.169.254/metadata/identity/oauth2/token)
succesfully acquired a token, userAssignedID MSI, msiEndpoint(http://169.254.169.254/metadata/identity/oauth2/token) clientID(xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)
succesfully made GET on instance metadata
...

Once you are done with the demo, clean up your resources:

kubectl delete pod demo
kubectl delete azureidentity $IDENTITY_NAME
kubectl delete azureidentitybinding $IDENTITY_NAME-binding
az role assignment delete --id $IDENTITY_ASSIGNMENT_ID
az identity delete -g $RESOURCE_GROUP -n $IDENTITY_NAME

Uninstall Notes

The NMI pods modify the nodes' iptables to intercept calls to IMDS endpoint within a node. This allows NMI to insert identities assigned to a pod before executing the request on behalf of the caller.

These iptables entries will be cleaned up when the pod-identity pods are uninstalled. However, if the pods are terminated for unexpected reasons, the iptables entries can be removed with these commands on the node:

# remove the custom chain reference
iptables -t nat -D PREROUTING -j aad-metadata

# flush the custom chain
iptables -t nat -F aad-metadata

# remove the custom chain
iptables -t nat -X aad-metadata

What To Do Next?

• Dive deeper into AAD Pod Identity by following the detailed Tutorial.
• Learn more about the design of AAD Pod Identity:
  • Concept
  • Block Diagram
• Learn how to debug this project at the Debugging wiki page.
• Join us by Contributing to AAD Pod Identity.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information, see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.