kiransripada / azure-keyvault-secret-rotation-demo

Azure Key Vault - Rotate Secrets using Function App Event Handler

Usecase - Rotate( Storage Account) Secrets
Current State

• Storage Account (ksripadastorageaccount) is being used by multiple applications/processes.
• Handover secrets manually /email to the applications /processes owners.
• Secrets have no expiry.

Why - MS recommendation is to rotate secrets, every 90 days

How - Rotate secrets using Azure Key Vault .
Other available options HashiCorp vault

Demo -

• Rotate & retrieve secrets using Azure KV.

Future State

• Ability to rotate secrets on a 90 day frequency
• The ability for applications to retrieve secrets instead of handing secrets to owners

Demo Highlights -

1. Create secret in AKV acces_keys for ksripadastorageaccount
2. Rotate Secrets using MS recommended option
3. Access Secrets (Client Applications, Processess) Retrieve secrets Secure secret retrieval
4. Notify & Alert(nice to have )
5. Learnings / Next steps

1.Create a secret in AKV

Azure Key Vault is a cloud service for securely storing and accessing secrets.
A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic key

2.Rotate Secrets

The azure key vault provides the option to set the expiry when we provision/store an entity in the Key Vault.
We can then monitor events related to an upcoming expiry date.

Rotate keys

      Prerequisites
        An Azure App Service plan
        A storage account to manage function app triggers
        An access policy to access secrets in Key Vault
        The Storage Account Key Operator Service role is assigned to the function app so it can access storage account access keys
        A key rotation function with an event trigger and an HTTP trigger (on-demand rotation)
        An Event Grid event subscription for the SecretNearExpiry eve

3.Access Secrets

Enable a system-assigned managed identity for the application.
Register the application with your Azure AD tenant.
Option #2 Applications can access secrets using service principal

Applications can access secrets using service principal

4. Notify & Alert

notify-webhook copy

5. Learnings

Intermittent Errors / Exceptions encountered during POC

1. [Error] Singleton lock renewal failed for blob 'ksripada-storagekey-rotation-fna/host' with error code 409:LeaseIdMismatchWithLeaseOperation.
   QuickFix:- Restart Function app event handler

2. [Error]Azure.KeyVault.Models.KeyVaultErrorException: Operation returned an invalid status code 'Forbidden'at Microsoft.Azure.KeyVault.KeyVaultClient.GetSecretWithHttpMessagesAsy
   QuickFix:- Validate permissions to function app . the Restart function app event handler

References #
https://github.com/Azure-Samples/KeyVault-Rotation-StorageAccountKey-PowerShell.git
https://dev.to/cheahengsoon/configure-and-manage-azure-key-vault-3foj
https://github.com/Azure-Samples/event-grid-node-publish-consume-events