razorspoint / azure-pipeline-agent-scaler

solution for using Azure Container Apps with KEDA scaler for scalable Azure DevOps agent pools run in containers

Getting Started

Read also my blog post for more details.

Update 22.02.2022
There is a thread about this approach regarding Container Apps only supporting ScaledObjects and not ScaledJobs. Which is needed to not cancel agent jobs when container apps are scaled down. So for now scaling is not really usable for ACI (Azure Container Apps).

See below tweet: https://twitter.com/jorgearteiro/status/1494324269371002882?s=21

It has been removed from the current public documentation as well but there is a feature request to support scaled jobs in the future. So use this as a basic concept and stay tuned until scaled jobs really work with Azure Pipelines.

Prerequisites

For this example here the following tools are needed:

• PowerShell V5.0 or higher
• PowerShell Module VSTeam --> Install-Module VSTeam
• Azure CLI
• an Azure Subscription
• an Azure DevOps Organization

Run the example

1. Make sure you have an ACR ready with an admin user activated.
2. Build the agent docker image (ps script azdoimage/imagebuildAndPush.ps1) and push it to your ACR. The image was taken from the docs.
3. Make sure you have an Azure DevOps Services organization and a PAT for that organization with 'Agent Pools, Read & manage' scope
4. fill variables in container-app/resource.ps1
5. run ps script container-app/resource.ps1. It creates
   1. resource group
   2. Analytics Workspace
   3. Azure Container App environment
   4. the Azure Container App
   5. agent pool in your organization (or reuses it)

More Information

On my blog I wrote a bit more than here especially about the connecting technologies and the heart of the scaler. It assumes you don't need a hand in any of those scripting languages or docker or containers.

But here are some more links to get you started with the used technologies:

• Azure Container Apps
• Azure DevOps
  • Azure Pipelines
• Docker
• KEDA
• Kubernetes

But wait! Isn't this idea not just specific to Azure DevOps

Basically yes! Because Container Apps don't care what you run. So you could take the principle to GitHub, GitLab and Jenkins. The thing you need is a scaler available in KEDA and an API on these System where you could check the queue of waiting jobs. Then you just go to KEDA's github repo and make a PR.