buildkite / on-demand Goto Github PK

Allow the stack to passively discover an IAM SSH Agent backend configured using SSM.

The IAM SSH Agent ARN is needed in several places, CloudFormation parameters, dynamic task definition generation. It might make sense to make this an SSM parameter that is set once when deploying IAM SSH Agent for your infrastructure, and all the components auto discover it.

Putting the stack name in the parameter would ensure multiple deployments in the same region can use or not use iam ssh agent.

Depends on capacity provider support in CloudFormation aws/containers-roadmap#631

This is done in the ScheduleTask lambda today but could be done in the EventBridge Pattern instead.

Currently the agent-scheduler stack either creates a toy vpc, or accepts a comma separated list of vpc subnets.

Should this support taking the name of another stack + stack export name so that it can import details of the VPC instead, to prevent deleting the VPC stack while the agent-scheduler is deployed?

Right now the fetch + decrypt policy assumes you are using the aws/ssm key, this should be moved to the Globals section of the transform as a default value and individual Secret entries should allow override.

To facilitate separate queues having access to different IAM roles in an account, the iam:PassRole grant can be scoped to a path scoped to each agent-scheduler that is deployed. This is needed because IAM is a global resource but agent-scheduler can be deployed more than once.

These paths need to be generated by the agent-scheduler and passed to the existing resources. They will also need to be consumed by agent-composer stacks to place IAM roles in an appropriate spot.

This could be done with an agent-scheduler stack export and by passing the agent-scheduler stack name in to the agent-composer sub stack.

We are trying to deploy this stack and are currently seeing an Access Denied message for the following S3 template URL's.

• https://s3.us-east-1.amazonaws.com/buildkite-serverless-apps-us-east-1/on-demand/template/695ea39a20031481dad67f7caaa1bfaa.template

• https://s3.us-east-1.amazonaws.com/buildkite-serverless-apps-us-east-1/on-demand/template/680ce2f16a1e7f87905f2452398904ea.template

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>P24WHP9ME7CYMW2B</RequestId>
<HostId>
yt7S1rUlHWRn8o5jyWOUXMr3exTyq6lUqu+Ed9naFflvW4YzGSl6DjO6rsc6THTxQRejogFKcrg=
</HostId>
</Error>

For first pipeline steps that clone a repository and upload a pipeline from the repo, an on-demand agent can introduce a lengthy wait. Perhaps this can be specialised by the ScheduleTask lambda to recognise simple cases and delegate them to lambda based 'specialised agents' instead.

Using the macro is very compelling, perhaps it should be deployed by default when deploying agent-scheduler to remove a step?

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-application.html

Resources:
  AgentMacro:
    Type: AWS::Serverless::Application
    Properties:
      Location:
        ApplicationId: 'arn:aws:serverlessrepo:us-east-1:832577133680:applications/buildkite-on-demand-transform'
        SemanticVersion: '0.1.0'

agent-scheduler creates a /aws/events/Buildkite CloudWatch Log group to monitor the event bridge events being received, but this is a globally named resource and prevents deploying the scheduler to a region more than once.

The agent token is also stored globally in SSM as /buildkite/agent-token.

These are hardcoded to keithduncan/buildkite-base and keithduncan/buildkite-sidecar repsectively.

https://aws.amazon.com/blogs/aws/amazon-ecs-supports-efs/

If these parameters were added it would remove the main reason to create a task role outside of the Buildkite::ECS::TaskDefinition macro. The macro can still inject permission to access the given iam-ssh-agent which would remove a stumbling block when moving from the synthesized role to an explicit role.