Coder Social home page Coder Social logo

sbt-ecr's Introduction

sbt-ecr

An SBT plugin for managing Docker images within Amazon ECR.

Download

Features

  • Create ECR repositories using ecr:createRepository
  • Login to the remote registry using ecr:login
  • Push local images using ecr:push

Installation

Add the following to your project/plugins.sbt file:

addSbtPlugin("com.mintbeans" % "sbt-ecr" % "0.16.0")

Add ECR settings to your build.sbt. The following snippet assumes a Docker image build using sbt-native-packager:

import com.amazonaws.regions.{Region, Regions}

enablePlugins(EcrPlugin)

region           in Ecr := Region.getRegion(Regions.US_EAST_1)
repositoryName   in Ecr := (packageName in Docker).value
localDockerImage in Ecr := (packageName in Docker).value + ":" + (version in Docker).value

// Create the repository before authentication takes place (optional)
login in Ecr := ((login in Ecr) dependsOn (createRepository in Ecr)).value

// Authenticate and publish a local Docker image before pushing to ECR
push in Ecr := ((push in Ecr) dependsOn (publishLocal in Docker, login in Ecr)).value

Keep in mind that ecr:createRepository is a completely optional step. If you have a managed infrastructure (e.g. create everything, including the repository, using AWS CloudFormation, Terraform or some other tool), then it might be better to skip this step, and assume that the repository exist, when you trigger the process.

That being said, it's a convenient feature, when you don't rely on any tool like this. We support several policy-related settings, that will allow you to fine-tune the repository, if needed (read ahead).

Usage

The plugin follows common security conventions. That is, you can use the following authentication methods:

To make it work locally, you may configure an AWS profile according to the reference page, and spawn the push process as such:

AWS_DEFAULT_PROFILE=<your_profile_name> sbt ecr:push

Tagging

By default, the produced image will be tagged as "latest". It is possible to provide arbitrary additional tags, for example to add the version tag to the image:

repositoryTags in Ecr ++= Seq(version.value)

If you don't want latest tag on your image you could override the repositoryTags value completely:

repositoryTags in Ecr := Seq(version.value)

If you want to make the tag environment-dependent you can use the following template:

repositoryTags in Ecr := sys.env.get("VERSION_TAG").map(Seq(_)).getOrElse(Seq("latest"))

And trigger the process using:

VERSION_TAG=myfeature sbt ecr:push

Tag immutability

By default, when the createRepository task is executed, the new repository will have Tag immutability disabled. You can control this behavior using the following setting:

imageTagsMutable in Ecr := false

Cross account publishing

By default, when the login task is executed, authentication will target the registry id and repository domain of the AWS account belonging to the role used.

If you need cross account authentication, you can override registry domain and target any registry id.

Example usage:

repositoryDomain in Ecr := Some("myecr.example.com")
registryIds in Ecr ++= Seq("your AWS account id")

Repository security policy configuration

By default, when the createRepository task is executed, the new repository does not have a security policy attached.

When you set repositoryPolicyText in your build.sbt file, and the createRepository is called, the created repository will have the configured policy.

Example usage:

repositoryPolicyText in Ecr := Some(IO.read(file("project") / "ecrpolicy.json")) 

Then in the project/ecrpolicy.json you can set your policy text. For example:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "BuildServerAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::YOUR_ACCOUNT_ID_HERE:role/YOUR_IAM_ROLE_NAME_HERE"
        ]
      },
      "Action": [
        "ecr:*"
      ]
    }
  ]
}

Configuring repositoryPolicyText will not affect existing repositories.

Repository lifecycle policy configuration

Configuring the repository lifecycle policy works the same as configuring the policy in the previous chapter.

By default, when the createRepository task is executed, the new repository does not have a lifecycle policy attached.

When you set repositoryLifecyclePolicyText in your build.sbt file, and the createRepository is called, the created repository will have the configured lifecycle policy.

Example usage:

repositoryLifecyclePolicyText in Ecr := Some(IO.read(file("project") / "ecrlifecyclepolicy.json")) 

Then in the project/ecrlifecyclepolicy.json you can set your policy text. For example:

{
  "rules": [
    {
      "rulePriority": 10,
      "description": "Lifecycle of release branch images",
      "selection": {
        "tagStatus": "tagged",
        "tagPrefixList": [
          "release"
        ],
        "countType": "imageCountMoreThan",
        "countNumber": 20
      },
      "action": {
        "type": "expire"
      }
    }
  ]
}

Configuring repositoryLifecyclePolicyText will not affect existing repositories.

sbt-ecr's People

Contributors

aberman avatar battermann avatar frosforever avatar machuz avatar mikegirkin avatar neowulf avatar ryancerf avatar sjednac avatar sullis avatar timsoethout avatar xuwei-k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

sbt-ecr's Issues

Cannot push image

I'm using version 0.3.0 of the sbt-ecr plugin, but I'm not able to push the image. Two other commands ecr:login and ecr:createRepository work OK, but push apparently does nothing.

I am using Docker 1.13 on a Mac, and my suspicion that tagging https://github.com/sbilinski/sbt-ecr/blob/master/src/main/scala/sbtecr/EcrPlugin.scala#L57) step does not work. I am not a Scala person, but if I understand this code correctly, it should add tag to the the image with the registry name prepended.

When I run docker images after ecr:push image is not tagged:

$ docker images
REPOSITORY                                          TAG                 IMAGE ID            CREATED             SIZE
myimage                                             latest              75e222321012        4 minutes ago       257 MB
<aws_account>.dkr.ecr.us-east-1.amazonaws.com/java  latest              d5c4c8395ef9        3 months ago        174 MB

This is my configuration:

name := "myimage"
version := "latest"

packageName in Docker := "myimage"
version     in Docker := version.value

dockerBaseImage := "<aws_account>.dkr.ecr.us-east-1.amazonaws.com/java:latest"
dockerExposedPorts := Seq(9000)

import com.amazonaws.regions.{Region, Regions}

region           in ecr := Region.getRegion(Regions.US_EAST_1)
repositoryName   in ecr := (packageName in Docker).value
localDockerImage in ecr := (packageName in Docker).value + ":" + (version in Docker).value

addCommandAlias("publish", ";clean;compile;docker:publishLocal;ecr:createRepository;ecr:login;ecr:push")

And docker version:

$ docker version                                                                                                                                                                                    U feature/build_with_jenkins:exco-corp-web/. 12:47:20
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Wed Jan 18 16:20:26 2017
 OS/Arch:      darwin/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Wed Jan 18 16:20:26 2017
 OS/Arch:      linux/amd64
 Experimental: true

Support repository namespaces

Currently there is only repository domain setting, which is automatically calculated, but as ECR supports namespaces, it would be very handy to be able to specify namespace as well.

Support ECS / CodeBuild provided credentials

This can be fixed in a few different ways:

  1. Allow the provider chain to be specified as an sbt setting. Onus on is on the user.
  2. Update current provider chain to use EC2ContainerCredentialsProviderWrapper which wraps the current InstanceProfileCredentialsProvider and supports container credentials. This is what's done in the aws DefaultAWSCredentialsProviderChain
  3. Use the standard aws DefaultAWSCredentialsProviderChain. It seems the only thing that's different in this project is the support for AWS_DEFAULT_PROFILE. This is not something that's supported in any AWS sdk. This was used for a time by the CLI but has since been deprecated in favor of AWS_PROFILE https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html which the DefaultAWSCredentialsProviderChain already handles.

After updating from 0.13.0 to 0.14.0 sbt fails with Reference to undefined setting

After I bump the version from 0.13.0 to 0.14.0 the plugin fails at sbt load. The error is

[error] Reference to undefined setting:
[error]
[error]   app / Ecr / repositoryLifecyclePolicyText from app / Ecr / createRepository ((sbtecr.EcrPlugin.tasks) EcrPlugin.scala:43)
[error]
[error] 	at sbt.internal.util.Init.Uninitialized(Settings.scala:360)
[error] 	at sbt.internal.util.Init.Uninitialized$(Settings.scala:347)
[error] 	at sbt.Def$.Uninitialized(Def.scala:20)
[error] 	at sbt.internal.util.Init.delegate(Settings.scala:260)
[error] 	at sbt.internal.util.Init.delegate$(Settings.scala:236)
[error] 	at sbt.Def$.delegate(Def.scala:20)
[error] 	at sbt.internal.util.Init.compiled(Settings.scala:194)
[error] 	at sbt.internal.util.Init.compiled$(Settings.scala:182)
[error] 	at sbt.Def$.compiled(Def.scala:20)
[...]

denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

Hi,

I am getting the following error:

denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.

Before the push, I can see:

[info] AWS account id: 07579032XXXX
The push refers to repository [07579032XXXX.dkr.ecr.ap-south-1.amazonaws.com/XXXX-service]

Which seems to suggest that the login was succesful.

Any ideas what may be the issue? (full log is attached)

Jenkins-error.txt

Artifact no longer found

[error] Error downloading com.mintbeans:sbt-ecr;sbtVersion=1.0;scalaVersion=2.12:0.16.0
[error]   Not found
[error]   Not found
[error]   not found: https://repo1.maven.org/maven2/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom
[error]   not found: /Users/fabienfoerster/.ivy2/local/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml
[error]   not found: https://davidmweber.github.io/flyway-sbt.repo/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml.sha1
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.typesafe.com/typesafe/ivy-releases/com.mintbeans/sbt-ecr/scala_2.12/sbt_1.0/0.16.0/ivys/ivy.xml.sha1
[error]   checksum format error: /Users/fabienfoerster/Library/Caches/Coursier/v1/https/repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com/mintbeans/sbt-ecr_2.12_1.0/0.16.0/sbt-ecr-0.16.0.pom.sha1

Update AWS SDK dependency

Hello,

any change of updating AWS SDK dependency to, for example, val amazonSdkV = "1.12.115"

This would get rid of this annoying console spam caused by AWS SDK (that has been finally fixed in SDK):

Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.ersion 0s
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.
Nov 22, 2021 11:34:06 AM com.amazonaws.auth.profile.internal.BasicProfileConfigLoader loadProfiles
WARNING: Your profile name includes a 'profile ' prefix. This is considered part of the profile name in the Java SDK, so you will need to include this prefix in your profile name when you reference this profile from your Java code.

Incompatible with latest AWS SDK

I'm using the latest aws-java-sdk (1.11.289) and it appears sbt-ecr is only compatible with sdks around version 1.11.33.

Following the examples in the README I hit a couple errors.

Here's the subset of my build.sbt pertaining to sbt-ecr:


region           in Ecr := Region.getRegion(Regions.US_EAST_1)
repositoryName   in Ecr := "homebay"
localDockerImage in Ecr := (packageName in Docker).value + ":" + (version in Docker).value

// Create the repository before authentication takes place (optional)
login in Ecr := ((login in Ecr) dependsOn (createRepository in Ecr)).value

// Authenticate and publish a local Docker image before pushing to ECR
push in Ecr := ((push in Ecr) dependsOn (publishLocal in Docker, login in Ecr)).value

The first is the createRepository task fails to instantiate com.amazonaws.services.ecr.AmazonECRClient:

[homebay] $ reload
[info] Loading global plugins from /home/jsimpson/.sbt/0.13/plugins
[info] Loading project definition from /home/jsimpson/workspace/homebay/project
[info] Set current project to homebay (in build file:/home/jsimpson/workspace/homebay/)
[homebay] $ feeds/ecr:push
[info] Wrote /home/jsimpson/workspace/homebay/feeds/target/scala-2.12/feeds_2.12-1.0.0.pom
[info] Wrote /home/jsimpson/workspace/homebay/target/scala-2.12/homebay_2.12-c9f759ef98a419ade612c71f1663c5eb48b33a9f-SNAPSHOT.pom
[info] Sending build context to Docker daemon  223.4MB
[info] Step 1/7 : FROM openjdk:latest
[info]  ---> db77212ffe05
[info] Step 2/7 : LABEL MAINTAINER "Jeff Simpson <[email protected]>"
[info]  ---> Using cache
[info]  ---> 99b1f1d6ca3a
[info] Step 3/7 : WORKDIR /.
[info]  ---> Using cache
[info]  ---> 46528284043e
[info] Step 4/7 : ADD --chown=daemon:daemon . /.
[info]  ---> Using cache
[info]  ---> f3e17eb90f59
[info] Step 5/7 : USER daemon
[info]  ---> Using cache
[info]  ---> b32c0ead8e64
[info] Step 6/7 : ENTRYPOINT bin/feeds
[info]  ---> Using cache
[info]  ---> b7e7906a8bbc
[info] Step 7/7 : CMD
[info]  ---> Using cache
[info]  ---> 5a109b8f36ca
[info] Successfully built 5a109b8f36ca
[info] Successfully tagged homebay-feeds:1.0.0
[info] Built image homebay-feeds:1.0.0
[trace] Stack trace suppressed: run last feeds/ecr:createRepository for the full output.
[error] (feeds/ecr:createRepository) Couldn't instantiate instance of class com.amazonaws.services.ecr.AmazonECRClient
[error] Total time: 3 s, completed Mar 6, 2018 9:31:57 PM

I also tried changing the login task to not depend on createRepository and then get a different error more telling

[homebay] $ feeds/ecr:push
[info] Wrote /home/jsimpson/workspace/homebay/feeds/target/scala-2.12/feeds_2.12-1.0.0.pom
[info] Wrote /home/jsimpson/workspace/homebay/target/scala-2.12/homebay_2.12-c9f759ef98a419ade612c71f1663c5eb48b33a9f-SNAPSHOT.pom
[info] Sending build context to Docker daemon  223.4MB
[info] Step 1/7 : FROM openjdk:latest
[info]  ---> db77212ffe05
[info] Step 2/7 : LABEL MAINTAINER "Jeff Simpson <[email protected]>"
[info]  ---> Using cache
[info]  ---> 99b1f1d6ca3a
[info] Step 3/7 : WORKDIR /.
[info]  ---> Using cache
[info]  ---> 46528284043e
[info] Step 4/7 : ADD --chown=daemon:daemon . /.
[info]  ---> Using cache
[info]  ---> f3e17eb90f59
[info] Step 5/7 : USER daemon
[info]  ---> Using cache
[info]  ---> b32c0ead8e64
[info] Step 6/7 : ENTRYPOINT bin/feeds
[info]  ---> Using cache
[info]  ---> b7e7906a8bbc
[info] Step 7/7 : CMD
[info]  ---> Using cache
[info]  ---> 5a109b8f36ca
[info] Successfully built 5a109b8f36ca
[info] Successfully tagged homebay-feeds:1.0.0
[info] Built image homebay-feeds:1.0.0
[trace] Stack trace suppressed: run last feeds/ecr:login for the full output.
[error] (feeds/ecr:login) java.lang.NoSuchMethodError: com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient.getCallerIdentity(Lcom/amazonaws/services/securitytoken/model/GetCallerIdentityRequest;)Lcom/amazonaws/services/securitytoken/model/GetCallerIdentityResult;
[error] Total time: 3 s, completed Mar 6, 2018 9:33:31 PM

Any chance we could get a new release updated against the latest API?

Support submodules

The plugin does support submodules.

The following config

lazy val core = (project in file("core")).
  settings(
    scalaVersion := "2.11.8",
    version := "1.0",
    name := "sbt-ecr-test",
    packageName in Docker := "helloworld-ecr",
    version     in Docker := version.value,
    region           in ecr := Region.getRegion(Regions.US_WEST_2),
    repositoryName   in ecr := (packageName in Docker).value,
    localDockerImage in ecr := (packageName in Docker).value + ":" + (version in Docker).value,
    push in ecr <<= (push in ecr) dependsOn (publishLocal in Docker)
  ).enablePlugins(JavaAppPackaging, DockerPlugin)

Will raise the following error while importing sbt-ecr.

[error] References to undefined settings:
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:localDockerImage ((sbtecr.EcrPlugin) EcrPlugin.scala:29)
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:push ((sbtecr.EcrPlugin) EcrPlugin.scala:48)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:login ((sbtecr.EcrPlugin) EcrPlugin.scala:37)
[error]
[error]   sbt-ecr-test/ecr:repositoryName from sbt-ecr-test/ecr:createRepository ((sbtecr.EcrPlugin) EcrPlugin.scala:33)
[error]
[error]   sbt-ecr-test/ecr:region from sbt-ecr-test/ecr:createRepository ((sbtecr.EcrPlugin) EcrPlugin.scala:33)

I think this can be fixed by by dropping the following line and enabling the plugin via .enableplugins just like you have to for sbt-native-packager

override def trigger = allRequirements

Thanks for the helpful plugin.

ECR version is always set to latest

The ecr image is always tagged as latest. Please consider adding a version Setting to the plugin which will be included when tagging the docker image.

Bintray is being sunset

Bintray will be sunset as of May 1st. Will this plugin be published to Maven Central or somewhere else in the future?

Allow profile to be set via system property as an alternative

Although, it's nice to execute:

AWS_DEFAULT_PROFILE="sandbox" sbt ecr:createRepository

it'd be nice to have an alternative to execute:

sbt -Daws.profile=sandbox ecr:createRepository

This also has a nice property that the build can be configured via the initialize sbt task which prevents for users to specify the profile by default if not found.

initialize ~= { _ =>
  System.setProperty("aws.profile", sys.props.getOrElse("aws.profile", "sandbox"))
}

Pushing to Region CN_North_1 doesn't work

I tried to automate an upload to AWS China (Region CN_North_1), however the domain
def domain(region: Region, accountId: String) = s"${accountId}.dkr.ecr.${region}.amazonaws.com"
doesn't take into account that AWS' domain in China is amazonaws.com.CN instead of only amazonaws.com.
The debug output from the AWS SDK displays the right URL during the build process:

INFORMATION: {ecr, cn-north-1} was not found in region metadata, trying to construct an endpoint using the standard pattern for this region: 'ecr.cn-north-1.amazonaws.com.cn'.
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
Error response from daemon: Get https://************.dkr.ecr.cn-north-1.amazonaws.com/v2/: dial tcp: lookup ************.dkr.ecr.cn-north-1.amazonaws.com on 192.168.65.1:53: no such host

Assume AWS role when executing AWS API

The shared role doesn't get assumed when executing the following:

sbt -DAWS_DEFAULT_PROFILE=shared ecr:createRepository

I have the following in my aws files:

[default]
aws_access_key_id = xxx
aws_secret_access_key = yyy

[shared]
aws_access_key_id = xxx
aws_secret_access_key = yyy
[default]
output = json
region = us-west-2

[profile shared]
role_arn = zzz
source_profile = default
output = json
region = us-west-2

I see this line in AWS.scala:

...
new ProfileCredentialsProvider(sys.env.getOrElse("AWS_DEFAULT_PROFILE", "default")),
...

Any pointers would be be helpful! I see Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.