bazaarvoice / s3-upload-maven-plugin

Allows you to upload a file to S3 from maven

s3-upload-maven-plugin

Uploads a file or (recursively) the contents of a directory to S3.

Configuration parameters

Parameter	Description	Required	Default
bucketName	The name of the bucket	yes
source	The source file or folder (was sourceFile before 1.2)	yes
destination	The destination file or destination folder (was destinationFile before 1.2)	yes
recursive	If this is a directory copy, recursively copy all contents (since 1.2)	no	false
profileName	AWS profile	no	if unspecified, uses the ProfileCredentialsProvider Provider, falling back to env variables
accessKey	S3 access key	no	if unspecified, uses the ProfileCredentialsProvider Provider, falling back to env variables
secretKey	S3 secret key	no	if unspecified, uses the ProfileCredentialsProvider Provider, falling back to env variables
sessionToken	Session token for AWS security token service (STS) credentials. Used with accessKey and secretKey	no	if unspecified, uses the ProfileCredentialsProvider Provider, falling back to env variables
doNotUpload	Dry run	no	false
endpoint	Use a different s3 endpoint	no	s3.amazonaws.com

Example: Upload a file

<build>
  ...

  <plugins>
    ...

    <plugin>
      <groupId>com.bazaarvoice.maven.plugins</groupId>
      <artifactId>s3-upload-maven-plugin</artifactId>
      <version>1.4</version>
      <configuration>
        <profileName>my-profile</profileName>
        <bucketName>my-s3-bucket</bucketName>
        <source>dir/filename.txt</source>
        <destination>remote-dir/remote-filename.txt</destination>
      </configuration>
    </plugin>
  </plugins>
</build>

Example: Recursively upload a folder

<build>
  ...

  <plugins>
    ...

    <plugin>
      <groupId>com.bazaarvoice.maven.plugins</groupId>
      <artifactId>s3-upload-maven-plugin</artifactId>
      <version>1.4</version>
      <configuration>
        <bucketName>my-s3-bucket</bucketName>
        <source>dir</source>
        <destination>remote-dir</destination>
        <recursive>true</recursive>
      </configuration>
    </plugin>
  </plugins>
</build>

s3-upload-maven-plugin's Issues

non-required destination

Hello!

Maybe I'm missing something but it doesn't seem possible to upload to the bucket itself? (i.e. without the need of a "folder" in it).
Maven (last time I checked) doesn't support "empty" values in plugin parameters.
Could "destination" be optional?

how to pass proxy server for s3 connection

I don't see a way to pass proxy server host and port as a configuration. I tried to provide proxy settings from maven settings.xml but this plugin seems not taking the details. so, currently I am unable to connect s3 through a proxy

[Question] Conditional Run?

When does this run exactly? I would like it to run conditionally, so wondering if there is a command line parameter or something to make this only run when I want to. I am trying to incorporate this into my Travis builds, so that way it will only deploy if I run a certain command. Thanks in advance for the help!

need semver key for files - how to do that?

I want to be able to upload files to S3 under a key that is dynamically generated using the groupId-artifactId-version of the project. Is that possible? If not, are you still actively accepting pull requests - if I were to make this change?

Thanks!

Need ability to configure a specific named AWS credential profile

I want to be able to use a named credential profile instead of default. I have already submitted pull request #16 I wonder if it can be merged and released to maven central.

[QUESTION] Does "recursive" option remove recursively ?

Hi.
This is a question.
The "recursive" option will upload files in directories recursively.
And I have an interest in that removed file on source directory will be removed on destination bucket ?
(like an unix "rsync" command with "--delete" option)

Doesn't take credentials from ~/.aws/config

It would be nice if this plugin played along with all the other AWS tools and took credentials from ~/.aws/config.

Bad Request

This plugin doesn't seem functional anymore with S3, it just yields Bad Request with all parameters configured the same way I would call the aws sdk. AccessKey, SecretKey, BucketName, Source, Destination & Recursive

[jdk11][jaxb] Plugin not compatible with a Java 11 environment

Problem: The plugin does not work with a Java 11 environment, the JAXB class javax.xml.bind.DatatypeConverter (certainly used by AWS dependencies inside the plugin) is missing:

Caused by: java.lang.ClassNotFoundException: javax.xml.bind.DatatypeConverter

Adding the following JAXB dependencies in my pom.xml doesn't make things better for the plugin:

        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>2.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-core</artifactId>
            <version>2.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-impl</artifactId>
            <version>2.3.0</version>
        </dependency>
        <dependency>
            <groupId>javax.activation</groupId>
            <artifactId>activation</artifactId>
            <version>1.1.1</version>
        </dependency>

Other details

Maven/Java/OS versions:

mvn -v
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T20:33:14+02:00)
Maven home: /opt/maven/apache-maven-3.5.4
Java version: 11, vendor: Oracle Corporation, runtime: /opt/java/openjdk-11.28
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.15.0-54-generic", arch: "amd64", family: "unix"

Command line for uploading:

mvn clean install com.bazaarvoice.maven.plugins:s3-upload-maven-plugin:1.4:s3-upload

Extract of the stacktrace error:

[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47.256 s
[INFO] Finished at: 2019-07-26T18:21:55+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.bazaarvoice.maven.plugins:s3-upload-maven-plugin:1.4:s3-upload (default-cli) on project turf-pari-services: Execution default-cli of goal com.bazaarvoice.maven.plugins:s3-upload-maven-plugin:1.4:s3-upload failed: A required class was missing while executing com.bazaarvoice.maven.plugins:s3-upload-maven-plugin:1.4:s3-upload: javax/xml/bind/DatatypeConverter
[ERROR] -----------------------------------------------------
[ERROR] realm =    plugin>com.bazaarvoice.maven.plugins:s3-upload-maven-plugin:1.4
[ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
[ERROR] urls[0] = file:/home/gekinci/.m2/repository/com/bazaarvoice/maven/plugins/s3-upload-maven-plugin/1.4/s3-upload-maven-plugin-1.4.jar
[ERROR] urls[1] = file:/home/gekinci/.m2/repository/org/codehaus/plexus/plexus-utils/3.0.22/plexus-utils-3.0.22.jar
[ERROR] urls[2] = file:/home/gekinci/.m2/repository/org/apache/commons/commons-lang3/3.4/commons-lang3-3.4.jar
[ERROR] urls[3] = file:/home/gekinci/.m2/repository/javax/enterprise/cdi-api/1.0/cdi-api-1.0.jar
[ERROR] urls[4] = file:/home/gekinci/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.2/org.eclipse.sisu.inject-0.3.2.jar
[ERROR] urls[5] = file:/home/gekinci/.m2/repository/org/codehaus/plexus/plexus-component-annotations/1.5.5/plexus-component-annotations-1.5.5.jar
[ERROR] urls[6] = file:/home/gekinci/.m2/repository/com/amazonaws/aws-java-sdk-core/1.10.69/aws-java-sdk-core-1.10.69.jar
[ERROR] urls[7] = file:/home/gekinci/.m2/repository/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.jar
[ERROR] urls[8] = file:/home/gekinci/.m2/repository/org/apache/httpcomponents/httpclient/4.3.6/httpclient-4.3.6.jar
[ERROR] urls[9] = file:/home/gekinci/.m2/repository/org/apache/httpcomponents/httpcore/4.3.3/httpcore-4.3.3.jar
[ERROR] urls[10] = file:/home/gekinci/.m2/repository/commons-codec/commons-codec/1.6/commons-codec-1.6.jar
[ERROR] urls[11] = file:/home/gekinci/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.5.3/jackson-databind-2.5.3.jar
[ERROR] urls[12] = file:/home/gekinci/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.5.0/jackson-annotations-2.5.0.jar
[ERROR] urls[13] = file:/home/gekinci/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.5.3/jackson-core-2.5.3.jar
[ERROR] urls[14] = file:/home/gekinci/.m2/repository/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.5.3/jackson-dataformat-cbor-2.5.3.jar
[ERROR] urls[15] = file:/home/gekinci/.m2/repository/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar
[ERROR] urls[16] = file:/home/gekinci/.m2/repository/com/amazonaws/aws-java-sdk-s3/1.10.69/aws-java-sdk-s3-1.10.69.jar
[ERROR] urls[17] = file:/home/gekinci/.m2/repository/com/amazonaws/aws-java-sdk-kms/1.10.69/aws-java-sdk-kms-1.10.69.jar
[ERROR] urls[18] = file:/home/gekinci/.m2/repository/com/amazonaws/aws-java-sdk-sts/1.10.69/aws-java-sdk-sts-1.10.69.jar
[ERROR] Number of foreign imports: 1
[ERROR] import: Entry[import  from realm ClassRealm[maven.api, parent: null]]
[ERROR] 
[ERROR] -----------------------------------------------------
[ERROR] : javax.xml.bind.DatatypeConverter

Do you think you can handle it? I was thinking about forking your project in order to resolve the issue for a Java 11 environment. Thanks.

question: How to include in deploy pipeline...

I'm very fresh when it comes to Maven, I don't suppose you could tell me why this throws an error when I run deploy, but works just fine when I run the plugin on it's own...

<plugin>
    <groupId>com.bazaarvoice.maven.plugins</groupId>
    <artifactId>s3-upload-maven-plugin</artifactId>
    <version>1.4</version>

    <executions>
        <execution>
            <id>s3-upload</id>
            <phase>deploy</phase>
            <goals>
                <goal>s3-upload</goal>
            </goals>

            <configuration>
                <bucketName>swordfish-node</bucketName>
                <source>target/swordfish-node.deb</source>
                <destination>swordfish-node.deb</destination>
            </configuration>

        </execution>
    </executions>
</plugin

The error:

Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy (default-deploy) on project swordfish-node: Deployment failed: repository element was not specified in the POM inside distributionManagement element or in -DaltDeploymentRepository=id::layout::url parameter -> [Help 1]

Maven Goal to execute the Plugin

I've been using the following plugins, but s3-upload-maven-plugin is not working for me.

<build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>1.7</source>
                    <target>1.7</target>
                </configuration>
            </plugin>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>scala-compile-first</id>
                        <phase>process-resources</phase>
                        <goals>
                            <goal>add-source</goal>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>scala-test-compile</id>
                        <phase>process-test-resources</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <finalName><JARNAME></finalName>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <mainClass><APP with Package></mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
              <groupId>com.bazaarvoice.maven.plugins</groupId>
              <artifactId>s3-upload-maven-plugin</artifactId>
              <version>1.2</version>
              <configuration>
                <bucketName><OnlyBucketName></bucketName>
                <source><SOURCE name></source>
                <destination><JAR NAME></destination>
              </configuration>
            </plugin>
        </plugins>
    </build>

I've been running

mvn clean install deploy -e

Consider a more permissive license

Please consider a more permissive license like Apache License 2.0 or MIT License.

It would allow the rest of Maven ecosystem to reuse your awesome tool.