Coder Social home page Coder Social logo

qaware / go-offline-maven-plugin Goto Github PK

View Code? Open in Web Editor NEW
160.0 17.0 15.0 59 KB

Maven Plugin used to download all Dependencies and Plugins required in a Maven build, so the build can be run without an internet connection afterwards.

License: Apache License 2.0

Java 100.00%

go-offline-maven-plugin's Introduction

Go Offline Maven Plugin

License Maven Central

Maven Plugin used to download all Dependencies and Plugins required in a Maven build, so the build can be run without an internet connection afterwards.

This is especially relevant for modern CI-Systems like Gitlab and Circle-CI which need a consistent local Maven repository in their cache to build efficiently.

The plugin can also be used to download all source files of all transitive dependencies of a project.

Maven already has an official way to do all this: the maven-dependency-plugin go-offline goal;
Unfortunately, the go-offline goal suffers from several drawbacks:

  • Multi-Module builds are not supported since the plugin tries to download reactor-dependencies from the remote repository
  • Most parameters simply do not work
  • There is no option to download dynamic dependencies

The Go Offline Maven Plugin fixes these drawbacks.

Requirements

  • Java 1.7 or higher
  • Maven 3.1.x or higher

Warning: Maven 3.5.2 has a Bug that causes the Plugin to hang. If you experience hangups during downloading dependencies, please upgrade to Maven 3.5.3 or higher (See MNG-6323 )

Warning: Maven versions 3.2.x and 3.3.x have a bug that can cause the plugin to fail if maven is started in parallel build mode (-T option). Please call the go-offline-maven-plugin without the -T option or upgrade Maven to a newer version. (See MNG-6170)

Goals

The Go Offline Maven Plugin only has one goal: "resolve-dependencies". This goal downloads all external dependencies and plugins needed for your build to your local repository. Dependencies that are built inside the reactor build of your project are excluded. For downloading, the repositories specified in your pom.xml are used.

Usage

Simply add the plugin to the pom.xml of your project. Use the root reactor pom in case of a multi module project. Make sure to configure any dynamic dependency your project has (see below).

<plugin>
    <groupId>de.qaware.maven</groupId>
    <artifactId>go-offline-maven-plugin</artifactId>
    <version>1.2.8</version>
    <configuration>
        <dynamicDependencies>
            <DynamicDependency>
                <groupId>org.apache.maven.surefire</groupId>
                <artifactId>surefire-junit4</artifactId>
                <version>2.20.1</version>
                <repositoryType>PLUGIN</repositoryType>
            </DynamicDependency>
            <DynamicDependency>
                <groupId>com.querydsl</groupId>
                <artifactId>querydsl-apt</artifactId>
                <version>4.2.1</version>
                <classifier>jpa</classifier>
                <repositoryType>MAIN</repositoryType>
            </DynamicDependency>
            <DynamicDependency>
                <groupId>org.flywaydb</groupId>
                <artifactId>flyway-commandline</artifactId>
                <version>4.0.3</version>
                <type>zip</type>
                <repositoryType>MAIN</repositoryType>
            </DynamicDependency>
        </dynamicDependencies>
    </configuration>
</plugin>

To download all dependencies to your local repository, use

mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies

Make sure to activate any profiles etc. so that all relevant modules of your project are included in the Maven run.

Dynamic Dependencies

Unfortunately some Maven Plugins dynamically load additional dependencies when they are run. Since those dependencies are not necessarily specified anywhere in the plugins pom.xml, the Go Offline Maven Plugin cannot know that it has to download those dependencies. Most prominently, the surefire-maven-plugin dynamically loads test-providers based on the tests it finds in the project.

You must tell the Go Offline Maven Plugin of those dynamic depenencies to ensure all dependencies are downloaded. For each dependency, add a DynamicDependency block to the plugin's configuration as seen in the Usage section. Each dynamic dependency block consists of the following parameters:

  • groupId The GroupId of the dynamic dependency to download
  • artifactId The ArtifactId of the dynamic dependency to download
  • version The version of the dynamic dependency to download
  • classifier (optional) The classifier of the dynamic dependency to download
  • type (optional) The type of the dynamic dependency to download
  • repositoryType Either 'MAIN' or 'PLUGIN' to control from which repository the dependency is downloaded

Note that Plugins are not consistent about where they pull their dynamic dependencies from. Some use the Plugin-Repository , some the Main-Repository. If one doesn't work, try the other.

Downloading Sources and Javadoc

The plugin can also download the source files and/or javadoc of the project's transitive dependencies. This behaviour can either be activated via the pom.xml or a command line parameter.

<plugin>
    <groupId>de.qaware.maven</groupId>
    <artifactId>go-offline-maven-plugin</artifactId>
    <version>1.2.8</version>
    <configuration>
       <downloadSources>true</downloadSources>
       <downloadJavadoc>true</downloadJavadoc>
    </configuration>
</plugin>          

or

mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies -DdownloadSources -DdownloadJavadoc

Usage in CI environments

The Go Offline Maven Plugin can be used to build a clean repository for build server environments. The resulting repository includes exactly the dependencies and plugins needed for building the project.

Gitlab

For gitlab, add the following build step to the front of your pipeline:

download-dependencies:
  image: maven:3-jdk-8
  stage: prepare
  script:
    - 'mvn de.qaware.maven:go-offline-maven-plugin:1.2.8:resolve-dependencies -Dmaven.repo.local=.m2/repository'
  cache:
    key: M2_REPO
    paths:
      - .m2/repository

This will fill the cache "M2_REPO" with all needed artifacts, reusing the previous "M2_REPO" cache to avoid downloading all artifacts on each run.

Build steps using the repository may then be configured like this:

.build
  image: maven:3-jdk-8
  stage: build
  script:
    - 'mvn install -Dmaven.repo.local=.m2/repository'
  cache:
    key: M2_REPO
    paths:
      - .m2/repository
    policy: pull

This will pull the previously filled cache into the build image and use it as the local maven repository. Policy: pull ensures that artifacts that are generated as part of the build are not written back to the cache .

License

Apache 2.0 (https://www.apache.org/licenses/LICENSE-2.0.txt)

go-offline-maven-plugin's People

Contributors

davidster avatar karianna avatar seanf avatar the-alchemist avatar theoderich avatar vlow avatar zregvart avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-offline-maven-plugin's Issues

plugins not fetched

Plugins seem to be not fetched at all.

I've tried on two projects: https://github.com/redisson/redisson and https://github.com/eclipse/winery.
After successfully completing the resolve-dependencies, their build in offline mode fails.

For redisson:
[ERROR] Plugin org.apache.maven.plugins:maven-surefire-plugin:2.12.4 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-surefire-plugin:jar:2.12.4 has not been downloaded from it before. -> [Help 1]

For winery:
[ERROR] Plugin org.apache.maven.plugins:maven-jar-plugin:3.0.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.apache.maven.plugins:maven-jar-plugin:jar:3.0.0 has not been downloaded from it before. -> [Help 1]

These plugins are not dynamic dependencies, they are directly mentioned in the build -> plugins sections of the corresponding projects.

Failed Jar dependencies

Hi, I'm having trouble in downloading all dependencies with mvn -B de.qaware.maven:go-offline-maven-plugin:resolve-dependencies. I've added google maven repository, which has only .aar files. So when I try to resolve-dependencies, all pom files downloads successfully, but I get a warning that .jar files could not be found for the google dependencies, even though I can see that .aar files were downloaded successfully. Then on mvn -B install -DskipTeststhe build fails with the following error. How can I fix that?

[ERROR] Failed to execute goal on project com.idenfy.idenfysdk.core: Could not resolve dependencies for project idenfy:com.idenfy.idenfysdk.core:aar:5.1.4: The following artifacts could not be resolved: androidx.appcompat:appcompat:jar:1.2.0, androidx.core:core:jar:1.3.1, androidx.cardview:cardview:jar:1.0.0, androidx.coordinatorlayout:coordinatorlayout:jar:1.1.0, androidx.annotation:annotation-experimental:jar:1.0.0, androidx.fragment:fragment:jar:1.0.0, androidx.lifecycle:lifecycle-runtime:jar:2.0.0, androidx.recyclerview:recyclerview:jar:1.0.0, androidx.transition:transition:jar:1.2.0, androidx.vectordrawable:vectordrawable:jar:1.1.0, androidx.viewpager2:viewpager2:jar:1.0.0, io.reactivex.rxjava2:rxandroid:jar:2.1.1: androidx.appcompat:appcompat:jar:1.2.0 was not found in https://maven.google.com/ during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of google-repo has elapsed or updates are forced -> [Help 1]

I'm attaching my pom.xml file

pom.xml.txt

resolve-dependencies does not download all dependencies in 1.2.1

The following dependency graph fails with version 1.2.1 of go-offline-maven-plugin, but works with 1.1.0.

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] repro-go-offline-maven-plugin
[INFO] repro-child-module
[INFO] repro-main-module
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building repro-go-offline-maven-plugin 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ repro-go-offline-maven-plugin ---
[INFO] com.github.uce:repro-go-offline-maven-plugin:pom:1.0-SNAPSHOT
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building repro-child-module 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ repro-child-module ---
[INFO] com.github.uce:repro-child-module:jar:1.0-SNAPSHOT
[INFO] \- org.xerial:sqlite-jdbc:jar:3.20.1:compile
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building repro-main-module 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ repro-main-module ---
[INFO] com.github.uce:repro-main-module:jar:1.0-SNAPSHOT
[INFO] \- com.github.uce:repro-child-module:jar:1.0-SNAPSHOT:compile
[INFO]    \- org.xerial:sqlite-jdbc:jar:3.25.2:compile
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] repro-go-offline-maven-plugin ...................... SUCCESS [  0.440 s]
[INFO] repro-child-module ................................. SUCCESS [  0.041 s]
[INFO] repro-main-module .................................. SUCCESS [  0.053 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.893 s
[INFO] Finished at: 2019-10-20T16:28:24+02:00
[INFO] Final Memory: 18M/309M
[INFO] ------------------------------------------------------------------------

Note the two different versions org.xerial:sqlite-jdbc:jar:3.20.1:compile and org.xerial:sqlite-jdbc:jar:3.25.2:compile. In 1.1.0, both dependencies are downloaded whereas in 1.2.1 only the first one is downloaded.


I've attached a sample Maven project in repro-go-offline-maven-plugin.zip that reproduces the issue in Docker. There are two Dockerfiles that only differ in the used go-offline-maven-plugin version.

# Fails with [ERROR] Failed to execute goal on project repro-main-module: Could not resolve dependencies for project com.github.uce:repro-main-module:jar:1.0-SNAPSHOT: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.xerial:sqlite-jdbc:jar:3.25.2 has not been downloaded from it before. -> [Help 1]
docker build -f Dockerfile-go-offline-maven-plugin-1.2.1 .
# Succeeds
docker build -f Dockerfile-go-offline-maven-plugin-1.1.0 .

There is some trickiness in reproducing this issue. The root pom imports a BoM that sets a property which repro-child-module overwrites. repro-main-module depends on repro-child-module and the resolve-dependencies goal only downloads the version specified in repro-child-module instead of the overwritten one from the BOM.


Let me know if you need more input or there is an issue in the example/I'm doing something unexpected here.

Overridden dependency versions not fetched

I ran ./mvnw de.qaware.maven:go-offline-maven-plugin:resolve-dependencies on https://github.com/trinodb/trino and found that if the root POM defines some dependency versions for use by all submodules, while a submodule overrides those versions in it's own POM, only one version (the root one) would get downloaded.

$ mv ~/.m2/repository ~/.m2/repository.old
$ ./mvnw de.qaware.maven:go-offline-maven-plugin:resolve-dependencies
... Wait for download ...
$ ./mvnw --offline clean install  -DskipTests
...
[ERROR] Failed to execute goal on project trino-pinot: Could not resolve dependencies for project io.trino:trino-pinot:trino-plugin:406-SNAPSHOT: The following artifacts could not be resolved: io.confluent:kafka-avro-serializer:jar:5.5.2, io.confluent:kafka-schema-registry-client:jar:5.5.2, io.confluent:common-config:jar:5.5.2, io.confluent:common-utils:jar:5.5.2, io.confluent:kafka-schema-serializer:jar:5.5.2: Cannot access starburstdata.releases (https://maven.starburstdata.net/starburstdata-artifacts/releases) in offline mode and the artifact io.confluent:kafka-avro-serializer:jar:5.5.2 has not been downloaded from it before. -> [Help 1]

I believe this is very similar to #23 so I left a comment there as well.

An option to fail the build if dependency can't be resolved

I think it would be good to have an option to fail the build, that can be defaulted to not fail the build, if one of the dependencies cannot be resolved.

This is useful for PR jobs that introduce a dependency and this plugin can be run first on a CI server to test that dependencies are resolvable.

1.1.0 downloaded Plugins too but 1.2.0 only downloads dependencies

I've just noticed whilst using 1.2.0 that I'm now missing an artefact that does appear in 1.1.0.

As far as I can see the other artefacts I have are declared as dependencies, whereas this one is declared as a plugin

Weirdly enough (for some reason) I do end up with the pom files for that plugin in my m2 repo but not the jar.

Would this be easy to resolve?

Notably, this is after running 1.1.0

xliffgen
xliffgen/ers-xliffgen-maven-plugin
xliffgen/ers-xliffgen-maven-plugin/1.2
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.pom.sha1
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.pom
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.jar.sha1
xliffgen/ers-xliffgen-maven-plugin/1.2/_remote.repositories
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.jar

and this is after running 1.2.0

xliffgen
xliffgen/ers-xliffgen-maven-plugin
xliffgen/ers-xliffgen-maven-plugin/1.2
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.pom.sha1
xliffgen/ers-xliffgen-maven-plugin/1.2/ers-xliffgen-maven-plugin-1.2.pom
xliffgen/ers-xliffgen-maven-plugin/1.2/_remote.repositories

Performance stats in readme?

Life with and without go-offline-maven-plugin would mean that a project would change a developer experience how? If faster builds (after first use), then what does that look like in terms of mins/secs or a percentage improvement? There's developer experience on workstation, but there's also the same timing imorvements in CI of course, and if the % is different then maybe it could be called out.

ConcurrentModificationException when running in parallel mode

I didn't explicitly specify parallel mode but I believe the slave may have had it configured in the M2 settings. Passing -threads 1 resolves the following issue.

It seems there was a ConcurrentModificationException encountered when running this plugin.

The useful stacktraces are as follows:

Exception in thread "ForkJoinPool-1-worker-3" java.util.ConcurrentModificationException
	at java.util.AbstractList.hashCode(AbstractList.java:540)
	at org.eclipse.aether.internal.impl.DataPool$GraphKey.<init>(DataPool.java:383)
	at org.eclipse.aether.internal.impl.DataPool.toKey(DataPool.java:160)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:563)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Exception in thread "ForkJoinPool-1-worker-1" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.next(ArrayList.java:865)
	at java.util.AbstractList.hashCode(AbstractList.java:540)
	at org.eclipse.aether.internal.impl.DataPool$GraphKey.<init>(DataPool.java:383)
	at org.eclipse.aether.internal.impl.DataPool.toKey(DataPool.java:160)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:563)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Exception in thread "ForkJoinPool-1-worker-2" java.lang.ArrayIndexOutOfBoundsException: 1
	at java.util.ArrayList.clear(ArrayList.java:562)
	at org.apache.maven.repository.internal.DefaultModelResolver.resetRepositories(DefaultModelResolver.java:130)
	at org.apache.maven.model.building.DefaultModelBuilder.configureResolver(DefaultModelBuilder.java:596)
	at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:370)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:375)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:231)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:573)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Exception in thread "ForkJoinPool-1-worker-3" java.lang.NullPointerException
	at org.apache.maven.repository.internal.DefaultVersionResolver$Key.<init>(DefaultVersionResolver.java:559)
	at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion(DefaultVersionResolver.java:183)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:298)
	at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:231)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:461)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Exception in thread "ForkJoinPool-1-worker-2" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.next(ArrayList.java:865)
	at java.util.AbstractList.hashCode(AbstractList.java:540)
	at org.eclipse.aether.internal.impl.DataPool$GraphKey.<init>(DataPool.java:383)
	at org.eclipse.aether.internal.impl.DataPool.toKey(DataPool.java:160)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:563)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Exception in thread "ForkJoinPool-1-worker-1" java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.next(ArrayList.java:865)
	at java.util.AbstractList.hashCode(AbstractList.java:540)
	at org.eclipse.aether.internal.impl.DataPool$GraphKey.<init>(DataPool.java:383)
	at org.eclipse.aether.internal.impl.DataPool.toKey(DataPool.java:160)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:563)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:261)
	at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:342)
	at de.qaware.maven.plugin.offline.DependencyDownloader.resolveDependencies(DependencyDownloader.java:169)
	at de.qaware.maven.plugin.offline.ResolveDependenciesMojo$ResolveProjectDependenciesJob.run(ResolveDependenciesMojo.java:98)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Support downloading javadoc

Currently supports download sources, but would also be nice if javadoc could be downloaded.

Main reason, can just use this plugin instead of this plugin + dependancy:resolve

Multi-module build failure

I am seeing failures like:

[ERROR] Error downloading dependencies for project
[ERROR] The following artifacts could not be resolved: org.commonjava.maven.ext:pom-manipulation-cli:jar:minimal:3.9-SNAPSHOT, org.commonjava.maven.ext:pom-manipulation-io:jar:tests:3.9-SNAPSHOT, org.commonjava.maven.ext:pom-manipulation-core:jar:tests:3.9-SNAPSHOT: Could not find artifact org.commonjava.maven.ext:pom-manipulation-cli:jar:minimal:3.9-SNAPSHOT

from running

mvn -Dmaven.repo.local=/tmp/repo de.qaware.maven:go-offline-maven-plugin:1.2.3:resolve-dependencies

with Maven 3.6.2. I am using repository https://github.com/release-engineering/pom-manipulation-ext with SHA db3be6d3668250e9f72be0b45c1f0e67f0fd936e on branch master.

Cannot find 'configuration' in class de.qaware.maven.plugin.offline.DynamicDependency

I'm not sure if I did something wrong or the dependency does not accept configuration tag inside a dynamicDependency, I did like that inside my pom.xml:

<plugin>
				<groupId>de.qaware.maven</groupId>
				<artifactId>go-offline-maven-plugin</artifactId>
				<version>1.2.5</version>
				<configuration>
					<dynamicDependencies>

						<DynamicDependency>
							<groupId>org.apache.maven.plugins</groupId>
							<artifactId>maven-surefire-plugin</artifactId>
							<repositoryType>PLUGIN</repositoryType>
							<version>2.22.0</version>
						</DynamicDependency>
						<DynamicDependency>
								<groupId>org.springframework.boot</groupId>
								<artifactId>spring-boot-maven-plugin</artifactId>
							<configuration>
								<mainClass>com.socgen.marvel.ego.Application</mainClass>
							</configuration>
							    <repositoryType>PLUGIN</repositoryType>
						</DynamicDependency>
					</dynamicDependencies>


				</configuration>
			</plugin>

I need to add mainClass because if I don't mention it I got an error when building a docker image container and runing the app:

 no main manifest attribute, in ego-1.0-SNAPSHOT.jar

Should I duplicate the dependency to take into consideration the mainClass and update the pom like this:

<plugins>
			<plugin>
				<groupId>de.qaware.maven</groupId>
				<artifactId>go-offline-maven-plugin</artifactId>
				<version>1.2.5</version>
				<configuration>
					<dynamicDependencies>

						<DynamicDependency>
							<groupId>org.apache.maven.plugins</groupId>
							<artifactId>maven-surefire-plugin</artifactId>
							<repositoryType>PLUGIN</repositoryType>
							<version>2.22.0</version>
						</DynamicDependency>
						<DynamicDependency>
								<groupId>org.springframework.boot</groupId>
								<artifactId>spring-boot-maven-plugin</artifactId>
							<configuration>
								<mainClass>com.socgen.marvel.ego.Application</mainClass>
							</configuration>
							    <repositoryType>PLUGIN</repositoryType>
						</DynamicDependency>
					</dynamicDependencies>


				</configuration>
			</plugin>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
				<configuration>
					<mainClass>com.socgen.marvel.ego.Application</mainClass>
				</configuration>
			</plugin>
		</plugins>

or does it another solution or configuration to do and I missed it?

What's the difference between this plugin and my script?

Hey man, I love your plugin.
3 years ago I came up with this script to download maven artifacts and their dependencies for offline work.
Your plugin greatly simplified the progress and I updated the gist to use it instead.
However I noticed there is a difference in the downloaded files' size.
My script gets much more files and I wonder what's the difference?
Also would it be possible to have the plugin work as a CLI without needing a POM file?
I'd like to have a way to pass a text file as an argument that lists the artifacts I want to download like my script can.
Many thanks!

Support maven-compiler-plugin's annotationProcessorPaths

The maven-compiler-plugin dynamically downloads dependencies configured via annotationProcessorPaths. Those dependencies are not required to apear in any dependency section and are thus not downloaded by the go-offline-maven-plugin. The plugin should read the annotationProcessorPaths configuration and download those dependencies.

Cannot specifiy type of dynamic dependency

We have themes in ZIP files that are unpacked by the dependency:unpack goal and specified with the artifactItems configuration. Therefore these artifacts will by downloaded dynamically during the build.

However, it is not possible to specify the type of a dynamic dependency:

<dynamicDependency>
	<groupId>com.company</groupId>
	<artifactId>site-themes</artifactId>
	<version>1.0</version>
	<classifier>blue</classifier>
	<type>zip</type>
	<repositoryType>MAIN</repositoryType>
</dynamicDependency>

It complains about the type element and without it tries to download a jar file.

version 1.2 & 1.2.1 no longer download plugin dependencies

I have the maven enforcer plugin configured like the following. Note the extra dependencies included:

<profile>
	<id>enforcer</id>
	<properties>
		<extra-enforcer-rules.version>1.2</extra-enforcer-rules.version>
		<maven-enforcer-plugin.version>3.0.0-M2</maven-enforcer-plugin.version>
		<ossindex-maven-enforcer-rules.version>3.0.4</ossindex-maven-enforcer-rules.version>
		<enforcer.phase>validate</enforcer.phase>
	</properties>
	<build>
		<pluginManagement>
			<plugins>
				<plugin>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-enforcer-plugin</artifactId>
					<version>${maven-enforcer-plugin.version}</version>
					<executions>
						<execution>
							<id>enforcer-rules</id>
							<goals>
								<goal>enforce</goal>
							</goals>
							<phase>${enforcer.phase}</phase>
							<configuration>
								<rules>
									<banVulnerable implementation="org.sonatype.ossindex.maven.enforcer.BanVulnerableDependencies">
									</banVulnerable>
							</configuration>
						</execution>
					</executions>
					<dependencies>
						<dependency>
							<groupId>org.codehaus.mojo</groupId>
							<artifactId>extra-enforcer-rules</artifactId>
							<version>${extra-enforcer-rules.version}</version>
						</dependency>
						<dependency>
							<groupId>org.sonatype.ossindex.maven</groupId>
							<artifactId>ossindex-maven-enforcer-rules</artifactId>
							<version>${ossindex-maven-enforcer-rules.version}</version>
						</dependency>
					</dependencies>
				</plugin>
			</plugins>
		</pluginManagement>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-enforcer-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
</profile>

When I activate the profile, the go-offline plugin version 1.0 downloads the maven-enforcer-plugin along with it's dependencies: extra-enforcer-rules & ossindex-maven-enforcer-rules. With version 2.0 & 2.1, the go-offline plugin no downloads the plugin's dependencies.

Dynamic dependency not fetched

Hey there!

I just tried out this plugin with this configuration

<plugin>
  <groupId>de.qaware.maven</groupId>
  <artifactId>go-offline-maven-plugin</artifactId>
  <version>1.2.8</version>
  <configuration>
    <dynamicDependencies>
      <DynamicDependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${version.maven.surefire}</version>
        <repositoryType>PLUGIN</repositoryType>
      </DynamicDependency>
    </dynamicDependencies>
  </configuration>
</plugin>

Unfortunately the maven-surefire-plugin's dynamic dependencies are not fetched.
Is this a bug or am I using a wrong configuration?

Failed to execute goal de.qaware.maven:go-offline-maven-plugin:1.2.5:resolve-dependencies (default-cli) on project ego: Execution default-cli of goal de.qaware.maven:go-offline-maven-plugin:1.2.5:resolve-dependencies failed. NullPointerException

I create a multi stage build using this dockerfile:

########################################################################
#####                   Download dependencies                      #####
#####                                                              #####
########################################################################
FROM slimhs/maven-jdk-11:slim as dependencies

COPY jssecacerts $JAVA_HOME/lib/security/
COPY settings.xml /usr/share/maven/ref/

COPY pom.xml ./
RUN mvn -B -f pom.xml -s /usr/share/maven/ref/settings.xml  \
de.qaware.maven:go-offline-maven-plugin:resolve-dependencies


########################################################################
#####                    Build the app with maven                  #####
#####                                                              #####
########################################################################

FROM dependencies as build

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

COPY . .

RUN mvn -o -B -s /usr/share/maven/ref/settings.xml package


########################################################################
#####                    run the app                               #####
#####                                                              #####
########################################################################

FROM build
RUN mkdir -p /usr/src/ego-api
WORKDIR /usr/src/ego-api

COPY --from=build /usr/src/app/target/ego-1.0-SNAPSHOT.jar .

ENTRYPOINT ["java"]
CMD ["-jar", "./target/ego-1.0-SNAPSHOT.jar"]

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
		 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
		 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<parent>
		<groupId>com.sc.bsc.sof</groupId>
		<artifactId>sof-spring-boot-pom-parent</artifactId>
		<version>1.0.10</version>
	</parent>

	<artifactId>ego</artifactId>
	<version>1.0-SNAPSHOT</version>

	<properties>
		<java.version>11</java.version>
		<spring-boot.version>2.3.2.RELEASE</spring-boot.version>
		<postgres.version>42.2.14</postgres.version>
		<h2.version>1.4.200</h2.version>
		<swagger.version>2.9.2</swagger.version>
		<marvel-rocket.version>1.2.16</marvel-rocket.version>
		<flyway.version>6.5.3</flyway.version>
	</properties>

	<dependencies>

		<!-- SPRING -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<!-- SECURITY -->
		<dependency>
			<groupId>com.socgen.marvel</groupId>
			<artifactId>marvel-rocket</artifactId>
			<version>${marvel-rocket.version}</version>
		</dependency>

		<!-- TESTS -->
		<dependency>
			<groupId>org.springframework.security</groupId>
			<artifactId>spring-security-test</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>junit</groupId>
					<artifactId>junit</artifactId>
				</exclusion>
				<exclusion>
					<groupId>org.junit.vintage</groupId>
					<artifactId>junit-vintage-engine</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>org.junit.platform</groupId>
			<artifactId>junit-platform-launcher</artifactId>
			<scope>test</scope>
		</dependency>

		<dependency>
			<groupId>io.springfox</groupId>
			<artifactId>springfox-swagger2</artifactId>
			<version>${swagger.version}</version>
		</dependency>

		<dependency>
			<groupId>io.springfox</groupId>
			<artifactId>springfox-swagger-ui</artifactId>
			<version>${swagger.version}</version>
		</dependency>
		<!-- DATABASE -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
		</dependency>
		<dependency>
			<groupId>org.postgresql</groupId>
			<artifactId>postgresql</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.flywaydb</groupId>
			<artifactId>flyway-core</artifactId>
			<version>${flyway.version}</version>
		</dependency>

		<!-- LOMBOK -->
		<dependency>
			<groupId>org.projectlombok</groupId>
			<artifactId>lombok</artifactId>
			<scope>provided</scope>
		</dependency>

		<!-- DEV TOOLS -->
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-configuration-processor</artifactId>
			<optional>true</optional>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-devtools</artifactId>
			<scope>runtime</scope>
			<optional>true</optional>
		</dependency>

	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>de.qaware.maven</groupId>
				<artifactId>go-offline-maven-plugin</artifactId>
				<version>1.2.5</version>
				<configuration>
				<dynamicDependencies>
					<DynamicDependency>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-maven-plugin</artifactId>
					</DynamicDependency>
				<DynamicDependency>
					<groupId>org.apache.maven.plugins</groupId>
					<artifactId>maven-surefire-plugin</artifactId>
					<version>2.22.0</version>
			    </DynamicDependency>
				</dynamicDependencies>
				</configuration>
			</plugin>
		</plugins>
	</build>

</project>

when I build the image I got this error:

[ERROR] Failed to execute goal de.qaware.maven:go-offline-maven-plugin:1.2.5:resolve-dependencies (default-cli) on project ego: Execution default-cli of goal de.qaware.maven:go-offline-maven-plugin:1.2.5:resolve-dependencies failed.
 NullPointerException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
ERROR: Service 'app' failed to build: The command '/bin/sh -c mvn -B -f pom.xml -s /usr/share/maven/ref/settings.xml  de.qaware.maven:go-offline-maven-plugin:resolve-dependencies' returned a non-zero code: 1

How can I fix it?

With multi-module build runs for one stage and skips the rest

I'm no maven expert, but I followed the instructions in the readme and when running mvn de.qaware.maven:go-offline-maven-plugin:resolve-dependencies . The first one finishes with SUCCESS and all the rest are SKIPPED. Not sure why or how to further debug after digging for a while.

Generate a json of required dependencies

Hello!

It is nice to introduce another task that instead of downloading a dependencies generate a list of dependencies.
This is nice-to-have feature that I would like to use by my https://github.com/wix-incubator/bazelizer

Generate a file under target directory (or just print to stdout or by provided path) a list like:

{ "groupId": "<...>", "artifactId": "< .. >", "version": "<...>" }
{ "groupId": "<...>", "artifactId": "< .. >", "version": "<...>" }
{ "groupId": "<...>", "artifactId": "< .. >", "version": "<...>" }
....

Plugin hangs

Hey,

I had a play. Your plugin hangs after a few hundred downloads. I can kill it and start again, and it completes.

To reproduce:

  1. git clone [email protected]:neo4j/neo4j.git
  2. mvn de.qaware.maven:go-offline-maven-plugin:1.1.0:resolve-dependencies -DexcludeGroupIds=org.neo4j -Dmaven.repo.local=lasse-playing --batch-mode

Good effort otherwise! I have a working workaround for https://issues.apache.org/jira/browse/MDEP-568 (which involves compiling) but yours is much more elegant.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.