phreed / ivypot-gradle-plugin

Ivypot is a Gradle plugin to help managing a local off-line repository for builds in restricted environments

IvyPot - A Gradle plugin for Off-line Repositories

Not all development teams have the luxury of relying on Maven Central, Bintray for day to day development. Some of them are still fortunate to proxy public repositories via in-house Artifactory or Nexus instances. Others have even more strict requirements including the ability to be able to build an isolated environment. This plugin is for them. It allows for public dependencies, including transitive dependencies, to be downloaded in a controlled manner and the internal security processes to be applied to the binary files before committing them to an internal storage area.

Gradle minimum versions

• As from 0.9 the minimum supported Gradle version is 5.0. I have very little time to maintain this low usage plugin, so trying to make it compatible with 4.0 as well is too much effort. There is no feature change from 0.8, so if you need to use Gradle 4.x, stay with 0.8.

• As from 0.7 the minimum supported Gradle version is 4.0. (Required if you want Google repository support).

• As from 0.5 the minimum supported Gradle version in 2.8.

Previous versions

• Release 0.9

• Release 0.8

• Release 0.7

• Release 0.6

• Release 0.5

• Release 0.4

• Release 0.3.7

• Release 0.3.6

• Release 0.3.5

• Release 0.3.4

• Release 0.3.2

• Release 0.3.1

Adding the plugin

build.gradle

plugins {
  id 'org.ysb33r.ivypot' version '{revnumber}'
}

Or for special cases you might want to do

build.gradle

buildscript {
  repositories {
      maven {
          url "https://plugins.gradle.org/m2/"
      }
  }
  dependencies {
    classpath 'org.ysb33r.gradle:ivypot:{revnumber}'
  }
}

apply plugin : 'org.ysb33r.ivypot'

If you are in a restricted environment, you might want to use the second approach and conditionally add the remote reference to work only when you are in a non-restricted environment. i.e.

  repositories {
    if(!System.getenv('RESTRICTED')) {
      maven {
          url "https://plugins.gradle.org/m2/"
      }
    }
  }

You might also consider using pluginManagement in settings.gradle for this case.

Defining Remote Repositories

The plugin provides a default task called syncRemoteRepositories which is of type org.ysb33r.gradle.ivypot.OfflineRepositorySync. (You are free to create more instances of this task type).

syncRemoteRepositories {
  repoRoot '/path/to/repo' // (1)

  repositories {  // (2)
    jcenter()
    google()
    mavenCentral()
    mavenLocal()
    maven {
      url 'http://foo/bar'
      credentials {
        username 'pig'
        password 'hog'
      }
    }

    ivy {
      url 'http://foo.bar'  // (3)
    }

  }

  configurations 'compile', 'testCompile' // (4)

  includeBuildScriptDependencies = false // (5)
}

1. Defines the location of the off-line repository root. Must be an absolute path, no relative path.

2. repositories in an implementation of RepositoryHandler and therefore supports all the standard repository definitions a Gradle user would expect.

3. Ivy repositories in all incantantions available to Gradle is supported.

4. Restrict the configurations you would like to have added to the synchronisation process. If none are supplied, dependencies from all configurations will be added.

5. Whether to add dependencies from buildscript. (See further down).

Usage

For normal usage one simply has to point to the local repository. It is assumed that this repository would be committed to some form of source control or artifact management which is accessible with the restricted environment. The layout of this repository is a standard Ivy layout.

repositories {
  ivy {
    url 'file:///path/to/repo' // (1)
    layout 'gradle' // (2)
  }
}

1. Define the path to the local repository here

2. The default layout is gradle. If this has been changed using repoArtifactPattern and repoIvyPattern, then then layout should be set to whatever pattern is correct.

Boostrapping plugins in a restricted environment

Just add the repository to repositories closure

buildscript {
    repositories {
      ivy {
        url 'file:///path/to/repo' // (1)
      }
    }
}

1. Define the path to the local repository here

Multi-project support

The best way to cache dependencies off-line for a multi-project is to create a special subproject just for synchronisation.

build.gradle

plugins {
  id 'org.ysb33r.ivypot' version '{revnumber}' apply false
}

allprojects {
    ext {
        offlineRepoDir = "${rootProject.projectDir}/repo"
    }
}

sync/build.gradle

apply plugin : 'org.ysb33r.ivypot'

syncRemoteRepositories {

    addAllProjects() // (1)

    addProject ':a:b'  // (2)

    addProject ':a:b','compile','testCompile'  // (3)
}

1. Adds all configurations from all subprojects and the rootproject with the exception of the current project. buildscript dependencies are not added via this call.

2. Adds all configurations from project :a:b as long as the current project is not called :a:b.

3. Adds only the compile and testCompile configurations from project :a:b. Once again :a:b must not be the current project.

Now you just have to run ./gradlew syncRemoteRepositories or ./gradlew :sync:syncRemoteRepositories from the top.

Note	Also see src/gradleTest/multiProject as an example of how this works. (That’s actually the compability test we use for multi-projects).

Caching arbitrary binaries

As from 0.10 it is now possible to cache arbitrary binary files with a path that typically matches that on the server. This is especially useful for people that need to perform tests where binaries are downloaded many times.

Configuring binary repositories

syncRemoteRepositories {
    binaryRepositories {
        nodejs { // (1)
            rootUri = 'https://nodejs.org/dist/' // (2)
            artifactPattern = 'v[revision]/[module]-v[revision]-[classifier].[ext]' // (3)
        }
    }
}

1. The name of the repository. This will also be treated as the group/organisation names.

2. The root URI of the remote repository. Only file and http(s) schemes are supported.

3. A pattern similar to that if Ivy for resolving the artifact path below the root URI. This is also used to calculate a relative path for storing the binary locally. Note that classifier is supported.

Specifying binaries locally

If you have one project then the easiest is to specify the binaries within the syncRemoteRepositories task using the cachedBinaries.add DSL keyword.

syncRemoteRepositories {
    cacheBinaries.add 'nodejs:node:7.10.0:[email protected]'
}

Specifying binaries in other projects

It might be useful to rather specify binaries to be cached within the subproject where they are required and then let the syncRemoteRepositories task discover them. In order to achieve this apply a plugin which adds an extension to the dependencies block.

plugins {
  id 'org.ysb33r.binarypot.base' version '{revnumber}' // (1)
}

dependencies {
    cachedBinaries.add 'nodejs:node:7.10.0:[email protected]' //(2)
}

1. Adds the cachedBinaries extenion to dependencies block.

2. The same syntax is used, but the organisation/group name must match that of a binary repository as defined in `syncRemoteRepositories.

Adding buildscript dependencies

By default buildscript dependencies will not be added to the synchronisation list. By setting includeBuildScriptDependencies = true in the configuration closure of the task these will be added.

Patterns

By default the pattern used for writing artifacts is the standard Ivy Pattern. This can be changed by setting

syncRemoteRepositories {
  repoArtifactPattern = '[organisation]/[module]/[revision]/[type]s/[artifact]-[revision](.[ext])'
  repoIvyPattern = '[organisation]/[module]/[revision]/[type]s/[artifact]-[revision](.[ext])'
}

Note

If this is not specified, the default layout which is known as 'gradle' will be used from 0.4 and onwards. In the 0.3.x and earlier releases the default layout will be ivy. The change was made because it was discovered that Gradle handles local Ivy repositories sligtly differently and that plugins such as groovy, scala and jruby-gradle which relies on finding a compiler jar in a certain named way failed when used with the local ivy layout.

Flat directories

The flatDir repository supported by Gradle is not supported as it does not make sense. The purpose of this plugin is to cache remote repositories into a usable local repository. If a user already has a flatDir it does not need be be cached and if need be it can simply be copied.

Limitations

• The resolution process cannot be fine-tuned at present - not to the level at least which is described in http://gradle.org/docs/current//userguide/dependency_management.html#sec:ivy_repositories.

• There are some limitations in Apache Ivy where Maven repositories that redirect artifacts to a different URL will cause failures. A known case are Ruby Gems. There is currently no known workaround.

• It is not possible to cache plugins that are specified in the plugins block. The current workaround is to create a configuration which lists the plugins and their versions and then cache that to the local repository.