Coder Social home page Coder Social logo

magik's Introduction

Turn your personal Github repository into your personal maven repository.

I was tired of Github Packages requiring authentication even for public access, Sonatype and Maven Central being a pain to setup, the first supporting only one snapshot at time (and you need to deal with cache) and Nexus in general being incredibly slow in comparison, therefore I decided to go on my own and write this

Usually it was already possible doing something like that, but this implicitely meant you had to have the repo cloned locally and manually committ and push. Now these limitations are gone, allowing for even faster development cycles.

What this plugin does is the following, for each project/module:

  • publishes locally to $buildDir/repo, but before doing this, it will download in advance metadata-maven.xml in order to avoid resetting/overwrite with a single entry every time and having gradle properly modify it instead
  • creates a tmp branch on the github repo
  • uploads file by file
  • creates a PR
  • merges the same PR back to master by squashing all the commits into a single one
  • deletes the tmp branch

Welcome Magik (MAven repository on Github written In Kotlin)

plugins {
  id("elect86.magik") version "0.3.3"
}

image

How to use

You have two options (!). Too many, I know, sorry. You can use:

  1. a Github repository acting as a pure Maven repository
  2. Github Packages without the authentication hassle, even for just consuming dependencies

Method 1, Github repository as Maven repository

Authentication

You should first be sure to be able to connect to GitHub using ssh.

Then create a personal access token, be sure to check repo. Then copy the token and paste it in gradle.property in your home (~/.gradle) as

githubToken=ghp_...

Take in account github is convention, you can change it with whatever you want (read the next section). Magik will look for your token in a variable names as ${githubRepositoryName}Token

Publishing
publishing {
    publications {
         // you can pass a name to overwrite the default "maven"
         // createGithubPublication("my-name") 
        createGithubPublication { this: MavenPublication
            // if your project already defines `groupId`, `artifactId` and `version`, then you can skip these here
            groupId = "org.gradle.sample"
            artifactId = "prova"
            version = "0.1"

            from(components["java"])
        }.github {
            // this adds another (snapshot) publication, copying from the previous one: 
            // - gav coordinates 
            // - component type (java, javaPlatform or war)
            // - name, by default appended with the `Snapshot` postfix, 
            // eg: publishMavenPublicationToGithubRepository ->
            // ->  publishMavenSnapshotPublicationToGithubRepository
            addSnapshotPublication()
        }
    }
    // don't use `repositories.github(..)`, it won't work
    // the dsl construct is necessary to distinguish it from a consume-only repo
    repositories {
        // don't use github(domain: String), that's for fetching, it won't work for publishing
        github {
            // this is optional since `github` is the default value, but it determines 
            // the token name to fetch and the consequent publishing task name 
            // eg: publishMavenPublicationToGithubRepository
            name = "github" 
            
            // this is mandatory instead: $owner/$repo on github domain
            domain = "kotlin-graphics/mary" // aka https://github.com/kotlin-graphics/mary
        }
    }
}
Fetching
repositories {
    github("kotlin-graphics/mary")
}

or

repositories {
    github("kotlin-graphics", "mary")
}
Publishing

Run publish(Maven)PublicationTo(Github)Repository or publishAllPublicationsTo(Github)Repository

where Maven is the publication name (capitalized) and Github the repository name (capitalized)

> Task :publishMavenPublicationToGithubRepository

kotlin.graphics:gli:0.8.3.0-18 published on kotlin-graphics/mary!

The printed GAV coordinates can then be easily copied and pasted where needed :)

Settings

Sometimes it happens you forget to commit before publishing. In order to avoid these situations, the default setting commitWithChanges will warn you whenever you are committing while there are changes to be committed or not staged for commit.

This requires git being available on path though, which is automatically set at begin in gitOnPath.

defaultCommitWithChanges will instead automatically highlight the given answer when asking if you want to commit anyway with changes.

[magik] Do you want to continue publishing anyway? Y/[N]:

dryRun for running without uploading anything.

verbose is self explanatory.

magik {
    commitWithChanges.convention(false)
    defaultCommitWithChanges.convention(false)
    gitOnPath.convention(configuringProject.exec {
        commandLine("git", "--version")
        standardOutput = ByteArrayOutputStream() // disable output with a dummy instance
    }.exitValue == 0)
    dryRun.convention(false)
    verbose.convention(false)
}

Setting the repository in settings.gradle.kts for all the modules in a multi-module project

Since this is a Project plugin (which applies to build.gradle.kts), there is no support for settings.gradle.kts. So, in case you prefer to apply the repositories once in the settings, just fallback to:

"https://raw.githubusercontent.com/$organization/$repo/$branch"

For example, settings.gradle.kts:

dependencyResolutionManagement {
    repositories {
        mavenCentral()
        maven("https://raw.githubusercontent.com/kotlin-graphics/mary/master")
    }
}

Method 2, clients consuming Github Packages without having to set up authentication

Create a token with the read:packages scope and add a credentials file in the root of your repository (branch master/main) hosting the packages with your nickname on the first line, and the token you just created on the second line (without the prefix ghp_, otherwise Github will detect the token code in the next commit and delete the corresponding token)

Example, mary's credentials

elect86
7V1YljEcKShzwPzPJuAPP0X55urEhF0RBWG2
Fetching
repositories {
    githubPackages("kotlin-graphics/mary")
}

Advantages and limitations

Big advantage for both: they rely on the Github infrastructure (aka bandwidth and reliability)

Method 1, advantages:
  • pure Maven repository
  • works with every client, Maven or Gradle or whatever
  • no need to expose a (read) token on the internet
  • no need to have a credentials file in your root repository
Disadvantages:
  • git is suboptimal for storing (large) binaries, it's fine for regular ones, but fat jars may represent a problem in the long run
  • every repository has a hard limit of 100 GB for its total size
  • every file has a hard limit of 100 MB for its size
Method 2, advantages:
  • is optimal for every kind of binaries, also fat jars
  • Snapshots do not work
    • Even though the docs state that snapshots are supported, we couldn’t get them to work. The first few builds work just fine; however, old snapshots are not being removed. Instead, new artifacts are constantly being added to that same release, which eventually stops working. From this point on, you will always fetch old snapshots.
  • Multiple artifacts per release do not work
    • As soon as you try to upload JavaDoc and source artifacts in addition to the library itself, the repository enters an unstable state. Even loading existing artifacts or publishing new library versions will fail with a 400 error. In our case, only deleting the artifact or repository could fix this.
  • Public versions cannot be deleted
    • While it’s possible to delete artifact versions in private repositories, GitHub doesn’t allow you to do this in public repositories. Therefore, you should carefully consider what to publish and what to keep private ;)
  • The token needs the correct scopes
    • The personal access token used to access the artifacts must have precisely the scopes listed above for everything to work. If any of these scopes is missing, you will receive error messages that often do not indicate the lack of permission!
  • Dynamic versions will prevent publishing
    • If you are using Spring Boot, and have dynamic versions enabled, the publishing will fail because the versions are not locked. While this is not directly related to GitHub packages, it is still good to be aware of. To fix this, you need to adjust the build.gradle file, so that there is a version mapping in the publishing block:

magik's People

Contributors

elect86 avatar korewalidesu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

magik's Issues

Do you have some android example project anywhere ?

I was getting this up and running but I use groovy not kotlin in gradle, I don't know if that is the reason or somethiung else but it fails ..

repositories {
    github {
        domain = "kotlin-graphics/mary"
    }
}

Fails with :
Caused by: org.gradle.internal.metaobject.AbstractDynamicObject$CustomMessageMissingMethodException: Could not find method github() for arguments [library_e4xwdkfqm5vk8ubh7idqabudu$_run_closure5$_closure14@9d1948e] on repository container of type org.gradle.api.internal.artifacts.dsl.DefaultRepositoryHandler.

Caused by: groovy.lang.MissingPropertyException: Could not get unknown property 'release' for SoftwareComponentInternal set of type org.gradle.api.internal.component.DefaultSoftwareComponentContainer.

Is it possible to disable merging to master?

Hi, I use this for publishing maven packages, and it usually works great, so thank you! But now I need to create a new repo from another existing repo, and I would like to publish the old releases that are way behind the master branch.

When I try locally, it just hangs forever. When I use GitHub actions it attempts to merge into master and fails when there is a merge conflict.

So basically, can I configure it to work the same as standard maven publishing if the release is outdated, without retroactively reverting back to standard gradle publishing?

Some problems with setting up the plugin

First of all, I am not that knowledgeable when it comes to gradle, so it is certainly possible that I have missed some things.

I tried using this plugin following the README file but came across two problems:

  1. Gradle is unable to build the project when using the alsoSnapshot() option
  2. Gradle is able to build but unable to publish when not using the alsoSnapshot() option

Some links:

  • Error log for the first problem
    • Line 31 is }.alsoSnapshot() // this clones the publication for snapshots
    • Following the stacktrace, magik most likely has a problem when running git describe --tags which outputs fatal: No names found, cannot describe anything. which it then attempts to parse to an int in the gitDistance function
    • Do I need tags for this to work? There is no mention in the README about that
  • Error log for the second problem
    • Caused by: java.lang.IllegalArgumentException: Illegal character in path at index 52: https://api.github.com/repos/kwerber/abc/contents/de\kwerber\abc\0.1\abc-0.1.jar
    • The illegal character is the first \
  • build.gradle.kts for the first problem
  • build.gradle.kts for the second problem

Some additional information:

  • I am testing this plugin on my https://github.com/kwerber/abc repo (where commit 2 and 3 cause problems 1 and 2)
  • Gradle version 7.0.2
  • Java openjdk version "16.0.1" 2021-04-20
  • and git version 2.31.1.windows.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.