Coder Social home page Coder Social logo

030 / yaam Goto Github PK

View Code? Open in Web Editor NEW
2.0 2.0 1.0 355 KB

Yet Another Artifact Manager (YAAM) is an artifact repository that is capable of storing and preserving artifacts.

License: MIT License

Go 97.07% Dockerfile 1.99% Ruby 0.05% Shell 0.89%
artifact-repository cache gradle maven artifactory nexus3 golang artifactory-oss npm verdaccio

yaam's Introduction

YAAM

CI GoDoc Widget GitHub go.mod Go version Go Report Card StackOverflow SE Questions DevOps SE Questions ServerFault SE Questions Docker Pulls Docker Image Size (latest semver) Issues Pull requests Total downloads GitHub forks GitHub watchers GitHub stars License Repository Size Contributors Commit activity Last commit Release date GitHub release (latest SemVer) Bugs Code Smells Coverage Duplicated Lines (%) Lines of Code Maintainability Rating Quality Gate Status Reliability Rating Security Rating Technical Debt Vulnerabilities codecov GolangCI Chocolatey yaam codebeat badge Conventional Commits semantic-release

Although there are many artifact managers, like Artifactory, Nexus3 and Verdaccio, they are either monoliths, consume a lot of resources (memory and CPU), lack Infrastructure as Code (IaC) or do not support all kind of artifact types. Yet Another Artifact Manager (YAAM):

  • is an artifact manager like Artifactory, Nexus3 or Verdaccio.
  • enforces IaC.
  • has no UI.
  • does not have a database.
  • scales horizontally.
  • supports downloading and publication of Apt, Generic, Maven and NPM artefacts, preserves NPM and Maven packages from public repositories and unifies Maven repositories.

Quickstart

Create a directory and change the permissions to ensure that YAAM can store artifacts:

mkdir ~/.yaam/repositories
sudo chown 9999 -R ~/.yaam/repositories

Configure YAAM by creating a ~/.yaam/config.yml with the following content:

---
caches:
  apt:
    3rdparty-ubuntu-nl-archive:
      url: http://nl.archive.ubuntu.com/ubuntu/
  maven:
    3rdparty-maven:
      url: https://repo.maven.apache.org/maven2/
    3rdparty-maven-gradle-plugins:
      url: https://plugins.gradle.org/m2/
    3rdparty-maven-spring:
      url: https://repo.spring.io/release/
    other-nexus-repo-releases:
      url: https://some-nexus/repository/some-repo/
      user: some-user
      pass: some-pass
  npm:
    3rdparty-npm:
      url: https://registry.npmjs.org/
groups:
  maven:
    hello:
      - maven/releases
      - maven/3rdparty-maven
      - maven/3rdparty-maven-gradle-plugins
      - maven/3rdparty-maven-spring
publications:
  generic:
    - something
  maven:
    - releases
  npm:
    - 3rdparty-npm

Start YAAM:

docker run \
  -e YAAM_LOG_LEVEL=trace \
  -e YAAM_USER=hello \
  -e YAAM_PASS=world \
  --rm \
  --name=yaam \
  -it \
  -v /home/${USER}/.yaam:/opt/yaam/.yaam \
  -p 25213:25213 utrecht/yaam:v0.5.4

Once YAAM has been started, configure a project to ensure that artifacts will be downloaded from this artifact manager.

Apt

sudo vim /etc/apt/auth.conf.d/hello.conf

machine http://localhost
login hello
password world

sudo vim /etc/apt/sources.list

deb http://localhost:25213/apt/3rdparty-ubuntu-nl-archive/ focal main restricted

Preserve the artifacts:

sudo apt-get update

Generic

Upload:

curl -X POST -u hello:world \
http://yaam.some-domain/generic/something/world4.iso \
--data-binary @/home/${USER}/Downloads/ubuntu-22.04.1-desktop-amd64.iso

Troubleshooting:

413 Request Entity Too Large

add:

data:
  proxy-body-size: 5G

and restart the controller pod.

Verify in the /etc/nginx/nginx.conf file that the client_max_body_size has been increased to 5G.

Download:

curl -u hello:world http://yaam.some-domain/generic/something/world6.iso \
-o /tmp/world6.iso

Gradle

Adjust the build.gradle and/or settings.gradle:

repositories {
  maven {
    allowInsecureProtocol true
    url 'http://localhost:25213/maven/releases/'
    authentication {
      basic(BasicAuthentication)
    }
    credentials {
      username "hello"
      password "world"
    }
  }
  maven {
    allowInsecureProtocol true
    url 'http://localhost:25213/maven/3rdparty-maven/'
    authentication {
      basic(BasicAuthentication)
    }
    credentials {
      username "hello"
      password "world"
    }
  }
  maven {
    allowInsecureProtocol true
    url 'http://localhost:25213/maven/3rdparty-maven-gradle-plugins/'
    authentication {
      basic(BasicAuthentication)
    }
    credentials {
      username "hello"
      password "world"
    }
  }
}

Publish:

publishing {
  publications {
    mavenJava(MavenPublication) {
      versionMapping {
        usage('java-api') {
          fromResolutionOf('runtimeClasspath')
        }
        usage('java-runtime') {
          fromResolutionResult()
        }
      }
    }
  }

  repositories {
    maven {
        allowInsecureProtocol true
        url 'http://localhost:25213/maven/releases/'
        authentication {
          basic(BasicAuthentication)
        }
        credentials {
          username "hello"
          password "world"
        }
    }
  }
}

Preserve the artifacts:

./gradlew clean

or publish them:

./gradlew publish

NPM

Create a .npmrc file in the directory of a particular NPM project:

registry=http://localhost:25213/npm/3rdparty-npm/
always-auth=true
_auth=aGVsbG86d29ybGQ=
cache=/tmp/some-yaam-repo/npm/cache20220914120431999

Note: the _auth key should be populated with the output of: echo -n 'someuser:somepass' | openssl base64.

npm i -d

Run

Next to docker, one could also use a binary or K8s or OpenShift to run YAAM:

Other

yaam's People

Contributors

030 avatar github-actions[bot] avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar

Forkers

pschakravarthi

yaam's Issues

caches.yaml should change to caches dir with npm.yaml and maven.yaml

current

~/.yaam/conf/caches.yaml
mavenReposAndUrls:
3rdparty-maven: https://repo.maven.apache.org/maven2/
3rdparty-maven-gradle-plugins: https://plugins.gradle.org/m2/
3rdparty-maven-spring: https://repo.spring.io/release/
3rdparty-npm: https://registry.npmjs.org/

expected

~/.yaam/conf/caches/maven.yaml
reposAndUrls:
3rdparty-maven: https://repo.maven.apache.org/maven2/
3rdparty-maven-gradle-plugins: https://plugins.gradle.org/m2/
3rdparty-maven-spring: https://repo.spring.io/release/

~/.yaam/conf/caches/npm.yaml
reposAndUrls:
3rdparty-npm: https://registry.npmjs.org/

group repos

Group multiple maven repos into one endpoint.

APT

  • How to configure apt caching.

AWS

Terraform

  • S3

SAM

  • API Gateway
  • Lambda @ edge
  • S3

Security scan

  • scan on push, do not save if high vulnerabilities and return 500.
  • scan on caching, prevent save if issues and return 500.
  • setup a trivy server
  • when an artifact is downloaded, send it to the server
  • if response not ok then ensure that user cannot download the artifact

POST to npm/v1/security/audits/quick fails

time="2022-10-16T23:30:12Z" level=error msg="requestURI: '/npm/3rdparty-npm/-/npm/v1/security/audits/quick' should start with a: '/' and contain an extension" func=main.httpInternalServerErrorReadTheLogs file="/go/yaam/cmd/yaam/main.go:27"

Documentation

  • Explain how to upload a generic artifact.
  • Configure Maven caching.
  • Maven unification.
  • Maven publishing.
  • NPM Publication.
  • NPM preserving.
  • How to download generic artifacts.
  • How to configure YAAM.
  • Why another Artifact manager?
  • Create a table with caching, unification and publication columns and indicate what artifact types are supported.
  • Construct CHANGELOG by applying the conventional commit policy.

https

  • Create doc how to enable https.
  • Add code that enables it.

open yaam.log: permission denied

time="2022-11-18T13:12:19Z" level=fatal msg="open yaam.log: permission denied" func=main.main file="/go/yaam/cmd/yaam/main.go:203"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.