drone / autoscaler

Automatically adds or removes instances based on build volume

Gitness

Gitness is an open source development platform packed with the power of code hosting and automated DevOps pipelines.

Overview

Gitness is an open source development platform packed with the power of code hosting and automated continuous integration pipelines.

Running Gitness locally

The latest publicly released docker image can be found on harness/gitness.

To install Gitness yourself, simply run the command below. Once the container is up, you can visit http://localhost:3000 in your browser.

docker run -d \
  -p 3000:3000 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /tmp/gitness:/data \
  --name gitness \
  --restart always \
  harness/gitness

The Gitness image uses a volume to store the database and repositories. It is highly recommended to use a bind mount or named volume as otherwise all data will be lost once the container is stopped.

See docs.gitness.com to learn how to get the most out of Gitness.

Where is Drone?

Gitness represents a massive investment in the next generation of Drone. Where Drone focused on continuous integration, Gitness adds source code hosting, bringing code management and pipelines closer together.

The goal is for Gitness to eventually be at full parity with Drone in terms of pipeline capabilities, allowing users to seemlessly migrate from Drone to Gitness.

But, we expect this to take some time, which is why we took a snapshot of Drone as a feature branch drone (README) so it can continue development.

As for Gitness, the development is taking place on the main branch.

For more information on Gitness, please visit gitness.com.

For more information on Drone, please visit drone.io.

Gitness Development

Pre-Requisites

Install the latest stable version of Node and Go version 1.20 or higher, and then install the below Go programs. Ensure the GOPATH bin directory is added to your PATH.

Install protobuf

• Check if you've already installed protobuf protoc --version
• If your version is different than v3.21.11, run brew unlink protobuf
• Get v3.21.11 curl -s https://raw.githubusercontent.com/Homebrew/homebrew-core/9de8de7a533609ebfded833480c1f7c05a3448cb/Formula/protobuf.rb > /tmp/protobuf.rb
• Install it brew install /tmp/protobuf.rb
• Check out your version protoc --version

Install protoc-gen-go and protoc-gen-go-rpc:

• Install protoc-gen-go v1.28.1 go install google.golang.org/protobuf/cmd/[email protected] (Note that this will install a binary in $GOBIN so make sure $GOBIN is in your $PATH)

• Install protoc-gen-go-grpc v1.2.0 go install google.golang.org/grpc/cmd/[email protected]

$ make dep
$ make tools

Build

First step is to build the user interface artifacts:

$ pushd web
$ yarn install
$ yarn build
$ popd

After that, you can build the gitness binary:

$ make build

Run

This project supports all operating systems and architectures supported by Go. This means you can build and run the system on your machine; docker containers are not required for local development and testing.

To start the server at localhost:3000, simply run the following command:

./gitness server .local.env

Auto-Generate Gitness API Client used by UI using Swagger

Please make sure to update the autogenerated client code used by the UI when adding new rest APIs.

To regenerate the code, please execute the following steps:

• Regenerate swagger with latest gitness binary ./gitness swagger > web/src/services/code/swagger.yaml
• navigate to the web folder and run yarn services

The latest API changes should now be reflected in web/src/services/code/index.tsx

User Interface

This project includes a full user interface for interacting with the system. When you run the application, you can access the user interface by navigating to http://localhost:3000 in your browser.

REST API

This project includes a swagger specification. When you run the application, you can access the swagger specification by navigating to http://localhost:3000/swagger in your browser (for raw yaml see http://localhost:3000/openapi.yaml).

For testing, it's simplest to just use the cli to create a token (this requires gitness server to run):

# LOGIN (user: admin, pw: changeit)
$ ./gitness login

# GENERATE PAT (1 YEAR VALIDITY)
$ ./gitness user pat "my-pat-uid" 2592000

The command outputs a valid PAT that has been granted full access as the user. The token can then be send as part of the Authorization header with Postman or curl:

$ curl http://localhost:3000/api/v1/user \
-H "Authorization: Bearer $TOKEN"

CLI

This project includes VERY basic command line tools for development and running the service. Please remember that you must start the server before you can execute commands.

For a full list of supported operations, please see

$ ./gitness --help

Contributing

Refer to CONTRIBUTING.md

License

Apache License 2.0, see LICENSE.

autoscaler's Issues

Switch the default watchtower image

I would suggest to update the default for the used Watchtower image as DockerHub will switch the payment policy and I won't way the huge amount which will lead to the removal of the Webhippie organization on DockerHub. If you want to continue using this Watchtower image you should switch to ghcr.io/dockhippie/watchtower or quay.io/webhippie/watchtower.

[Feature Request] GCP spot instance support

It looks like AWS spot instance is supported (https://github.com/search?q=repo%3Adrone%2Fautoscaler%20spot&type=code) but not GCP yet? Thanks!

This is a huge cost saving for us since spot instance is ~5x cheaper

Creating a new instance on Hetznercloud fails with "image not found"

Since today, the autoscaler is unable to provision new instances on hetzner cloud. The drone cli tells me the ubuntu 16.04 image could not be found:

$ drone server info $(drone server ls)
Name: agent-wawR6snj
Address: 
Region:  
Size:    
State:   error
Error:   image 'ubuntu-16.04' not found (invalid_input)

My guess is hetzner deprecated the ubuntu 16.04 image.

I was able to fix this by setting the DRONE_HETZNERCLOUD_IMAGE environment variable to ubuntu-20.04 - I think the default should be set to 20.04. I could send a PR if you'd accept one.

(Hetzner) Sometimes the server is not destroyed

Once in a while the created instance on Hetzner does not get destroyed. This results in a big bill from Hetzner because the server is still running.

Can it be that the autoscaler forgets that some servers exist when the autoscaler gets restarted, or the git server gets restarted?

This is our config:

drone-autoscaler:
    image: drone/autoscaler:1.8.2
    restart: unless-stopped
    volumes:
      - drone_autoscaler_data:/data
    environment:
      - DRONE_POOL_MIN=0
      - DRONE_POOL_MAX=4
      - DRONE_POOL_MIN_AGE=1h
      - DRONE_CAPACITY_BUFFER=0
      - DRONE_AGENT_CONCURRENCY=5
      - DRONE_SERVER_PROTO=https
      - DRONE_SERVER_HOST=REDACTED
      - DRONE_SERVER_TOKEN=${DRONE_SERVER_TOKEN}
      - DRONE_AGENT_TOKEN=${DRONE_RPC_SECRET}
      - DRONE_HETZNERCLOUD_DATACENTER=${DRONE_HETZNERCLOUD_DATACENTER}
      - DRONE_HETZNERCLOUD_IMAGE=ubuntu-20.04
      - DRONE_HETZNERCLOUD_TYPE=cx51
      - DRONE_HETZNERCLOUD_SSHKEY=${DRONE_HETZNERCLOUD_SSHKEY}
      - DRONE_HETZNERCLOUD_TOKEN=${DRONE_HETZNERCLOUD_TOKEN}
      - DRONE_INTERVAL=10s
      - DRONE_LOGS_DEBUG=false

Support multiple aws images, each for a diferent platform

clone: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Running an autoscaler to test arm64 builds.
AWS EC2 m6g.large
Sporadically seeing failed builds on newly launched instances:

clone: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

latest: Pulling from drone/git
Digest: sha256:091ecd02ee4ac5154fd76133c5055b2345a61cbc17182b00612df1fa7eef1510
Status: Downloaded newer image for drone/git:latest

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

However, inot consistently, maybe 10% of the time. Most of the time, no errors.

What comes to mind is to put a "sleep 1" or "systemctl restart docker" someplace. A very small change.

It looks like that is already present "[ systemctl, restart, docker ]"

Now trying a modification in drivers/internal/userdata/userdata.go . Change

  - path: /etc/docker/daemon.json
    content: |
      {
        "hosts": [ "0.0.0.0:2376", "unix:///var/run/docker.sock" ],
        "tls": true,
        "tlsverify": true,
        "tlscacert": "/etc/docker/ca.pem",
        "tlscert": "/etc/docker/server-cert.pem",
        "tlskey": "/etc/docker/server-key.pem"
      }

to

  - path: /etc/docker/daemon.json
    content: |
      {
        "hosts": [ "0.0.0.0:2376", "unix:///var/run/docker.sock" ],
        "containerd": "/run/containerd/containerd.sock",
        "tls": true,
        "tlsverify": true,
        "tlscacert": "/etc/docker/ca.pem",
        "tlscert": "/etc/docker/server-cert.pem",
        "tlskey": "/etc/docker/server-key.pem"
      }

Adding "containerd": "/run/containerd/containerd.sock" , based on this article https://stackoverflow.com/questions/68823645/what-is-purpose-of-the-switch-containerd-in-command-dockerd

I'll report back, if it eventually improves the condition.

[Feature Request] Setting capacity number per server

It seems like the capacity number for each server is always the number of cores and not configurable?

Would it make sense to have an option similar to DRONE_RUNNER_CAPACITY for autoscaler?

Add DRONE_AGENT_PUBLISHED_PORTS to Documenation

Spent the last few hours grappling with my agent docker containers deployed via autoscaler not having port 3000 exposed. As a result, drone could push jobs to the agent. I dove the source code and found the ENV var mentioned in the subject. It would have saved a lot of time if this was documented at autoscale.drone.io. Not sure how to go about contributing, figured it was worth a mention.

Furthermore, why does this var not have a default of 3000:3000 in the code? is this by design? or an overlooked issue? figure I may be doing something wrong and would like confirmation before opening a PR to make that change.

How to specify Hetzner Cloud ssh key

Documentation says that DRONE_HETZNERCLOUD_SSHKEY must be an integer. How does such integer represent an SSH key?
I already added the public key to Hetzner, but how do I provide the private key to the autoscaler and how does an SSH key relate to a simple integer?

release notes for 1.8.0

needed to tag a quick release for a bug fix, without delay. the release notes are in progress (we merged a ton of pull requests) and are coming soon. thank you for your patience!