wolfgang42 / vagrant-deb

Unofficial deb repository for Vagrant

Deprecated!

Use the new offical repository instead.

This unofficial repository will be supported until April 30, 2021, after which time the repository will be unavailable.

There will also be periodic brownouts (see brownouts.txt for dates) during which attempting apt-get update will return 503 Service Unavailable, to give advance notice before the repo goes permanently offline.

About

HashiCorp provides Debian packages for Vagrant, but they don't provide a repository so you have to download the packages and reinstall them manually every time there's an update. The feature request to add a repository is not a particularly high priority for them, so as a public service I made a repository and set it up at vagrant-deb.linestarve.com.

Originally, I was using Aptly to generate the repository. However, this required downloading and serving the deb package, which was annoying because I have a fairly small disk on my server and the packages were taking up an awful lot of room.

This version generates the repository directly from the HashiCorp releases page and provides redirects to the actual packages. This has a number of benefits:

• I can publish every version of Vagrant, not just the ones that fit on my disk.
• Package files will be downloaded directly from HashiCorp, alleviating concerns about tampering.
• Increased integrity verification: the SHA256 in the Packages file comes directly from their API, and the signature of the SHA256SUMS file is now checked against the HashiCorp GPG key.

Development

The code is intended to be run in a Docker container. You will need to make a GPG key and export it as signing.private.key; it will be embedded in the generated Docker image.

To run:

docker build -t vagrant-deb .
docker run --rm -v $(pwd)/public_html/:/app/public_html/ -v $(pwd)/cache/:/app/cache/ vagrant-deb

This will create a cache/ folder (see below) as well as a public_html/ folder to be served. The public_html/ folder will also contain redirects-{amd64,i386}.conf files which should be included in the Nginx configuration, to publish the repository's pool.

Code Overview

This program creates all of the repository files from scratch, without using any of the usual helper programs. This is because they all expect that you have a package and want to publish it yourself; since I don't want to download the packages this won't work. The documentation on the Debian repository format will probably be very helpful in trying to understand how all of the pieces fit together.

update.sh is the main script, which handles orchestrating all of the various pieces. It delegates to various Python scripts to actually generate the files, then handles compressing, signing, and so on. It avoids touching files that haven't changed, so that they can be cached and don't need to be re-downloaded. It has four main steps:

1. Create various folders and download releases.json
2. Update the files for each architecture.
3. Update the Release files for the repository.
4. Update index.html with the latest version and update time.

build-packages.py generates a Packages file for each architecture, as well as the redirects file for that architecture's /pool/. It leans heavily on releaseinfo.py, which provides various library routines for extracting information about individual Vagrant versions. (See the comments in that file for details on each of the functions.)

build-release-checksums.py generates the checksum portion of the Release file.

index.tpl is a template for index.html; it contains various placeholders which envsubst will replace during the update process.

vagrant-deb's Issues

apt complains about weak digest algorithm (SHA1)

W: http://vagrant-deb.linestarve.com/dists/any/InRelease: Signature by key AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4 uses weak digest algorithm (SHA1)

Please upgrade aptly to 0.9.7

smira/aptly#366

1 signature not checked due to a missing key

A user reports the following error:

thanks for the effort. Unfortunately, the key install does not work
---
root@server: ~ # apt-key adv --keyserver hkp://keyserver.ubuntu.com:80
--recv-key AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4
Executing: /tmp/apt-key-gpghome.koQrR4shPy/gpg.1.sh --keyserver
hkp://keyserver.ubuntu.com:80 --recv-key
AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4
gpg: key CE3F3DE92099F7A4: 1 signature not checked due to a missing key
gpg: key CE3F3DE92099F7A4: "vagrant-deb.linestarve.com automatic signing
key <[email protected]>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
---
afterwards, only the old package (1.9.x) is installed. I am on a
debian 9 box.
root@server: ~ # cat /etc/debian_version
9.8
with all updates applied.

Redirect package downloads to upstream source

This has been supported by apt-get since 2009, so I'm not worried about breaking compatibility. This has the advantage of not using my bandwidth, and also ensuring that the supplied deb package really is identical to the upstream one (alleviating any concerns that I may be modifying the packages).

In addition, I might even be able to discard the actual package files after extracting their metadata, thus freeing up disk space and allowing me to serve more than just the most recent few versions, or even (per #3) other Hashicorp projects.

Key fingerprint is missing

On the index page, the apt-key adv command is missing the actual key fingerprint where $GPG_KEY is supposed to be, making it impossible to follow the setup instructions.

This seems to be a result of the following error:

gpg: can't connect to the agent: IPC connect call failed

This has happened at least once before; at the time I assumed it was a fluke but apparently not.

I should probably set -o pipefail so if something goes wrong it doesn't barrel on and update the page to claim that the last check was OK.

gpgkeys: key AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4 can't be retrieved

Hello!

I am trying to setup the repo on my Ubuntu 16.04 desktop, but I have troubles inserting the key.
Please refer to the command output for details:

$ sudo apt-key adv --keyserver pgp.mit.edu --recv-key AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4
Executing: /tmp/tmp.BNY8FAnNRf/gpg.1.sh --keyserver
pgp.mit.edu
--recv-key
AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4
gpg: requesting key 2099F7A4 from hkp server pgp.mit.edu
gpgkeys: key AD319E0F7CFFA38B4D9F6E55CE3F3DE92099F7A4 can't be retrieved
gpg: no valid OpenPGP data found.
gpg: Total number processed: 0
gpg: keyserver communications error: keyserver helper general error
gpg: keyserver communications error: unknown pubkey algorithm
gpg: keyserver receive failed: unknown pubkey algorithm

Updates to redirect file need to kick the webserver

The Vagrant 2.0.1 release was detected successfully, but I forgot that nginx would need to have its configuration reloaded to get the new file.

Alternatively, the redirect file could be changed to use a pattern, since all of the files on the Hashicorp releases server have the same pattern. This would probably be easier to implement (I don't need to figure out how to punch out of the container or add anything to the cronjob) but would break on the off chance that they changed the download link location.

Vagrant 2.0.3 deb download

Currently APT server tells that there is an updated version, but fails on fetching the .deb file with E: Failed to fetch https://vagrant-deb.linestarve.com/pool/any/main/v/vagrant/vagrant_2.0.3_amd64.deb 404 Not Found. Looks like the meta data is already there, but the file it self went missing (still processing?)

Can you please check\explain why this is happening? Currently this behavior is breaking default apt upgrade process.

Thanks!

Repository deprecation and 503 Service Unavailable

HashiCorp now has an official repository which includes packages for Vagrant. This unofficial repository will be supported until April 30, 2021, after which time it will no longer be available.

👉 Please see the repository homepage for instructions on migrating to the new repository.

⚠️ Important: In addition to switching apt lists, there is an extra step you need to take to ensure you will get updates from the new repository. Please make sure you read the instructions carefully so you don't miss future vagrant upgrades!

---

There will also be periodic brownouts during which attempting to fetch the repository index will return 503 Service Unavailable, to give advance notice before the repo goes permanently offline. This results in the following warning from apt-get update:

W: Failed to fetch https://vagrant-deb.linestarve.com/dists/any/InRelease  503  Service Temporarily Unavailable [IP: 199.38.183.38 443]
W: Some index files failed to download. They have been ignored, or old ones used instead.

If you’re getting this error, the repo is in one of these brownout periods; it will come back but please take some time to follow the instructions for migrating to the new repository.

---

If you have questions about any of this, please email me ([email protected]) or leave a comment below.

Server down

Looks like the server is down.

vagrant 2.2.5

the latest release is of 19 june as say the changelog https://github.com/hashicorp/vagrant/blob/master/CHANGELOG.md
Instead the repo is updated for 2.2.4

InRelease file expired

I am getting this error in apt update, says that the file is expired since 20h 46min 27s.

Assertion error while generating updates

Updates stopped on 2018-01-29; the failure is as follows:

Traceback (most recent call last):                                                                                                                                                                                               
  File "/app/build-packages.py", line 35, in <module>                                                                                                                                                                            
    sys.stdout.write(releaseinfo.build_control_entry(build, 'control'))                                                                                                                                                          
  File "/app/releaseinfo.py", line 70, in build_control_entry
    build_control_file(build)
  File "/app/releaseinfo.py", line 15, in wrapper
    result = getresult(*args)
  File "/app/releaseinfo.py", line 51, in build_control_file
    assert head[36:56] == '0     0     100644  ' # Owner, group, mode
AssertionError

Wrong last version on page

On https://vagrant-deb.linestarve.com/ says that is 2.2.9 but instead is 2.2.14 the latest version.

Publish ASCII-Armored Public Key

Requested via email from a user, so they can easily use it with Ansible's apt_key module.

Suggested interim workaround: Specify the MIT keyserver and key ID, and have Ansible do the lookup.

Support other Hashicorp projects

I have most of the code I need to also support the other projects on https://releases.hashicorp.com already; I think it's just a matter of doing the inital import and setting up a loop to run the code on $PROJECT rather than vagrant.

I'm not actually working on this at the moment; if anyone else wants to submit a PR I'll take it gladly. If not I'll get around to it eventually.

Add Valid-Until to Release file

Currently, if the repository is accessed over unsecured HTTP it is possible for an attacker to serve an older version of the repository, causing the victim to get an out-of-date version of Vagrant which may have security holes. Fortunately HTTPS is the default so most users are not vulnerable to this (barring attacks on HTTPS itself), but it is still a good idea to fix this. Instead of updating the main Release file only when a new version comes out, it should be updated daily with a Valid-Until of two days in the future (to account for one day of missed update).

This repository should be deprecated; Hashicorp has an official one now

via:

Announcing the HashiCorp Linux Repository (Jul 24 2020)

Today we’re happy to announce HashiCorp’s official Linux repository, a source of Debian and RPM packages for HashiCorp products. These packages will provide Linux users with a better installation and upgrade experience.

I'll keep this repo running, but the front page should mention that the official one is now available and therefore preferred. Occasional brownouts to get people to notice wouldn't be a bad idea either.