esell / deb-simple Goto Github PK
View Code? Open in Web Editor NEWA lightweight, bare-bones apt repository server
License: MIT License
A lightweight, bare-bones apt repository server
License: MIT License
Add some documentation around how you would add your repo into the distro's sources.
We've run into an issue using deb-simple
where new packages can't be installed until a couple of minutes after uploading them. We have quite a few version of ~50Mb packages, so it takes a little while to hash them all to produce the Packages file.
One potential fix for this would be to move back to the pre-fsnotify behaviour and rebuild the Packages file as part of the package upload handler. This would mean our CI build wouldn't count as complete until the package was ready to install, which makes sense.
I can understand that being able to drop files into the packages directory is useful behaviour for some users, so I don't want to remove it.
@esell what do you think about making the behaviour here configurable? A setting to control if either the directory should be watched, or the Packages file should be rebuilt as a blocking part of the upload.
As I am no longer using deb-simple in my day-to-day life, it has been mostly ignored as you can likely see from the issues/PRs. If you are interested in taking over and becoming the maintainer of deb-simple, please let me know
Right now everything is put into the "stable" distro and the "main" section. Would be nice to be able to change these defaults.
Would be nice to support auto creation of Let's Encrypt certs
I've tried to build debian package in docker container golang:1.12-buster and hit an error
root@3da6c33c12a8:/go/src/github.com/esell/deb-simple# make build-deb
which -s dpkg-deb || { echo "dpkg-deb does not exist, exiting..."; exit 1; }
Illegal option -s
Usage: /usr/bin/which [-a] args
dpkg-deb does not exist, exiting...
make: *** [Makefile:26: build-deb] Error 1
here is my patch to fix it
index 3976ed9..548c147 100644
--- a/Makefile
+++ b/Makefile
@@ -23,8 +23,7 @@ build-linux:
build-osx:
GOOS=darwin go build -o release/$(BINARY)-$(VERSION)-osx
build-deb:
- which -s dpkg-deb || { echo "dpkg-deb does not exist, exiting..."; exit 1; }
- mkdir release/$(BINARY)-$(VERSION)
+ dpkg-deb --version >/dev/null || { echo "dpkg-deb does not exist, exiting..."; exit 1; }
mkdir -p release/$(BINARY)-$(VERSION)/usr/local/bin
mkdir -p release/$(BINARY)-$(VERSION)/etc/deb-simple
cp -r DEBIAN release/$(BINARY)-$(VERSION)/
Hi,
I would like to mention I created a golang package to debianize files and directories. It is not completely finished but it is already functional.
https://github.com/xor-gate/debpkg
Let me know what you think and maybe you can point me to the right direction or contribute if you would like to (when there is a use-case).
As this project is about a debian package server, is there a reason why there is none for this project? Could you create one?
In func inspectPackageControl
, we return the args via return controlBuf.String(), nil
. However, I've encountered control files that have a trailing newline. The return string from this func is used to create the 'Packages.gz' file, and if the return string has newlines, these make their way into the Packages.gz file but apt clients (esp. Ubuntu 12.04) can get unhappy about stray newlines.
The fix is to trim trailing blank chars from the controlBuf.String()
Is there a way to use this in a global production environment by blocking add/delete requests?
Hi,
I currently looking into deb-simple
as it looks promising. Unfortunately I encounter a problem that I can't circumvent.
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/apt/lists/localhost:9090_dists_stable_main_binary-amd64_Packages
E: The package lists or status file could not be parsed or opened.
my conf.json looks like the default one
{
"listenPort" : "9090",
"rootRepoPath" : "/opt/mirror",
"supportedArch" : ["amd64"],
"distroNames" : ["stable"],
"sections" : ["main"],
"enableSSL" : true,
"SSLcert" : "server.crt",
"SSLkey" : "server.key",
"enableAPIKeys" : false,
"enableSigning" : true,
"privateKey" : "./private.key"
}
I generated the private/public key with -k -ke -kn
option. The SSLcert is just a snakeoil from my ubuntu.
After initial start I upload some test files from my current deb mirror via for file in $(find . -name *_amd64.deb); do curl -k -X POST 'https://localhost:9090/upload?arch=amd64&distro=stable§ion=main' -F "file=@$file"; done
. I don't get an error message on upload from curl, nor from the server output (started with -v
flag).
My /etc/apt/sources.list
contains deb [arch=amd64] https://localhost:9090/ stable main
.
How to prevent the error from happening? I already tried sudo rm /var/lib/apt/lists/* -vf
without success.
It seems that Bionic Beaver (18.04) requires more fields than Xenial (16.04) in the Packages.gz file...
This worked on Xenial:
Filename: dists/xenial/main/binary-amd64/<name>_<version>_amd64.deb
Size: <size>
MD5sum: <md5>
SHA1: <sha>
SHA256: <sha>
But does not work on Bionic; I added these lines to the extracted file at /var/lib/apt/lists/:
Package: <name>
Architecture: amd64
Version: <version>
Priority: standard
Description: <description>
And it worked fine.
This may be an issue with my debian packaging tool, not necessarily an issue with deb-simple, but just thought you might want to know.
Right now uploads are stored in memory before being written out to disk. If you had a bunch of uploads going for large packages this could potentially be an issue.
Might be worthwhile to research other possible methods/options for uploading files.
Currently only testing for the "stable" distro so a bug was missed (#10) where createPackagesGz() was only using the default value. Create another test for non-default distros.
If you do not want to build from source you can just download a pre-built binary from the Releases section.
Uhm, there is no release. Could you make one?
When integrating with other golang software. deb-simple
could benefit from an API client package. E.g:
package main
import (
"github.com/esell/deb-simple/client"
)
func main() {
client.New(&client.Config {
APIKey: "1337",
})
client.PublishDebFile("myfile.deb")
}
Most times we already have a directory full of packages that we'd like to have served via deb-simple
. Is there an easy way to add a config flag that would allow this dir to be served instead of iterating over every file in the dir and POSTing it to the server (which takes a long time when the files are too many or too large).
Happy to take on the work if you think this is functionality you agree with (and have thoughts on how to implement).
This is a great project, thanks for making it!
Have you considered adding support for using this as a CLI tool? (Or would you accept a PR adding this mode?)
What I mean by that is I would love to be able to run ./deb-simple batch <dir full of deb files>
and have it generate all the Packages, Release, GPG-signed files, etc. I would then host the generated files on some static hosting, e.g. S3, GitHub Pages, etc.
In practice, I would run it as part of the CI pipeline for one of my projects, after goreleaser
generates the deb files. goreleaser
doesn't plan on doing this work itself in the near future: goreleaser/goreleaser#912
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.