madjlzz / gopypi

Implementation of a PyPi server in Golang.

🤓 Hey. Welcome aboard!

Thanks for joining my page!
I'm Julien, Cloud architect from 🇫🇷 Metz, France, currently working in 🇱🇺 Luxembourg.

Things I code with

OSS projects

🎁 Projects	⭐ Stars	📚 Forks	🛎 Issues	📬 Pull requests
MadProbe
MadProbe CLI
Gopypi
Chaloperie

Where to find me

All credits to Thomas Guibert for his neat blog post on Medium.

---

This README file is generated every 6 hours!
Last refresh: Thursday, 18 July, 14:33 CEST

gopypi's Issues

`pip` supports only basic authentication

pip actually supports only Basic Authentication at the moment.

When gopypi is deployed to AppEngine, we need to find a way to authenticate and authorize pip clients.

`pip` install resolves package but cannot install it

Here is the stacktrace pip is experiencing when trying to install a package from a local gopypi instance.

klaer@MadDows MINGW64 /k/sandbox/test-gopypi
$ pip install --index-url http://localhost:8080/simple/ -r requirements.txt
Looking in indexes: http://localhost:8080/simple/
Collecting pydantic==1.8.2
  Downloading https://storage.cloud.google.com/gopypi/pydantic/1.8.2/pydantic-1.8.2.tar.gz
     / 99 kB 1.7 MB/s
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1645, in gzopen
    t = cls.taropen(name, mode, fileobj, **kwargs)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1621, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1484, in __init__
    self.firstmember = self.next()
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 2289, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1094, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\gzip.py", line 276, in read
    return self._buffer.read(size)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\gzip.py", line 463, in read
    if not self._read_gzip_header():
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\gzip.py", line 411, in _read_gzip_header
    raise OSError('Not a gzipped file (%r)' % magic)
OSError: Not a gzipped file (b'\n<')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\cli\base_command.py", line 173, in _main
    status = self.run(options, args)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\cli\req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\commands\install.py", line 316, in run
    reqs, check_supported_wheels=not options.target_dir
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 95, in resolve
    collected.requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 341, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 140, in __bool__
    return any(self)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 128, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 32, in _iter_built
    candidate = func()
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 209, in _make_candidate_from_link
    version=version,
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 301, in __init__
    version=version,
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 156, in __init__
    self.dist = self._prepare()
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 227, in _prepare
    dist = self._prepare_distribution()
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\resolution\resolvelib\candidates.py", line 306, in _prepare_distribution
    self._ireq, parallel_builds=True
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 508, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 552, in _prepare_linked_requirement
    self.download_dir, hashes
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\operations\prepare.py", line 249, in unpack_url
    unpack_file(file.path, location, file.content_type)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\utils\unpacking.py", line 256, in unpack_file
    untar_file(filename, location)
  File "k:\sandbox\test-gopypi\venv\lib\site-packages\pip\_internal\utils\unpacking.py", line 181, in untar_file
    tar = tarfile.open(filename, mode, encoding="utf-8")
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1591, in open
    return func(name, filemode, fileobj, **kwargs)
  File "C:\Users\klaer\AppData\Local\Programs\Python\Python37\lib\tarfile.py", line 1649, in gzopen
    raise ReadError("not a gzip file")
tarfile.ReadError: not a gzip file

Running on App Engine

Hi,

First of all - great thanks! This is a great little project, exactly what I was looking for - go for quick startup, simple code, GCS backend. Super!

I am trying to setup the gopypi server on App Engine, just as you've suggested. However, the templates seem to have trouble loading. What I get is

panic: html/template: pattern matches no files: `/layers/google.go.appengine_gomod/srv/*/*.gohtml`

goroutine 1 [running]:
html/template.Must(...)
	/usr/local/go/src/html/template/template.go:372
github.com/MadJlzz/gopypi/internal/pkg/template.New(0xc000102120)
	/layers/google.go.appengine_gomod/srv/internal/pkg/template/template.go:19 +0xea
main.main()
	/layers/google.go.appengine_gomod/srv/main.go:23 +0x199

It works just fine locally, of course.
I tried running from cmd/gcp/main.go as well as copying the main.go to the root folder.

The app.yaml is:

runtime: go115
service: gopypi
#main: ./cmd/gcs # either this or default from root

entrypoint: main -bucket *bucketname* -port 8080 -credentials ""

Could you perhaps share an example app.yaml?
Any other hints ?

Thanks again!