devpi fails to get packages from pypi.python.org about devpi

[devpi-bot]

Original comment by @lbrack

Which version are you running? Please note that 1.0 has been released. In the latest version, /root/dev has been removed as a default index. If you are using < 1.0, I suggest you upgrade. Cheers

[devpi-bot]

Original comment by @jbasko

I'm using 1.0 and I created /root/dev myself I think (but my memory may be failing).

[devpi-bot]

Original comment by @lbrack

Actually I got the same exact error on our side over the weekend. I restarted the server and things appear to be back to normal but this is definitely a bug.

[devpi-bot]

Original comment by @jbasko

For me restart doesn't help. I can access /root/pypi/+simple/ but /root/pypi/ fails with HTTP 500 and the message quoted above in logs.

When I do "devpi use root/pypi" and "devpi list" from command line then the listing of available packages works.

I did just try to delete my index root/dev and create it again, but the problem persists.

[devpi-bot]

Original comment by @lbrack

from my side, things are still working after 24 hours but this doesn't help you much.

For me restart doesn't help. I can access /root/pypi/+simple/ but /root/pypi/ fails with
HTTP 500 and the message quoted above in logs.

Do you mean via the web interface? If so, I get the same error as well (bug).

But the original issue stated:

I cannot use my index root/dev which inherits from root/pypi to install packages which
are provided in root/pypi because devpi fails to fetch from pypi.python.org

does $devpi install ... works? (my guess is that it does since you can use and list). Yesterday, before the restart, this (any) command would eventually time out.

If devpi install fails:

Can you set the DEVPI_SERVERDIR env. var. and point it to another directory (you will start with a fresh index), does the server work? can you access through /root/dev?

on my side I did (as we don't have root/dev):

#!bash
$ devpi index -c root/dev bases=root/pypi
root/dev:
  type=stage
  bases=root/pypi
  volatile=True
  uploadtrigger_jenkins=None
  acl_upload=root

$ devpi use root/dev
using index: http://devpi-eu.dolby.net/root/dev/ (logged in as root)

$ devpi install jsontree
--> $ bin/pip install --pre -U -i http://devpi-eu.dolby.net/root/dev/+simple/ jsontree
Downloading/unpacking jsontree
  Downloading jsontree-0.4.3.tar.gz
  Running setup.py egg_info for package jsontree
 ...
Installing collected packages: jsontree
  Running setup.py install for jsontree
...
Successfully installed jsontree
Cleaning up...

and it worked. from your side could you issue:

#!bash

$ devpi getjson http://localhost:3141

or whatever your server url is. I got something like

#!python

{
    "result": {
        "root": {
            "indexes": {
                "dev": {
                    "acl_upload": [
                        "root"
                    ], 
                    "bases": [
                        "root/pypi"
                    ], 
                    "type": "stage", 
                    "uploadtrigger_jenkins": null, 
                    "volatile": true
                }, 
                "pypi": {
                    "acl_upload": [
                        "root"
                    ], 
                    "bases": [], 
                    "type": "mirror", 
                    "uploadtrigger_jenkins": null, 
                    "volatile": false
                }
            }, 
            "username": "root"
        }, 
    }, 
    "type": "list:userconfig"

so we can see how your indices are configured.

If changing the server data dir worked for you, I am thinking that the problem may lie with a specific package you have installed. listing the files in ~/.devpi/server may be useful to try to reproduce this issue with the exact packages you have installed.

[devpi-bot]

Original comment by @jbasko

Thanks @lbrack !

I did as you instructed and devpi install worked and pip install ... -i http://localhost:3142/root/dev/ continued to not work. And then I started to question one assumption that I had taken after 1.0 release -- I still haven't spent the time to read about what indices are and how they work and I just somehow assumed that the "+simple/" part is not needed anymore in 1.0 and I stopped using it in package index URL. Once I added it back, it all started working again (except that /root/pypi/ 500 issue which you are saying is a bug).

Sorry for wasting your time. My builds are green again!

[devpi-bot]

Original comment by @jbasko

Regarding the /root/pypi 500 issue: I have now reproduced this at least 3 times and in all cases the last file (in alphabetic order) in root/pypi/links directory that gets downloaded, excluding ones that there were before I browsed to root/pypi/ in browser, is "aerolito" so I guess either this or the next package could be causing trouble for devpi.

[devpi-bot]

Original comment by @hpk42

fix issue46 -- for GET /root/pypi/ only show a link to the
simple page instead of computing "latest in-stage packages"
which is only useful for devpi's user indices.

→ <<cset 8e7d9e801b91>>

[devpi-bot]

Original comment by @hpk42

On a side note, i agree it would be nice to be able to use ".../USER/INDEX/" directly without the +simple/ suffix. Maybe we could try to detect the user agent string used by pip/easy_install and redirect accordingly. Or we could re-arrange the URL name space of devpi-server, and show the simple page at /USER/INDEX/ directly. Both approaches require experimentation. Please file an enhancement issue and/or experiment yourself.