mitsuhiko / pipsi

pip script installer

pipsi

⚠️ pipsi is no longer maintained. See pipx for an actively maintained alternative. https://github.com/pipxproject/pipx

---

pipsi = pip script installer

What does it do?

pipsi makes installing python packages with global entry points painless. These are Python packages that expose an entry point through the command line such as Pygments.

If you are installing Python packages globally for cli access, you almost certainly want to use pipsi instead of running sudo pip .... so that you get

• Isolated dependencies to guarantee no version conflicts
• The ability to install packages globally without using sudo
• The ability to uninstall a package and its dependencies without affecting other globally installed Python programs

pipsi is not meant for installing libraries that will be imported by other Python modules.

How do I get it?

curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python

to see installation options, including not automatically modifying the PATH environment variable

curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python - --help

How does it work?

pipsi is a wrapper around virtualenv and pip which installs scripts provided by python packages into isolated virtualenvs so they do not pollute your system's Python packages.

pipsi installs each package into ~/.local/venvs/PKGNAME and then symlinks all new scripts into ~/.local/bin (these can be changed by PIPSI_HOME and PIPSI_BIN_DIR environment variables respectively).

Here is a tree view into the directory structure created by pipsi after installing pipsi and running pipsi install Pygments.

/Users/user/.local
├── bin
│   ├── pipsi -> /Users/user/.local/venvs/pipsi/bin/pipsi
│   └── pygmentize -> /Users/user/.local/venvs/pygments/bin/pygmentize
├── share
│   └── virtualenvs
└── venvs
    ├── pipsi
    └── pygments

Compared to pip install --user each PKGNAME is installed into its own virtualenv, so you don't have to worry about different packages having conflicting dependencies. As long as ~/.local/bin is on your PATH, you can run any of these scripts directly.

Installing scripts from a package:

$ pipsi install Pygments

Installing scripts from a package using a particular version of python:

$ pipsi install --python /usr/bin/python3.5 hovercraft

Uninstalling packages and their scripts:

$ pipsi uninstall Pygments

Upgrading a package:

$ pipsi upgrade Pygments

Showing what's installed:

$ pipsi list

How do I get rid of pipsi?

$ pipsi uninstall pipsi

How do I upgrade pipsi?

With 0.5 and later just do this:

$ pipsi upgrade pipsi

On older versions just uninstall and reinstall.

pipsi's Issues

Make updating easier

It should be easier to tell when a venv needs to be updated and to update.

Add outdated <venv> command to show outdated packages like Homebrews brew outdated and updates does.
Add upgrade --all option to upgrade all venvs.

Unable to find installed commands using "which"

After installing a command using pipsi, the which command does not appear to be able to find the command, yet it runs normally if I just type the command:

$ hg
... output indicating it ran...

$ which hg
$

[enhancement] configure the virtualenv PATH

Currently pipsi always use a virtualenv from the PATH.

I'd like to use a python and virtualenv installed by pyenv.

plumbing commands and a simple database

in order to give people more freedoms (like installing a set of packages and scripts into a single pipsi environment lower level commands should be exposed

• add a pipsi owned json file that tracks the environments and their requested packages/scripts/script packages
• command to create a named empty environment
• command to destroy a environment and its belonging scripts
• command to install a package into a environment, supporting all pip arguments ( pip ?)
• command install all or specific scripts of a package from a environment to the bin folder
• command to recreate a environment and its packages in the easy case (must refuse in the "hard case")
• command to try and recreate a environment even in the hard case

the implementation logic of pipsi install would boil down to the following

pipsi create ${PACKAGENAME} && $(pipsi path  ${PACKAGENAME})/bin/pip install ${PACKAGENAME}
if !pipsi link_scripts ${PACKAGENAME} ${PACKAGENAME}
then
  echo ' ${PACKAGENAME} has no scripts!'
  pipsi destroy ${PACKAGENAME}
fi

what should be tracked in the "database" is still subject to planing and discssion

at least the scripts installed and the packages requested should be tracked somehow
but given the flexibility of pip

Use pipsi's python to install packages

pipsi install should maybe use the same python version it is being run under for install.

% head -n1 =pipsi
#!/home/user/.local/venvs/pipsi/bin/python
% /home/user/.local/venvs/pipsi/bin/python --version
Python 2.7.8
% pipsi install .
Using base prefix 'pyenv/3.4.1'
New python executable in /home/user/.local/venvs/ranger/bin/python3.4
…

I've installed pipsi explicitly using Python 2 and expected that packages installed through it would be installed using the same python version.

install tox under python3 failed

On my system (Archlinux), virtualenv is for py3, and virtualenv2 for py2 . 'curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python' will install get-pipsi with py3 and virtualenv. After that, I will install tox with pipsi, but failed.

Successfully installed tox virtualenv py
Cleaning up...
Traceback (most recent call last):
  File "/home/phoenix/.local/bin/pipsi", line 9, in <module>
    load_entry_point('pipsi==0.6', 'console_scripts', 'pipsi')()
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 488, in __call__
    return self.main(*args, **kwargs)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 474, in main
    self.invoke(ctx)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 758, in invoke
    return self.invoke_subcommand(ctx, cmd, cmd_name, ctx.args[1:])
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 767, in invoke_subcommand
    return cmd.invoke(cmd_ctx)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 659, in invoke
    ctx.invoke(self.callback, **ctx.params)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 325, in invoke
    return callback(*args, **kwargs)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/click/core.py", line 325, in invoke
    return callback(*args, **kwargs)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/pipsi.py", line 241, in install
    if repo.install(package, python):
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/pipsi.py", line 140, in install
    linked_scripts = self.link_scripts(scripts)
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/pipsi.py", line 86, in link_scripts
    for script in scripts:
  File "/home/phoenix/.local/venvs/pipsi/lib/python3.4/site-packages/pipsi.py", line 80, in find_scripts
    filename.startswith(prefix) and \
TypeError: startswith first arg must be bytes or a tuple of bytes, not str

~/.local/venvs/tox is created, but ~/.local/bin/tox not.

Allow passing --allow-external to pip

Trying to install pyrasite-gui with pipsi fails:

Downloading/unpacking meliae (from pyrasite-gui==2.0beta7)
  Could not find any downloads that satisfy the requirement meliae (from pyrasite-gui==2.0beta7)
  Some externally hosted files were ignored (use --allow-external meliae to allow).

Would be nice if I could pass through options to pip, such as --allow-external , --allow-all-external, --allow-unverified , etc.

Or possibly just allow passing a string to the underlying pip invocation (might help with #13?).

Add an equivalent of `pip install -e .`

I usually use my in-dev versions of my cli tools and I would love to be able to you pipsi for that. :)

Support for "pipsi install --editable"

I am not sure if this makes sense / is possible, but I would like to install some programs from their source code checkout.

No documentation for using Python 3

I tried just piping the install script to Python 3.5:

esc@omnitech ~ % curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python3.5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2805  100  2805    0     0   7277      0 --:--:-- --:--:-- --:--:--  7285
Installing pipsi
Running virtualenv with interpreter /home/esc/git/gmail2slack/.venv/bin/python2
Using real prefix '/usr'
New python executable in /home/esc/.local/venvs/pipsi/bin/python2
Not overwriting existing python script /home/esc/.local/venvs/pipsi/bin/python (you must use /home/esc/.local/venvs/pipsi/bin/python2)

But as you can see, the environment is still python2.

Would you like to register this package with homebrew?

This tool is awesome!
But I would like to manage this package with homebrew on Mac OSX.

Do you have any plans to register this with homebrew?

Transfer project to PyPA?

I think it's a good fit for that org, and would help with contributor activity.

Install multiple versions of the same package

Hi guys,

I am looking to pipsi while looking for a proper way to install multiple version of a package and its dependencies.

I am well aware that virtualenv main purpose is to separate contexts but I would need to find a way to install multiple version of the same package within the same environment.

pipsi seems to have a great approach but the package name is used to determine the virtualenv name in ~/.local/venvs/ and the symlink in ~/.local/bin.

It might be an idea to compute a virtualenv name based on the package name and the version. Using PR #28 accepting settings from environment variable, it should be also possible to configure multiple pipsi environments.

I don't know if it would be possible or not but I'd like to ask for your thoughts ?

Here would be a simple case scenario where myLib 1.0 relies on requests v1.2 and myLib 2.0 relies on requests v2.0 for instance:

from myLib_1_0 import MyObject as MyObject1
from myLib_2_0 import MyObject as MyObject2

# Do some work to migrate a MyObject1 instance to a MyObject2 instance.

Thanks!

override default location for pipsi using an environment variable

Currently pipsi wants to put things in ~/.local

I'd like to be able to set an environment variable (e.g. $PIPSI_LOCAL_DIR) which pipsi uses to determine the location of the .local folder.

My usecase:
I'm on a Chromebook, using crouton and ~/.local happens to be located on a partition that has no space. The ~/Downloads folder actually has space. If I were able to export PIPSI_LOCAL_DIR=$HOME/Downloads/.local then I would be able to have my packages on a partition with space. (Normally I'd use symlinking but I get errors related to multiple levels of symlinks)
Implementing this simple feature would open the use of pipsi to Chromebook/crouton users.

multi package envs

i'd like to use a single pipsi env for tox and detox, and get the tox and detox scripts
currently there is no way to setup something like that

i suggest a optional parameter for extra packages

Support creating a virtualenv with --system-site-packages

I have some python software that has a dependency on a system-provided python library that isn't available via the usual means (pip, etc.). I still want to install the software into a virtualenv, using pipsi, but I need pipsi to create the virtualenv using virtualenv --system-site-packages. Can this be added as an option to pipsi install?

Installing powerline-status fails

Installing powerline-status fails and uninstalls when it doesn't find any scripts, even though setup.py clearly created scripts:

pipsi install powerline-status
[...snip...]
    changing mode of /home/me/.local/venvs/powerline-status/bin/powerline-render to 775
    changing mode of /home/me/.local/venvs/powerline-status/bin/powerline-daemon to 775
    changing mode of /home/me/.local/venvs/powerline-status/bin/powerline-lint to 775
    changing mode of /home/me/.local/venvs/powerline-status/bin/powerline-config to 775
Successfully installed powerline-status
Cleaning up...
Did not find any scripts.  Uninstalling.

Unable to install `jupyter` meta package with pipsi

The jupyter package is used to install all of Jupyter's cli tools and dependencies at once. Unfortunately, it doesn't provide any console scripts in its setup.py, instead relying on the console scripts declarations in its dependencies. pipsi doesn't seem able to detect these.

Any ideas how to successfully install this package with pipsi?

Compare jupyter/jupyter/pull/149 (unfortunately, this doesn't end up working completely either)

Actually set up travis and appveyor

Upgrading pip in virtualenvs managed by pipsi

I just spent a small amount of time wondering why pip would nag me to upgrade while using pipsi after I had already upgraded my system pip and the pip in the pipsi virtualenv. I finally realized that you need to upgrade pip in every single virtualenv (which is pretty obvious in hindsight, but well).

It would be nice if pipsi could offer a command to upgrade pip in a selected managed virtualenv, or in all managed virtualenvs at once.

fails to install xonsh: "setup.py --name" includes logo

When trying to install xonsh with pipsi it fails because xonsh's setup.py is fancy and prints a logo..

pipsi uses setup.py --name to get the name:

% pipsi install -e ~v/xonsh 
Traceback (most recent call last):
  File "/home/user/.local/bin/pipsi3", line 9, in <module>
    load_entry_point('pipsi==0.9.dev0', 'console_scripts', 'pipsi')()
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/decorators.py", line 26, in new_func
    return ctx.invoke(f, ctx.obj, *args[1:], **kwargs)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "…/pipsi/pipsi.py", line 366, in install
    if repo.install(package, python, editable):
  File "…/pipsi/pipsi.py", line 215, in install
    venv_path = self.get_package_path(package)
  File "…/pipsi/pipsi.py", line 187, in get_package_path
    return join(self.home, normalize_package(package))
  File "…/pipsi/pipsi.py", line 61, in normalize_package
    requirement = Requirement.parse(value)
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2986, in parse
    reqs = list(parse_requirements(s))
  File "/home/user/.local/venvs3/pipsi/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2917, in parse_requirements
    raise RequirementParseError("Missing distribution spec", line)
pkg_resources.RequirementParseError: Missing distribution spec ╓██▄

Initially reported for xonsh at: xonsh/xonsh#288 and I've been asked to also report it for pipsi.

What do you think?
Should xonsh just don't output the logo altogether?
Is there a good way for xonsh to only output the logo in "safe" circumstances?
Can pipsi do something about this?

Manual package install option

It would be helpful to have an option for pipsi install that would prevent it from uninstalling if it can't find any scripts. Maybe print out a big warning message if that happens with instructions on how to manually create the symlinks. Kind of like how homebrew guides similar issues with brew install.

I imaging there are a lot of projects that are "a total mess"... :-)

Originally mentioned in #56

pips fails on `list` when nothing installed

It would be great to have meaningful output when no packages have been installed yet, but the user runs pipsi list. Right now this is the result on my system:

$ pipsi list
Packages and scripts installed through pipsi:
Traceback (most recent call last):
  File "/usr/local/bin/pipsi", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pipsi.py", line 416, in list_cmd
    for venv, scripts in repo.list_everything():
  File "/usr/local/lib/python2.7/site-packages/pipsi.py", line 307, in list_everything
    for venv in os.listdir(self.home):
OSError: [Errno 2] No such file or directory: '/Users/ibeekman/.local/venvs'

This might help in creating an acceptable test do block in a Homebrew formula. (See #60 )

Cannot install ansible (OSX)

Looks like pipsi isn't finding the ansible scripts.

✗ pipsi install ansible
New python executable in /Users/erik/.local/venvs/ansible/bin/python2.7
Also creating executable in /Users/erik/.local/venvs/ansible/bin/python
Installing setuptools, pip, wheel...done.
Collecting ansible
Collecting paramiko (from ansible)
  Using cached paramiko-1.15.2-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages (from ansible)
Collecting PyYAML (from ansible)
Collecting pycrypto>=2.6 (from ansible)
Collecting jinja2 (from ansible)
  Using cached Jinja2-2.8-py2.py3-none-any.whl
Collecting ecdsa>=0.11 (from paramiko->ansible)
  Using cached ecdsa-0.13-py2.py3-none-any.whl
Collecting MarkupSafe (from jinja2->ansible)
Installing collected packages: ecdsa, pycrypto, paramiko, PyYAML, MarkupSafe, jinja2, ansible
Successfully installed MarkupSafe-0.23 PyYAML-3.11 ansible-1.9.2 ecdsa-0.13 jinja2-2.8 paramiko-1.15.2 pycrypto-2.6.1
Did not find any scripts.  Uninstalling.✗ pipsi install ansible
New python executable in /Users/erik/.local/venvs/ansible/bin/python2.7
Also creating executable in /Users/erik/.local/venvs/ansible/bin/python
Installing setuptools, pip, wheel...done.
Collecting ansible
Collecting paramiko (from ansible)
  Using cached paramiko-1.15.2-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages (from ansible)
Collecting PyYAML (from ansible)
Collecting pycrypto>=2.6 (from ansible)
Collecting jinja2 (from ansible)
  Using cached Jinja2-2.8-py2.py3-none-any.whl
Collecting ecdsa>=0.11 (from paramiko->ansible)
  Using cached ecdsa-0.13-py2.py3-none-any.whl
Collecting MarkupSafe (from jinja2->ansible)
Installing collected packages: ecdsa, pycrypto, paramiko, PyYAML, MarkupSafe, jinja2, ansible
Successfully installed MarkupSafe-0.23 PyYAML-3.11 ansible-1.9.2 ecdsa-0.13 jinja2-2.8 paramiko-1.15.2 pycrypto-2.6.1
Did not find any scripts.  Uninstalling.

"pip install" should exit non-zero if it failed with "Did not find any scripts. Uninstalling."

Currently it does exit with 0.

% pipsi install ranger
Already using interpreter …/pyenv/3.4.3/bin/python3.4
Using base prefix '…/pyenv/3.4.3'
New python executable in /home/user/.local/venvs/ranger/bin/python3.4
Also creating executable in /home/user/.local/venvs/ranger/bin/python
Installing setuptools, pip...done.
You are using pip version 6.1.1, however version 7.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting ranger
  Using cached ranger-0.9.tar.gz
Installing collected packages: ranger
  Running setup.py install for ranger
Successfully installed ranger-0.9
Did not find any scripts.  Uninstalling.
% echo $?
0

Maybe with os.EX_CANTCREAT (73 for me), but that might not be defined(?!):

Exit code that means a user specified output file could not be created.
Availability: Unix.

A means of disabling the self update feature for Homebrew installs

@DomT4 (Homebrew team) said:

Is there any way to disable the self-update feature? That sort of thing doesn't really play nice with package management

And then:

It's not necessarily a blocker but ideally we'd file an Issue upstream asking for a way to disable the self-update mechanism impacting the tool itself specifically.

So they may not accept the Homebrew formula unless there is a build time option that will disable pispsi upgrade pipsi. Would this be possible to do?

Thanks

Python prefix is incorrect

Hi,

When I do a pipsi install, the python prefix in the scripts first line is incorrect and point to the system python. I'm trying to figure why, but I can't find an explanation. Do you have any pointer where I should look? I'm using ZSH on OSX if that matters somehow.

This is the output with virtualenv -v and pip -v -v -v https://gist.github.com/marcaurele/ef8dd46b802bd0cde77a

If I do it manually, of course it works (virtualenv & pip), but that's not the point.

pipsi dont work on Windows

Thanks for that, however, pipsi dont work on Windows

If you do not have the intention to support the windows, it would be nice to make this explicit in the README and classifiers field from setup.py

Display python version/location with `pipsi --version`

It would be useful to make pipsi --version display the Python version and location it is being run with.

pipsi upgrade fails with an nonsatisfiable setuptools dependency

Hello, there's a strange behaviour with pipsi upgrade for a package I'm trying to install (it's actually a package I'm writing and for which I'd like to recommend pipsi as the way to install it).

When I install it, everything is fine.

When I try to upgrade it (there's no new build on pypi, mind you) I get an error about not being able to satisfy a setuptool dependency.

My package actively requires setuptools because it uses pkg_resources.

That's the full log (OS and python-pip-pipsi version info at the end):

alan@mybuntu:~/sources$ pipsi install drb
New python executable in /home/alan/.local/venvs/drb/bin/python
Installing setuptools............done.
Installing pip...............done.
Downloading/unpacking drb
  Downloading drb-2.0.dev51.tar.gz
  Running setup.py egg_info for package drb

Downloading/unpacking Click==3.3 (from drb)
  Downloading click-3.3.tar.gz (259kB): 259kB downloaded
  Running setup.py egg_info for package Click

    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
Requirement already satisfied (use --upgrade to upgrade): setuptools in /home/alan/.local/venvs/drb/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from drb)
Installing collected packages: drb, Click
  Running setup.py install for drb

    Installing docker-rpm-builder script to /home/alan/.local/venvs/drb/bin
  Running setup.py install for Click

    warning: no previously-included files matching '*.pyc' found under directory 'docs'
    warning: no previously-included files matching '*.pyo' found under directory 'docs'
    warning: no previously-included files matching '*.pyc' found under directory 'tests'
    warning: no previously-included files matching '*.pyo' found under directory 'tests'
    warning: no previously-included files matching '*.pyc' found under directory 'examples'
    warning: no previously-included files matching '*.pyo' found under directory 'examples'
    no previously-included directories found matching 'docs/_build'
Successfully installed drb Click
Cleaning up...
  Linked script /home/alan/.local/bin/docker-rpm-builder
Done.
alan@mybuntu:~/sources$ pipsi upgrade drb
Requirement already up-to-date: drb in /home/alan/.local/venvs/drb/lib/python2.7/site-packages
Requirement already up-to-date: Click==3.3 in /home/alan/.local/venvs/drb/lib/python2.7/site-packages (from drb)
Could not find any downloads that satisfy the requirement setuptools in /home/alan/.local/venvs/drb/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from drb)
Downloading/unpacking setuptools (from drb)
No distributions at all found for setuptools in /home/alan/.local/venvs/drb/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg (from drb)
Storing complete log in /tmp/tmpLV7p4G
Failed to upgrade through pip.  Aborting.
alan@mybuntu:~/sources$ pipsi --version
pipsi, version 0.8
alan@mybuntu:~/sources$ pip --version
pip 6.0.6 from /usr/local/lib/python2.7/dist-packages (python 2.7)
alan@mybuntu:~/sources$ python --version
Python 2.7.6
alan@mybuntu:~/sources$ uname -a
Linux mybuntu 3.13.0-44-generic #73-Ubuntu SMP Tue Dec 16 00:22:43 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
alan@mybuntu:~/sources$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"

Allow to installation of more packages into a venv

Say I have installed beet (http://beets.radbox.org/) with pipsi. beet has some optional dependencies which need to be installed if you enable one of the provided plugins. To use discogs you need the discogs-client package, to use lastgenre you need the pylast package.

So, in order to use those plugins it should be possible to install more packages into the venv.

Result: "Did not find any scripts. Uninstalling."

For some of the packages (like sh) I get reponse "Did not find any scripts. Uninstalling." at the end of installation process and the package isn't installed:

$ pipsi install sh
New python executable in /home/zoresvit/.local/venvs/sh/bin/python
Installing setuptools, pip...done.
Downloading/unpacking sh
  Using download cache from /home/zoresvit/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fs%2Fsh%2Fsh-1.09.tar.gz
  Running setup.py (path:/home/zoresvit/.local/venvs/sh/build/sh/setup.py) egg_info for package sh

Installing collected packages: sh
  Running setup.py install for sh

Successfully installed sh
Cleaning up...
Did not find any scripts.  Uninstalling.

Is there a way to workaround this and keep all the packages using pipsi?

missed python3 compatibility

$ pip install pipsi
Downloading/unpacking pipsi
  Downloading pipsi-0.8.tar.gz
  Running setup.py (path:/tmp/pip_build_proofit404/pipsi/setup.py) egg_info for package pipsi

Downloading/unpacking Click (from pipsi)
  Downloading click-3.3-py2.py3-none-any.whl (58kB): 58kB downloaded
Installing collected packages: pipsi, Click
  Running setup.py install for pipsi
      File "/home/proofit404/.pyenv/versions/3.3.2/lib/python3.3/site-packages/pipsi.py", line 189
        raise exc_type, exc_value, tb
                      ^
    SyntaxError: invalid syntax


    Installing pipsi script to /home/proofit404/.pyenv/versions/3.3.2/bin
Successfully installed pipsi Click
Cleaning up...

$ pyenv version
3.3.2 (set by /home/proofit404/.pyenv/version)

Missing scripts in Sphinx

pipsi install sphinx has missed a lot of utility scripts starts with rst2, e.g. rst2html.py:

$ ls ~/.local/bin/
pipsi             sphinx-apidoc     sphinx-autogen    sphinx-build      sphinx-quickstart

$ ls ~/.local/venvs/sphinx/bin/
__pycache__           easy_install-3.5      python                rst2man.py            rst2xml.py            wheel
activate              pip                   python-config         rst2odt.py            rstpep2html.py
activate.csh          pip3                  python3               rst2odt_prepstyles.py sphinx-apidoc
activate.fish         pip3.5                python3.5             rst2pseudoxml.py      sphinx-autogen
activate_this.py      pybabel               rst2html.py           rst2s5.py             sphinx-build
easy_install          pygmentize            rst2latex.py          rst2xetex.py          sphinx-quickstart

Installing multiple versions of the same python package

This isn't a bug report, but more of a question: Is there a way to use pipsi to install multiple versions of the same package in parallel? E.g., ansible==2.0.2.0 as ansible2.0 and ansible==2.1.0.0 as ansible? ATM pipsi refuses to install additional versions, because it seems to ignore the version specification to distinguish different venvs.

Cannot install on Windows

Executing the curl -skS https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python results in the following error:

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2805 100 2805 0 0 3404 0 --:--:-- --:--:-- --:--:-- 4112
Installing pipsi
Using base prefix 'c:\python34'
New python executable in /home/uioporqwerty/.local/venvs/pipsi\Scripts\python.exe
Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
File "", line 118, in
File "", line 94, in main
File "", line 74, in install_files
File "/usr/lib/python2.7/subprocess.py", line 522, in call
return Popen(_popenargs, *_kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 710, in init
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory

Accept "scripts" keyword as a script to install.

Some packages use the scripts keyword in setup.py instead of entry_point as a way to install scripts.

Some investigation into the code, reveals, that find_links(the function responsible to find those scripts), only checks out the entry_point entry.

I'm not much of a expert on the python ecosystem, but I'll be glad to help, if you need anything.

PS: That Popen hack is scary, and I have no idea why it's there.

pipsi uses virtualenv found in $PATH instead of its own

Hi,

is there any reason pipsi uses virtualenv binary found in $PATH instead of the one in its own virtualenv? This prevents from having the system Python installation clean of every third party tools.

E.g.
pip3.5 install virtualenv
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python3.5
pip3.5 uninstall virtualenv
pipsi install pylint
FileNotFoundError: [Errno 2] No such file or directory: 'virtualenv'

Of course if I instead repeat the procedure with pipsi's virtualenv activated it works perfectly.

The fix should be a matter of changing line 233:

args = ['virtualenv', '-p', python or sys.executable, venv_path]

into something like:

args = [os.path.join(os.path.dirname(sys.executable), 'virtualenv'), '-p', python or sys.executable, venv_path]

I apologize if I am missing something obvious.

Thanks

no installs from local fs

pipsi install ./ fails as well as pipsi install somepackage-0.1.tar.gz

Ubuntu 12.04 pip install pipsi - leaves pipsi in non-working state

1. sudo pip install pipsi suceeds
2. however it seems it does not auto-install virtualenv
3. every single pipsi command fail with no such directory exist etc ..

---

installing by

curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python

spotted the issue and after sudo pip install virtualenv it worked like a charm

---

Sorry for such brief issue report. I'm wiling to add any other troubleshooting info, I'm just not aware what it might be (non familiar with python tooling)

PS> Thanks a lot for your awesome work. At the moment of trying your mitsuhiko/unp project. Kudos!

Option to explicitly tell the script name

This would allow to handle broken packages

it needs away to store per env extra data needs ( #52 )

Supersedes #57

Update pipsi on pypi

The recommended way to install pipsi seems to be to use the get_pipsi.py script from the repository. This script effectively uses pip install pipsi, which installs an older version from pipy. This had me confused because I was getting errors that were inconsistent with the code on master. Specifically the version currently on pypi doesn't work at all with Python 3, and the support environment variables seems to be broken as well. Installing directly from master fixes all these issues. Would it be possible to push a new release to pipy?

publish new release

pipsi with python 3 support from #21 is not installable via pip currently (which the get-pipsi script uses), which means that we get to miss out on all the python3 fun.

please publish a new release. :)

Cannot install awscli on Debian 8

The installation runs through to the end and finishes with the remark:

Cleaning up...
Did not find any scripts.  Uninstalling.

I encountered this problem on my vserver but it can be reproduced with the debain:8 image from docker like so:

docker run -i -t debian:8 bash
apt-get update
apt-get install curl python-pip virtualenv
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
export PATH=/root/.local/bin:$PATH
pipsi install awscli

Maybe there is a package missing. I could not figure it out. With Debian 7 it seems to work.

detailed/extended environment creation/update

after playing around with some ideas i came up with a scheme for more complex usage of pipsi

use cases examples are things like

• install tox and detox in the same env and get all commands
• install py.test with pytest-cache and pytest-xdist plugins
• install ipython with notebook support

this mainly involves installing more than one package with commands as well as installing plugins for packages with the commands

this creates the need to track a sets of details

1. list of packages/requirements to instal
2. list of packages whose commands to symlink
3. a list of extra pip arguments to propperly support #27 even for updates

i think the best way to support storing that is a json file in the virtualenv root

once them metadata is in place one can add extra commands on top of it

what i currently have in mind is

• create_env $env -- creates a empty venv
• set_pip_args $env -- configures the pip arguments for a venv
• install_package $env spec -- install a python package
• install_command $env spec -- maybe install a python package and link its scripts

once those details are in place, the update command can be altered to take all packages and the extra args into account

since this is a initial draft idea, please scorn and take appart

Allow pip option passthrough

Which would, by default, give you things like "different python intepretors"

My system default is python2, but I'd like to be able to install a package with python3.

Did not find any scripts. Uninstalling. (only with --editable)

pipsi install /path/to/project

works

pipsi install --editable /path/to/project

doesn't work

my setup.py contains

    scripts=['bin/run-consul'],

Did not find any scripts. Uninstalling.

Running
pipsi install storm

Gives out this output

New python executable in /home/user/.local/venvs/storm/bin/python2.7
Also creating executable in /home/user/.local/venvs/storm/bin/python
Installing setuptools, pip...done.
Downloading/unpacking storm
  Running setup.py (path:/home/user/.local/venvs/storm/build/storm/setup.py) egg_info for package storm

Installing collected packages: storm
  Running setup.py install for storm

    building 'storm.cextensions' extension
    gcc -pthread -fno-strict-aliasing -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -DNDEBUG -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -fPIC -I/usr/include/python2.7 -c storm/cextensions.c -o build/temp.linux-x86_64-2.7/storm/cextensions.o
    gcc -pthread -shared -Wl,-O1,--sort-common,--as-needed,-z,relro build/temp.linux-x86_64-2.7/storm/cextensions.o -L/usr/lib -lpython2.7 -o build/lib.linux-x86_64-2.7/storm/cextensions.so
Successfully installed storm
Cleaning up...
Did not find any scripts.  Uninstalling.

missing scripts

I have a project (not on PyPI, but the source is in https://github.com/warner/git-foolscap) for which pipsi fails to find any installed scripts:

git clone https://github.com/warner/git-foolscap.git
cd git-foolscap
pipsi install .
...
Successfully installed cffi-1.1.1 characteristic-14.3.0 cryptography-0.9.1 enum34-1.0.4 foolscap-0.8.0 git-foolscap-0.1 idna-2.0 ipaddress-1.0.7 pyOpenSSL-0.15.1 pyasn1-0.1.7 pyasn1-modules-0.0.5 pycparser-2.13 service-identity-14.0.0 six-1.9.0 twisted-15.2.1 zope.interface-4.1.2
Did not find any scripts.  Uninstalling.

My project's setup.py uses "scripts=" to install the two command-line executables, rather than entrypoints:

    'scripts': ["bin/git-foolscap", "bin/git-remote-pb"],

After a bit of digging, I found that pipsi's find_scripts() method is using dist.get_metadata_lines('RECORD') and getting lines like:

../../../bin/git-foolscap,sha256=...
../../../bin/git-remote-pb,sha256=...
git_foolscap-0.1.dist-info/DESCRIPTION.rst,sha256=...
git_foolscap-0.1.dist-info/METADATA,sha256=...

find_scripts() is then normalizing the ../../../bin/git-foolscap pathname relative to the current directory, which is (in this case) the git-foolscap Git checkout (but would be whatever random directory the user happens to run 'pipsi install' from, for normal installs from PyPI). The resulting absolute path is not in the virtualenv's bin/ prefix, and so it is not recognized as a script.

The path-filtering part of find_scripts() looks like this:

    for filename in files:
        filename = os.path.normpath(os.path.realpath(filename))
        if os.path.isfile(filename) and \
           filename.startswith(prefix) and \
           os.access(filename, os.X_OK):
            yield filename

I think it might be sufficient to replace the realpath(filename) with realpath(os.path.join(SOMETHING,filename)), where the SOMETHING is the virtualenv's site-packages/ directory, but I don't know how to compute that directory from here. (Is there some other Distribution method that tells you what the RECORD paths are relative to?)

I don't know if this bug is a consequence of using scripts= instead of entrypoints= in my setup.py, or if I'm doing something else unusual.

Incidentally, it seems a bit strange that entry_points.txt is the last option used.. seems like it's the one that's most likely to produce good results (not for my package, but maybe for others). RECORD kinda seems like a last resort.

thanks!
-Brian

use /usr/local when running as root

Starting here: http://thread.gmane.org/gmane.comp.python.distutils.devel/25267/focus=25436

there was some discussion of the fact that sudo pip install is a bit of an antipattern.

I raised pipsi as a possible replacement.

One issue though is that pipsi defaults to ~ even when running as root.

Do you think it would be reasonable to make pipsi, when run as root, do the equivalent of pipsi --home=/usr/local/pipsi/environments --bin-dir=/usr/local/bin ?