pytest-dev / pytest-runner

Setup scripts can use pytest-runner to add setup.py test support for pytest runner.

Deprecation Notice

pytest-runner depends on deprecated features of setuptools and relies on features that break security mechanisms in pip. For example 'setup_requires' and 'tests_require' bypass pip --require-hashes. See also pypa/setuptools#1684.

It is recommended that you:

• Remove 'pytest-runner' from your setup_requires, preferably removing the setup_requires option.
• Remove 'pytest' and any other testing requirements from tests_require, preferably removing the tests_requires option.
• Select a tool to bootstrap and then run tests such as tox.

Usage

• Add 'pytest-runner' to your 'setup_requires'. Pin to '>=2.0,<3dev' (or similar) to avoid pulling in incompatible versions.
• Include 'pytest' and any other testing requirements to 'tests_require'.
• Invoke tests with setup.py pytest.
• Pass --index-url to have test requirements downloaded from an alternate index URL (unnecessary if specified for easy_install in setup.cfg).
• Pass additional py.test command-line options using --addopts.
• Set permanent options for the python setup.py pytest command (like index-url) in the [pytest] section of setup.cfg.
• Set permanent options for the py.test run (like addopts or pep8ignore) in the [pytest] section of pytest.ini or tox.ini or put them in the [tool:pytest] section of setup.cfg. See pytest issue 567.
• Optionally, set test=pytest in the [aliases] section of setup.cfg to cause python setup.py test to invoke pytest.

Example

The most simple usage looks like this in setup.py:

setup(
    setup_requires=[
        'pytest-runner',
    ],
    tests_require=[
        'pytest',
    ],
)

Additional dependencies require to run the tests (e.g. mock or pytest plugins) may be added to tests_require and will be downloaded and required by the session before invoking pytest.

Follow this search on github for examples of real-world usage.

Standalone Example

This technique is deprecated - if you have standalone scripts you wish to invoke with dependencies, use pip-run.

Although pytest-runner is typically used to add pytest test runner support to maintained packages, pytest-runner may also be used to create standalone tests. Consider this example failure, reported in jsonpickle #117 or this MongoDB test demonstrating a technique that works even when dependencies are required in the test.

Either example file may be cloned or downloaded and simply run on any system with Python and Setuptools. It will download the specified dependencies and run the tests. Afterward, the the cloned directory can be removed and with it all trace of invoking the test. No other dependencies are needed and no system configuration is altered.

Then, anyone trying to replicate the failure can do so easily and with all the power of pytest (rewritten assertions, rich comparisons, interactive debugging, extensibility through plugins, etc).

As a result, the communication barrier for describing and replicating failures is made almost trivially low.

Considerations

Conditional Requirement

Because it uses Setuptools setup_requires, pytest-runner will install itself on every invocation of setup.py. In some cases, this causes delays for invocations of setup.py that will never invoke pytest-runner. To help avoid this contingency, consider requiring pytest-runner only when pytest is invoked:

needs_pytest = {'pytest', 'test', 'ptr'}.intersection(sys.argv)
pytest_runner = ['pytest-runner'] if needs_pytest else []

# ...

setup(
    #...
    setup_requires=[
        #... (other setup requirements)
    ] + pytest_runner,
)

For Enterprise

Available as part of the Tidelift Subscription.

This project and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use.

Learn more.

Security Contact

To report a security vulnerability, please use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

pytest-runner's Issues

move under pytest-dev team?

Originally reported by: Anatoly Bubenkov (Bitbucket: bubenkoff, GitHub: bubenkoff)

---

Please consider moving under shiny new pytest-dev team

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/5

set development status to inactive 7

as pytest-runner is intentionally on the way out, we should add a release that sets it to inactive in the trove classifiers

Code not pep8 compliant due to the usage of tabs instead of spaces

A build on one of my projects just failed because pytest-runner uses tabs instead of spaces as indentation: https://travis-ci.org/WoLpH/python-progressbar/jobs/310044371

It seems that the latest version of the pep8 tool doesn't allow for tabs anymore: https://pypi.python.org/pypi/pep8

Using test=pytest in setup.cfg gives "ModuleNotFoundError: No module named 'ptr'"

As of a few hours ago, when my CI runs my unit tests with python setup.py test using pytest-runner, I get a traceback (listed below the minimal example).

Here's how to reproduce it. In an empty directory:

Create setup.py:

from setuptools import setup

setup(
    name='PytestTest',
    python_requires='>=3.6',
    setup_requires=['pytest-runner'],
    tests_require=['pytest'],
)

Create setup.cfg:

[aliases]
test=pytest

Run the following in a clean virtualenv eg. using Virtualenvwrapper:

mktmpenv -p /usr/bin/python3
cd the_dir_you_were_just_in
python setup.py test

Traceback:

$ python setup.py test
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    setup_requires=['pytest-runner'],
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.6/distutils/core.py", line 134, in setup
    ok = dist.parse_command_line()
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/setuptools/dist.py", line 710, in parse_command_line
    result = _Distribution.parse_command_line(self)
  File "/usr/lib/python3.6/distutils/dist.py", line 472, in parse_command_line
    args = self._parse_command_opts(parser, args)
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/setuptools/dist.py", line 1025, in _parse_command_opts
    nargs = _Distribution._parse_command_opts(self, parser, args)
  File "/usr/lib/python3.6/distutils/dist.py", line 528, in _parse_command_opts
    cmd_class = self.get_command_class(command)
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/setuptools/dist.py", line 845, in get_command_class
    self.cmdclass[command] = cmdclass = ep.load()
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/home/mydomain.net/username/.virtualenvs/tmp-3c301df20ea12392/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'ptr'

Other info: OS is Ubuntu 18.04. Pytest is... I think whatever is on PyPI right now?

$ pip list
Package       Version
------------- -------
pip           19.0.2 
pkg-resources 0.0.0  
setuptools    40.8.0 

Caveat: I'm not 100% sure this is a Pytest issue and not a Setuptools issue. Happy to re-file over on Setuptools if I'm wrong.

How do I setup pytest-runner correctly

Maybe I'm reading this incorrectly but doesn't the first recommendation contradict the Usage ??

It is recommended that you:

Remove 'pytest-runner' from your setup_requires, preferably removing the setup_requires option.
Remove 'pytest' and any other testing requirements from tests_require, preferably removing the tests_requires option.
Select a tool to bootstrap and then run tests such as tox.

Usage

Add 'pytest-runner' to your 'setup_requires'. Pin to '>=2.0,<3dev' (or similar) to avoid pulling in incompatible versions.
Include 'pytest' and any other testing requirements to 'tests_require'.
Invoke tests with setup.py pytest.
Pass --index-url to have test requirements downloaded from an alternate index URL (unnecessary if specified for easy_install in setup.cfg).

Crash on Python 3.7

If pytest-runner is included in setup's setup_requires, things go wrong on Python 3.7:

$ python3.7 setup.py --version
/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/distutils/dist.py:261: UserWarning: Unknown distribution option: 'use_scm_version'
  warnings.warn(msg)

Installed /private/var/folders/cm/h_fnh5lx0b5b1b6wvnlc7qlw0000gn/T/easy_install-x0vcd5t0/pytest-runner-2.9/.eggs/setuptools_scm-0.0.0-py3.7.egg
Traceback (most recent call last):
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 157, in save_modules
    yield saved
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 248, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 278, in run
    return func()
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 246, in runner
    _execfile(setup_script, ns)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/var/folders/cm/h_fnh5lx0b5b1b6wvnlc7qlw0000gn/T/easy_install-x0vcd5t0/pytest-runner-2.9/setup.py", line 60, in <module>
    'courier = rinoh.fonts.adobe14:courier',
  File "/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/__init__.py", line 852, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'setuptools_scm>=1.9' distribution was not found and is required by the application

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 94, in <module>
    'Topic :: Text Processing :: Markup :: XML'
  File "/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/__init__.py", line 849, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1121, in best_match
    return self.obtain(req, installer)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1133, in obtain
    return installer(requirement)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 429, in fetch_build_egg
    return cmd.easy_install(req)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 665, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 695, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 876, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1115, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/command/easy_install.py", line 1101, in run_setup
    run_setup(setup_script, args)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 251, in run_setup
    raise
  File "/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/contextlib.py", line 100, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/contextlib.py", line 100, in __exit__
    self.gen.throw(type, value, traceback)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 169, in save_modules
    saved_exc.resume()
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 144, in resume
    six.reraise(type, exc, self._tb)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 157, in save_modules
    yield saved
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 198, in setup_context
    yield
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 248, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 278, in run
    return func()
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 246, in runner
    _execfile(setup_script, ns)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/var/folders/cm/h_fnh5lx0b5b1b6wvnlc7qlw0000gn/T/easy_install-x0vcd5t0/pytest-runner-2.9/setup.py", line 60, in <module>
    'courier = rinoh.fonts.adobe14:courier',
  File "/Users/brechtm/.pyenv/versions/3.7-dev/lib/python3.7/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/Users/brechtm/Documents/Code/rinohtype/.tox/py37/lib/python3.7/site-packages/pkg_resources/__init__.py", line 852, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'setuptools_scm>=1.9' distribution was not found and is required by the application

Please leave PYTHONPATH alone / provide a way to override your code that modifies it

PyTest.run() puts a lot of unwanted stuff into environment that's supposed to run tests by calling paths_on_pythonpath and project_on_sys_path. This is undesirable, in particular, because it creates an environment, where you have the source directory on sys.path before the actual directory, where the installed code resides. So, beside other things, it makes it impossible to test any package that has native modules, or any package that calls its own scripts in tests etc.

I have no idea why you needed this in the first place, but if there was some legal reason, please make this configurable (preferably, not the default).

Pytest run should honor verbose option to distutils

Currently, global -v options doesn't affect pytest output.

flake8 safety reports error

python 3.8 with flake8 , safety reports following error.

+============================+===========+==========================+==========+
| package | installed | affected | ID |
+============================+===========+==========================+==========+
| pytest-runner | 5.3.1 | >0 | 43313 |
+==============================================================================+

Fails to download from pypi.org

When I try to download superset it fails on resolving dependencies for pydruid package.

Error is below:
[root@tardis python36]# /root/pydev/py36-venv/bin/pip download --proxy=http://proxyuser:proxypass superset --dest=/tmp/python36
...
Collecting pydruid>=0.4.3 (from superset)
File was already downloaded /tmp/python36/pydruid-0.5.0.tar.gz
Complete output from command python setup.py egg_info:
Download error on https://pypi.org/simple/pytest-runner/: [Errno 101] Network is unreachable -- Some packages may not be found!
Couldn't find index page for 'pytest-runner' (maybe misspelled?)
Download error on https://pypi.org/simple/: [Errno 101] Network is unreachable -- Some packages may not be found!
No local packages or working download links found for pytest-runner
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-download-e3u9k_pm/pydruid/setup.py", line 44, in
include_package_data=True,
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/setuptools/init.py", line 142, in setup
_install_setup_requires(attrs)
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/setuptools/init.py", line 137, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/setuptools/dist.py", line 586, in fetch_build_eggs
replace_conflicting=True,
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/pkg_resources/init.py", line 780, in resolve
replace_conflicting=replace_conflicting
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/pkg_resources/init.py", line 1063, in best_match
return self.obtain(req, installer)
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/pkg_resources/init.py", line 1075, in obtain
return installer(requirement)
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/setuptools/dist.py", line 653, in fetch_build_egg
return cmd.easy_install(req)
File "/root/pydev/py36-venv/lib64/python3.6/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pytest-runner')

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-download-e3u9k_pm/pydruid/

I don't have problems downloading other packages.

Best,
Martin

README points to jaraco.collections which no longer seems to use pytest-runner

README.rst#L56 points to jaraco.collections as a real world example. But its setup.py does not use pytest-runner.

Is there another real world example?

test failure

Originally reported by: wiz (Bitbucket: wiz, GitHub: wiz)

---

When running the tests I see:

running pytest
running egg_info
writing pytest_runner.egg-info/PKG-INFO
writing top-level names to pytest_runner.egg-info/top_level.txt
writing entry points to pytest_runner.egg-info/entry_points.txt
writing dependency_links to pytest_runner.egg-info/dependency_links.txt
reading manifest file 'pytest_runner.egg-info/SOURCES.txt'
writing manifest file 'pytest_runner.egg-info/SOURCES.txt'
running build_ext
================================================================== test session starts ===================================================================
platform netbsd7 -- Python 3.4.3, pytest-2.8.2, py-1.4.30, pluggy-0.3.1
rootdir: /scratch/devel/py-test-runner/work/pytest-runner-2.6.2, inifile: pytest.ini
collected 0 items / 1 errors 

========================================================================= ERRORS =========================================================================
_____________________________________________________________ ERROR collecting docs/conf.py ______________________________________________________________
docs/conf.py:15: in <module>
    version = setuptools_scm.get_version(root='..')
/usr/pkg/lib/python3.4/site-packages/setuptools_scm/__init__.py:56: in get_version
    version = version_from_scm(root)
/usr/pkg/lib/python3.4/site-packages/setuptools_scm/__init__.py:27: in version_from_scm
    raise LookupError('no scm found for %r' % root)
E   LookupError: no scm found for '/scratch/devel/py-test-runner/work'
================================================================ 1 error in 0.02 seconds =================================================================
*** Error code 1

Using the tarball from pypi (MD5 (pytest-runner-2.6.2.tar.gz) = b81b698289addb13c2ed8d6ba6c8feb4).

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/9

no copyright/license included with source

Apparently it is MIT licensed, there is nothing I can see with the source to confirm this. Such as a copy of the MIT license for example.

6.0.0: pytest is failing in two units

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using 'installer` module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-runner-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-runner-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-runner-6.0.0
configfile: pytest.ini
plugins: shutil-1.7.0, virtualenv-1.7.0
collected 5 items

ptr/__init__.py .                                                        [ 20%]
tests/test_ptr.py FF..                                                   [100%]

=================================== FAILURES ===================================
________________________ test_egg_fetcher[setuptools-] _________________________

venv = <pytest_virtualenv.VirtualEnv object at 0x7f4b86b12490>
setuptools_req = 'setuptools', test_args = []

    @pytest.mark.xfail('platform.system() == "Windows"')
    @pytest.mark.parametrize(
        'setuptools_req, test_args', itertools.product(setuptools_reqs, args_variants)
    )
    def test_egg_fetcher(venv, setuptools_req, test_args):
        test_args = test_args.split()
        # Install pytest & pytest-runner.
        venv.run('pip install pytest .', cwd=os.getcwd())
        # Install setuptools version.
        venv.run('pip install -U'.split() + [setuptools_req])
        # For debugging purposes.
        venv.run('pip freeze --all')
        # Prepare fake index.
        index_dir = (venv.workspace / 'index').mkdir()
        for n in range(5):
            dist_name = 'barbazquux' + str(n + 1)
            dist_version = '0.1'
            dist_sdist = '%s-%s.tar.gz' % (dist_name, dist_version)
            dist_dir = (index_dir / dist_name).mkdir()
            make_sdist(
                dist_dir / dist_sdist,
                (
                    (
                        'setup.py',
                        textwrap.dedent(
                            '''
                    from setuptools import setup
                    setup(
                        name={dist_name!r},
                        version={dist_version!r},
                        py_modules=[{dist_name!r}],
                    )
                    '''
                        ).format(dist_name=dist_name, dist_version=dist_version),
                    ),
                    (dist_name + '.py', ''),
                ),
            )
            with (dist_dir / 'index.html').open('w') as fp:
                fp.write(
                    DALS(
                        '''
                    <!DOCTYPE html><html><body>
                    <a href="{dist_sdist}" rel="internal">{dist_sdist}</a><br/>
                    </body></html>
                    '''
                    ).format(dist_sdist=dist_sdist)
                )
        # Move barbazquux1 out of the index.
        shutil.move(index_dir / 'barbazquux1', venv.workspace)
        barbazquux1_link = (
            'file://'
            + str(venv.workspace.abspath())
            + '/barbazquux1/barbazquux1-0.1.tar.gz'
            + '#egg=barbazquux1-0.1'
        )
        # Prepare fake project.
        project_dir = (venv.workspace / 'project-0.1').mkdir()
        with open(project_dir / 'setup.py', 'w') as fp:
            fp.write(
                DALS(
                    '''
                from setuptools import setup
                setup(
                    name='project',
                    version='0.1',
                    dependency_links = [
                        {barbazquux1_link!r},
                    ],
                    setup_requires=[
                        'pytest-runner',
                    ],
                    install_requires=[
                        'barbazquux1',
                    ],
                    tests_require=[
                        'pytest',
                        'barbazquux2',
                    ],
                    extras_require={{
                        ':"{sys_platform}" in sys_platform': 'barbazquux3',
                        ':"barbazquux" in sys_platform': 'barbazquux4',
                        'extra': 'barbazquux5',
                    }}
                )
                '''
                ).format(sys_platform=sys.platform, barbazquux1_link=barbazquux1_link)
            )
        with open(project_dir / 'setup.cfg', 'w') as fp:
            fp.write(
                DALS(
                    '''
                [easy_install]
                index_url = .
                '''
                )
            )
        with open(project_dir / 'test_stuff.py', 'w') as fp:
            fp.write(
                DALS(
                    '''
                import pytest

                def test_stuff():
                    import barbazquux1
                    import barbazquux2
                    import barbazquux3
                    with pytest.raises(ImportError):
                        import barbazquux4
                    if {importable_barbazquux5}:
                        import barbazquux5
                    else:
                        with pytest.raises(ImportError):
                            import barbazquux5
                '''
                ).format(importable_barbazquux5=('--extras' in test_args))
            )
        # Run fake project tests.
        cmd = 'python setup.py pytest'.split()
        cmd += ['--index-url=' + index_dir.abspath()]
        cmd += test_args
>       venv.run(cmd, cwd=project_dir)

tests/test_ptr.py:170:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/pytest_virtualenv.py:151: in run
    return super(VirtualEnv, self).run(args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pytest_virtualenv.VirtualEnv object at 0x7f4b86b12490>
cmd = ['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpieav6gre/index']
capture = False, check_rc = True, cd = Path('/tmp/tmpieav6gre'), shell = False
kwargs = {'cwd': Path('/tmp/tmpieav6gre/project-0.1'), 'env': {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-O2 -g -grecord-gcc-switch...es -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none', ...}}
p = <subprocess.Popen object at 0x7f4b86ad2520>, out = None, _ = None
err = CalledProcessError(1, ['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpieav6gre/index'])

    def run(self, cmd, capture=False, check_rc=True, cd=None, shell=False, **kwargs):
        """
        Run a command relative to a given directory, defaulting to the workspace root

        Parameters
        ----------
        cmd : `str` or `list`
            Command string or list. Commands given as a string will be run in a subshell.
        capture : `bool`
            Capture and return output
        check_rc : `bool`
            Assert return code is zero
        cd : `str`
            Path to chdir to, defaults to workspace root
        """
        if isinstance(cmd, string_types):
            shell = True
        else:
            # Some of the command components might be path objects or numbers
            cmd = [str(i) for i in cmd]

        if not cd:
            cd = self.workspace

        with cmdline.chdir(cd):
            log.debug("run: {0}".format(cmd))
            if capture:
                p = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kwargs)
            else:
                p = subprocess.Popen(cmd, shell=shell, **kwargs)
            (out, _) = p.communicate()

            if out is not None and not isinstance(out, string_types):
                out = out.decode('utf-8')

            if self.debug and capture:
                log.debug("Stdout/stderr:")
                log.debug(out)

            if check_rc and p.returncode != 0:
                err = subprocess.CalledProcessError(p.returncode, cmd)
                err.output = out
                if capture and not self.debug:
                    log.error("Stdout/stderr:")
                    log.error(out)
>               raise err
E               subprocess.CalledProcessError: Command '['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpieav6gre/index']' returned non-zero exit status 1.

/usr/lib/python3.8/site-packages/pytest_shutil/workspace.py:132: CalledProcessError
---------------------------- Captured stdout setup -----------------------------
created virtual environment CPython3.8.16.final.0-64 in 214ms
  creator CPython3Posix(dest=/tmp/tmpieav6gre/.env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/tkloczko/.local/share/virtualenv)
    added seed packages: pip==23.0.1, setuptools==67.6.1, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
----------------------------- Captured stdout call -----------------------------
Processing /home/tkloczko/rpmbuild/BUILD/pytest-runner-6.0.0
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pytest
  Using cached pytest-7.3.1-py3-none-any.whl (320 kB)
Collecting iniconfig
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting packaging
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting exceptiongroup>=1.0.0rc8
  Using cached exceptiongroup-1.1.1-py3-none-any.whl (14 kB)
Collecting pluggy<2.0,>=0.12
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting tomli>=1.0.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Building wheels for collected packages: pytest-runner
  Building wheel for pytest-runner (pyproject.toml): started
  Building wheel for pytest-runner (pyproject.toml): finished with status 'done'
  Created wheel for pytest-runner: filename=pytest_runner-6.0.0-py3-none-any.whl size=7196 sha256=65556ab1621181fe7d5e06aa40dc2d44952ad4f9bc134a4ba23530bb2129f021
  Stored in directory: /home/tkloczko/.cache/pip/wheels/20/75/81/4dac296b481f05df33e0675a9487292faad54899b9e83be8c8
Successfully built pytest-runner
Installing collected packages: tomli, pytest-runner, pluggy, packaging, iniconfig, exceptiongroup, pytest
Successfully installed exceptiongroup-1.1.1 iniconfig-2.0.0 packaging-23.1 pluggy-1.0.0 pytest-7.3.1 pytest-runner-6.0.0 tomli-2.0.1
Requirement already satisfied: setuptools in ./.env/lib/python3.8/site-packages (67.6.1)
Collecting setuptools
  Using cached setuptools-67.7.2-py3-none-any.whl (1.1 MB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 67.6.1
    Uninstalling setuptools-67.6.1:
      Successfully uninstalled setuptools-67.6.1
Successfully installed setuptools-67.7.2
exceptiongroup==1.1.1
iniconfig==2.0.0
packaging==23.1
pip==23.0.1
pluggy==1.0.0
pytest==7.3.1
pytest-runner @ file:///home/tkloczko/rpmbuild/BUILD/pytest-runner-6.0.0
setuptools==67.7.2
tomli==2.0.1
wheel==0.40.0
running pytest
----------------------------- Captured stderr call -----------------------------

[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip

[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip
ERROR: Could not find a version that satisfies the requirement barbazquux2 (from versions: none)
ERROR: No matching distribution found for barbazquux2
/tmp/tmpieav6gre/.env/lib/python3.8/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.fetch_build_eggs(dist.setup_requires)
/tmp/tmpieav6gre/.env/lib/python3.8/site-packages/setuptools/command/test.py:194: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  ir_d = dist.fetch_build_eggs(dist.install_requires)
/tmp/tmpieav6gre/.env/lib/python3.8/site-packages/setuptools/command/test.py:195: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  tr_d = dist.fetch_build_eggs(dist.tests_require or [])
error: Command '['/tmp/tmpieav6gre/.env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpow8yjmh1', '--quiet', '--find-links', 'file:///tmp/tmpieav6gre/barbazquux1/barbazquux1-0.1.tar.gz#egg=barbazquux1-0.1', 'barbazquux2']' returned non-zero exit status 1.
____________________ test_egg_fetcher[setuptools---extras] _____________________

venv = <pytest_virtualenv.VirtualEnv object at 0x7f4b86a2bee0>
setuptools_req = 'setuptools', test_args = ['--extras']

    @pytest.mark.xfail('platform.system() == "Windows"')
    @pytest.mark.parametrize(
        'setuptools_req, test_args', itertools.product(setuptools_reqs, args_variants)
    )
    def test_egg_fetcher(venv, setuptools_req, test_args):
        test_args = test_args.split()
        # Install pytest & pytest-runner.
        venv.run('pip install pytest .', cwd=os.getcwd())
        # Install setuptools version.
        venv.run('pip install -U'.split() + [setuptools_req])
        # For debugging purposes.
        venv.run('pip freeze --all')
        # Prepare fake index.
        index_dir = (venv.workspace / 'index').mkdir()
        for n in range(5):
            dist_name = 'barbazquux' + str(n + 1)
            dist_version = '0.1'
            dist_sdist = '%s-%s.tar.gz' % (dist_name, dist_version)
            dist_dir = (index_dir / dist_name).mkdir()
            make_sdist(
                dist_dir / dist_sdist,
                (
                    (
                        'setup.py',
                        textwrap.dedent(
                            '''
                    from setuptools import setup
                    setup(
                        name={dist_name!r},
                        version={dist_version!r},
                        py_modules=[{dist_name!r}],
                    )
                    '''
                        ).format(dist_name=dist_name, dist_version=dist_version),
                    ),
                    (dist_name + '.py', ''),
                ),
            )
            with (dist_dir / 'index.html').open('w') as fp:
                fp.write(
                    DALS(
                        '''
                    <!DOCTYPE html><html><body>
                    <a href="{dist_sdist}" rel="internal">{dist_sdist}</a><br/>
                    </body></html>
                    '''
                    ).format(dist_sdist=dist_sdist)
                )
        # Move barbazquux1 out of the index.
        shutil.move(index_dir / 'barbazquux1', venv.workspace)
        barbazquux1_link = (
            'file://'
            + str(venv.workspace.abspath())
            + '/barbazquux1/barbazquux1-0.1.tar.gz'
            + '#egg=barbazquux1-0.1'
        )
        # Prepare fake project.
        project_dir = (venv.workspace / 'project-0.1').mkdir()
        with open(project_dir / 'setup.py', 'w') as fp:
            fp.write(
                DALS(
                    '''
                from setuptools import setup
                setup(
                    name='project',
                    version='0.1',
                    dependency_links = [
                        {barbazquux1_link!r},
                    ],
                    setup_requires=[
                        'pytest-runner',
                    ],
                    install_requires=[
                        'barbazquux1',
                    ],
                    tests_require=[
                        'pytest',
                        'barbazquux2',
                    ],
                    extras_require={{
                        ':"{sys_platform}" in sys_platform': 'barbazquux3',
                        ':"barbazquux" in sys_platform': 'barbazquux4',
                        'extra': 'barbazquux5',
                    }}
                )
                '''
                ).format(sys_platform=sys.platform, barbazquux1_link=barbazquux1_link)
            )
        with open(project_dir / 'setup.cfg', 'w') as fp:
            fp.write(
                DALS(
                    '''
                [easy_install]
                index_url = .
                '''
                )
            )
        with open(project_dir / 'test_stuff.py', 'w') as fp:
            fp.write(
                DALS(
                    '''
                import pytest

                def test_stuff():
                    import barbazquux1
                    import barbazquux2
                    import barbazquux3
                    with pytest.raises(ImportError):
                        import barbazquux4
                    if {importable_barbazquux5}:
                        import barbazquux5
                    else:
                        with pytest.raises(ImportError):
                            import barbazquux5
                '''
                ).format(importable_barbazquux5=('--extras' in test_args))
            )
        # Run fake project tests.
        cmd = 'python setup.py pytest'.split()
        cmd += ['--index-url=' + index_dir.abspath()]
        cmd += test_args
>       venv.run(cmd, cwd=project_dir)

tests/test_ptr.py:170:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.8/site-packages/pytest_virtualenv.py:151: in run
    return super(VirtualEnv, self).run(args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <pytest_virtualenv.VirtualEnv object at 0x7f4b86a2bee0>
cmd = ['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpaeexcplr/index', '--extras']
capture = False, check_rc = True, cd = Path('/tmp/tmpaeexcplr'), shell = False
kwargs = {'cwd': Path('/tmp/tmpaeexcplr/project-0.1'), 'env': {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-O2 -g -grecord-gcc-switch...es -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none', ...}}
p = <subprocess.Popen object at 0x7f4b86a2b430>, out = None, _ = None
err = CalledProcessError(1, ['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpaeexcplr/index', '--extras'])

    def run(self, cmd, capture=False, check_rc=True, cd=None, shell=False, **kwargs):
        """
        Run a command relative to a given directory, defaulting to the workspace root

        Parameters
        ----------
        cmd : `str` or `list`
            Command string or list. Commands given as a string will be run in a subshell.
        capture : `bool`
            Capture and return output
        check_rc : `bool`
            Assert return code is zero
        cd : `str`
            Path to chdir to, defaults to workspace root
        """
        if isinstance(cmd, string_types):
            shell = True
        else:
            # Some of the command components might be path objects or numbers
            cmd = [str(i) for i in cmd]

        if not cd:
            cd = self.workspace

        with cmdline.chdir(cd):
            log.debug("run: {0}".format(cmd))
            if capture:
                p = subprocess.Popen(cmd, shell=shell, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kwargs)
            else:
                p = subprocess.Popen(cmd, shell=shell, **kwargs)
            (out, _) = p.communicate()

            if out is not None and not isinstance(out, string_types):
                out = out.decode('utf-8')

            if self.debug and capture:
                log.debug("Stdout/stderr:")
                log.debug(out)

            if check_rc and p.returncode != 0:
                err = subprocess.CalledProcessError(p.returncode, cmd)
                err.output = out
                if capture and not self.debug:
                    log.error("Stdout/stderr:")
                    log.error(out)
>               raise err
E               subprocess.CalledProcessError: Command '['python', 'setup.py', 'pytest', '--index-url=/tmp/tmpaeexcplr/index', '--extras']' returned non-zero exit status 1.

/usr/lib/python3.8/site-packages/pytest_shutil/workspace.py:132: CalledProcessError
---------------------------- Captured stdout setup -----------------------------
created virtual environment CPython3.8.16.final.0-64 in 237ms
  creator CPython3Posix(dest=/tmp/tmpaeexcplr/.env, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/tkloczko/.local/share/virtualenv)
    added seed packages: pip==23.0.1, setuptools==67.6.1, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
----------------------------- Captured stdout call -----------------------------
Processing /home/tkloczko/rpmbuild/BUILD/pytest-runner-6.0.0
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pytest
  Using cached pytest-7.3.1-py3-none-any.whl (320 kB)
Collecting iniconfig
  Using cached iniconfig-2.0.0-py3-none-any.whl (5.9 kB)
Collecting tomli>=1.0.0
  Using cached tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting packaging
  Using cached packaging-23.1-py3-none-any.whl (48 kB)
Collecting pluggy<2.0,>=0.12
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting exceptiongroup>=1.0.0rc8
  Using cached exceptiongroup-1.1.1-py3-none-any.whl (14 kB)
Building wheels for collected packages: pytest-runner
  Building wheel for pytest-runner (pyproject.toml): started
  Building wheel for pytest-runner (pyproject.toml): finished with status 'done'
  Created wheel for pytest-runner: filename=pytest_runner-6.0.0-py3-none-any.whl size=7196 sha256=efecc567bbfc38971325ca49e92486f725c820dcdb7766637312948b1da7ca27
  Stored in directory: /home/tkloczko/.cache/pip/wheels/20/75/81/4dac296b481f05df33e0675a9487292faad54899b9e83be8c8
Successfully built pytest-runner
Installing collected packages: tomli, pytest-runner, pluggy, packaging, iniconfig, exceptiongroup, pytest
Successfully installed exceptiongroup-1.1.1 iniconfig-2.0.0 packaging-23.1 pluggy-1.0.0 pytest-7.3.1 pytest-runner-6.0.0 tomli-2.0.1
Requirement already satisfied: setuptools in ./.env/lib/python3.8/site-packages (67.6.1)
Collecting setuptools
  Using cached setuptools-67.7.2-py3-none-any.whl (1.1 MB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 67.6.1
    Uninstalling setuptools-67.6.1:
      Successfully uninstalled setuptools-67.6.1
Successfully installed setuptools-67.7.2
exceptiongroup==1.1.1
iniconfig==2.0.0
packaging==23.1
pip==23.0.1
pluggy==1.0.0
pytest==7.3.1
pytest-runner @ file:///home/tkloczko/rpmbuild/BUILD/pytest-runner-6.0.0
setuptools==67.7.2
tomli==2.0.1
wheel==0.40.0
running pytest
----------------------------- Captured stderr call -----------------------------

[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip

[notice] A new release of pip is available: 23.0.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip
ERROR: Could not find a version that satisfies the requirement barbazquux2 (from versions: none)
ERROR: No matching distribution found for barbazquux2
/tmp/tmpaeexcplr/.env/lib/python3.8/site-packages/setuptools/__init__.py:84: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  dist.fetch_build_eggs(dist.setup_requires)
/tmp/tmpaeexcplr/.env/lib/python3.8/site-packages/setuptools/command/test.py:194: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  ir_d = dist.fetch_build_eggs(dist.install_requires)
/tmp/tmpaeexcplr/.env/lib/python3.8/site-packages/setuptools/command/test.py:195: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!

        ********************************************************************************
        Requirements should be satisfied by a PEP 517 installer.
        If you are using pip, you can try `pip install --use-pep517`.
        ********************************************************************************

!!
  tr_d = dist.fetch_build_eggs(dist.tests_require or [])
error: Command '['/tmp/tmpaeexcplr/.env/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpqh_fcder', '--quiet', '--find-links', 'file:///tmp/tmpaeexcplr/barbazquux1/barbazquux1-0.1.tar.gz#egg=barbazquux1-0.1', 'barbazquux2']' returned non-zero exit status 1.
=========================== short test summary info ============================
FAILED tests/test_ptr.py::test_egg_fetcher[setuptools-] - subprocess.CalledPr...
FAILED tests/test_ptr.py::test_egg_fetcher[setuptools---extras] - subprocess....
========================= 2 failed, 3 passed in 48.63s =========================

Here is list of installed modules in build env

Package                       Version
----------------------------- -----------------
alabaster                     0.7.13
autocommand                   2.2.1
Babel                         2.12.1
build                         0.10.0
cffi                          1.15.1
charset-normalizer            3.1.0
cryptography                  39.0.2
distlib                       0.3.6
distro                        1.8.0
docutils                      0.19
exceptiongroup                1.0.0
execnet                       1.9.0
filelock                      3.9.0
gpg                           1.19.0
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.6.0
importlib-resources           5.12.0
iniconfig                     2.0.0
installer                     0.7.0
jaraco.classes                3.2.3
jaraco.packaging              9.1.2
jaraco.tidelift               1.5.1
jeepney                       0.8.0
Jinja2                        3.1.2
keyring                       23.13.1
libcomps                      0.1.19
MarkupSafe                    2.1.2
mock                          4.0.3
more-itertools                9.1.0
packaging                     23.1
path                          16.6.0
platformdirs                  3.1.1
pluggy                        1.0.0
ply                           3.11
pycparser                     2.21
Pygments                      2.15.1
pyproject_hooks               1.0.0
pytest                        7.3.1
pytest-fixture-config         1.7.0
pytest-shutil                 1.7.0
pytest-virtualenv             1.7.0
python-dateutil               2.8.2
pytz                          2023.2
requests                      2.28.2
requests-toolbelt             0.10.1
rst.linker                    2.4.0
SecretStorage                 3.3.3
setuptools                    67.7.2
setuptools-scm                7.1.0
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        6.2.1
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2.dev20230415
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1.dev20230415
sphinxcontrib-qthelp          1.0.3.dev20230415
sphinxcontrib-serializinghtml 1.1.5
termcolor                     2.3.0
tomli                         2.0.1
typing_extensions             4.5.0
urllib3                       1.26.15
virtualenv                    20.21.0
wheel                         0.40.0
zipp                          3.15.0

How to run multiple isolated tests?

I've got multiple test folders - 'tests', 'tests_integration', 'tests_integration2',... Integration tests use different betamax/session configurations. And, they collide, if they are run together with: python -m pytest

I need to run each test folder individually:

• python -m pytest tests
• python -m pytest tests_integration
• python -m pytest tests_integration2

How can I achieve this with pytest-runner?

Ignore files with testpaths

Hello!

subject

I run python setup.py pytest or python setup.py test (does not matter which) and testpaths option from setup.cfg doesn't work. As a consequence, I am getting testing results for all subdirectories (but I need test-results just for one dir, described further).

So:

• python setup.py pytest runs ALL tests (even excluded)
• python libs/pytest.py runs required set of tests (in memorybuffer folder)

I tried to use python_files option. But it doesn't change anything.

details

I have the following file structure:

/project
    + setup.py
    + setup.cfg
    /memorybuffer
        + __init__.py
        + file.py
        + file_test.py
    /libs
        + pytest.py
        + pep8.py
        /pyflakes
            # 
            # PYFLAKES FILES, with tests
            # 
        /_pytest
            # PYTEST files

The content in libs folder is generated during installation: pip install -r requirements-dev.txt -t libs

Here is my setup.cfg:

[aliases]
test=pytest

[tool:pytest]
testpaths=./memorybuffer/
#python_files = memorybuffer/*/*.py

[tool:test]
testpaths=./memorybuffer/
#python_files = memorybuffer/*/*.py

My setup.cfg:

from setuptools import setup

setup(
    setup_requires=['pytest-runner', 'setuptools-pep8'],
    tests_require=['pytest']
)

environment

• Python 2.7.6
• pip 8.1.2
• pytest version 3.0.0

5.3.1: sphinx 4.0.3 warning

+ /usr/bin/python3 setup.py build_sphinx -b man --build-dir build/sphinx
running build_sphinx
Running Sphinx v4.0.3
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 2 added, 0 changed, 0 removed
reading sources... [100%] index
CHANGES (links).rst:97: WARNING: Unknown target name: "840ff4c < https://github.com/pytest-dev/pytest-runner/commit/840ff4c2bf6c752d9770f0dd8d64a841060cf9bc>".
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-pytest-runner.3 { history } done
build succeeded, 1 warning.

setup.py uses obsolete setup_requires

setup_requires is broken for off-line builds. Recommended alternative is PEP 518:

https://www.python.org/dev/peps/pep-0518/

tests_require support

Originally reported by: Ronny Pfannschmidt (Bitbucket: RonnyPfannschmidt, GitHub: RonnyPfannschmidt)

---

the setuptools test command does support the tests_require
keyword to list test-only packages

those only get on setup.py test

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/4

tox-runner

tox has a similar setuptools command. Making a similar tox-runner package would probably duplicate a lot of what's in this project.

Do you think there would be a way to incorporate running tox with python setup.py test in this project? Or should this be done as a separate project despite the possible overlap?

5.2: test suite is failing

+ cd pytest-runner-5.2
+ tox --skip-missing-interpreters
.tox create: /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2/.tox/.tox
.tox installdeps: tox-pip-version>=0.0.6, tox-venv, tox >= 3.2
python create: /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2/.tox/python
python: pip_version is pip
python installdeps: setuptools>=31.0.1
python develop-inst: /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2
python installed: apipkg==1.5,appdirs==1.4.4,attrs==20.3.0,black==20.8b1,click==7.1.2,contextlib2==0.6.0.post1,coverage==5.3.1,distlib==0.3.1,docutils==0.16,execnet==1.7.1,filelock==3.0.12,flake8==3.8.4,iniconfig==1.1.1,mccabe==0.6.1,mock==4.0.3,more-itertools==8.6.0,mypy-extensions==0.4.3,packaging==20.8,path==15.0.1,path.py==12.5.0,pathspec==0.8.1,pluggy==0.13.1,py==1.10.0,pycodestyle==2.6.0,pyflakes==2.2.0,pyparsing==2.4.7,pytest==6.2.1,pytest-black==0.3.12,pytest-black-multipy==1.0.0,pytest-checkdocs==2.2.0,pytest-cov==2.10.1,pytest-fixture-config==1.7.0,pytest-flake8==1.0.7,# Editable install with no version control (pytest-runner==5.2),-e /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2,pytest-shutil==1.7.0,pytest-virtualenv==1.7.0,regex==2020.11.13,six==1.15.0,termcolor==1.1.0,toml==0.10.2,typed-ast==1.4.2,typing-extensions==3.7.4.3,virtualenv==20.2.2
python run-test-pre: PYTHONHASHSEED='270599559'
python run-test: commands[0] | pytest
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.3, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
cachedir: .tox/python/.pytest_cache
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2, configfile: pytest.ini
plugins: shutil-1.7.0, black-0.3.12, virtualenv-1.7.0, flake8-1.0.7, cov-2.10.1, checkdocs-2.2.0, black-multipy-1.0.0
collected 14 items

ptr.py .F.                                                                                                                                                           [ 21%]
setup.py ...                                                                                                                                                         [ 42%]
docs/conf.py ..                                                                                                                                                      [ 57%]
tests/test_ptr.py ......                                                                                                                                             [100%]

================================================================================= FAILURES =================================================================================
____________________________________________________________________________ Black format check ____________________________________________________________________________
--- /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2/ptr.py      2019-10-26 22:07:01 +0000
+++ /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2/ptr.py      2021-01-02 17:00:33.900912 +0000
@@ -36,12 +36,12 @@

     allow_hosts = None
     index_url = None

     def fetch_build_egg(self, req):
-        """ Specialized version of Distribution.fetch_build_egg
-        that respects respects allow_hosts and index_url. """
+        """Specialized version of Distribution.fetch_build_egg
+        that respects respects allow_hosts and index_url."""
         from setuptools.command.easy_install import easy_install

         dist = Distribution({'script_args': ['easy_install']})
         dist.parse_config_files()
         opts = dist.get_option_dict('easy_install')


----------- coverage: platform linux, python 3.8.3-final-0 -----------
Name                                                                                                     Stmts   Miss  Cover   Missing
--------------------------------------------------------------------------------------------------------------------------------------
.tox/python/lib/python3.8/site-packages/appdirs.py                                                         257    205    20%   29-39, 78-86, 88-90, 96, 131-163, 196, 202, 236-254, 290-311, 345-353, 388-404, 411-415, 419, 424, 429, 434, 439, 444, 449, 460-476, 480-503, 507-530, 533-556, 559-571, 577-608
.tox/python/lib/python3.8/site-packages/distlib/__init__.py                                                  8      0   100%
.tox/python/lib/python3.8/site-packages/distlib/compat.py                                                   39      3    92%   474-475, 477
.tox/python/lib/python3.8/site-packages/distlib/resources.py                                               225    154    32%   30-33, 44, 53-69, 74-75, 93, 98-100, 104, 108, 116, 125, 130-132, 135, 140-147, 150, 153, 156-165, 168, 171-172, 175, 178-181, 184, 189-206, 214-222, 225, 228-243, 246-248, 251, 254, 257-258, 261-273, 276-284, 295-296, 300-301, 305, 316-332, 345-355
.tox/python/lib/python3.8/site-packages/distlib/scripts.py                                                 164     65    60%   57-63, 100, 139, 144, 153-155, 163-164, 174, 191, 214-215, 230-231, 237, 278-279, 288-291, 304, 310-359, 363, 367, 405, 416-419
.tox/python/lib/python3.8/site-packages/distlib/util.py                                                   1167    937    20%   65-142, 150-262, 269-289, 293-299, 312-315, 319-331, 335-341, 344-387, 391-406, 411-415, 419-424, 429-434, 444-449, 461-475, 491, 505-511, 516-528, 531-543, 549, 555, 563, 574-581, 584-598, 601-619, 622-631, 638-641, 644-659, 662-673, 685, 692-699, 712-714, 722, 725, 731, 735, 755-778, 791-795, 799-801, 805-816, 820-822, 825-832, 845-861, 876-880, 883-902, 908-925, 930-933, 936-938, 958-960, 966, 972-982, 990, 1002-1010, 1019-1022, 1029, 1042-1052, 1059-1061, 1064, 1067-1080, 1083-1085, 1088-1096, 1101, 1105-1127, 1132-1178, 1182-1190, 1201-1250, 1255-1264, 1277-1282, 1285-1292, 1295-1296, 1299-1300, 1303-1305, 1309, 1313-1320, 1323-1329, 1333-1348, 1352-1360, 1373-1379, 1383-1407, 1424-1455, 1463-1465, 1477-1481, 1484-1491, 1504, 1514-1526, 1531-1532, 1535-1543, 1548-1549, 1552-1564, 1569-1581, 1589-1596, 1607, 1610, 1615-1623, 1626, 1629-1634, 1640-1641, 1644-1651, 1663-1664, 1667-1695, 1698-1701, 1705-1709, 1717-1718, 1725-1739, 1742-1755, 1761
.tox/python/lib/python3.8/site-packages/filelock.py                                                        179     64    64%   39-40, 49-50, 57-58, 95-96, 99-101, 122, 125-126, 197, 203, 276-290, 345-358, 361-372, 388-389, 414-421, 424-432, 444, 448-451
.tox/python/lib/python3.8/site-packages/pytest_cov/embed.py                                                 77     66    14%   16-39, 53-79, 82-90, 97-137
.tox/python/lib/python3.8/site-packages/six.py                                                             493    239    52%   49-72, 98-99, 112, 118-121, 131-133, 145, 154-157, 192-193, 203, 222-223, 308, 488, 496, 501-507, 519-525, 530-532, 538-540, 545, 550, 554-568, 583, 586, 589, 592, 600-616, 628, 631, 645-647, 653-673, 679, 683, 687, 691, 698-721, 737-738, 743-795, 797-804, 814-834, 845-861, 870-873, 893-897, 914-920, 935, 939, 950-957, 978-979
.tox/python/lib/python3.8/site-packages/virtualenv/__init__.py                                               4      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/__main__.py                                              48     14    71%   18-22, 63-73
.tox/python/lib/python3.8/site-packages/virtualenv/activation/__init__.py                                    9      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/activation/activator.py                                  15      1    93%   44
.tox/python/lib/python3.8/site-packages/virtualenv/activation/bash/__init__.py                               8      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/activation/batch/__init__.py                             15      5    67%   16-18, 22-23
.tox/python/lib/python3.8/site-packages/virtualenv/activation/cshell/__init__.py                             9      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/activation/fish/__init__.py                               6      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/activation/powershell/__init__.py                         6      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/activation/python/__init__.py                            23      2    91%   33-34
.tox/python/lib/python3.8/site-packages/virtualenv/activation/via_template.py                               44      3    93%   16, 23, 30
.tox/python/lib/python3.8/site-packages/virtualenv/activation/xonsh/__init__.py                              9      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/__init__.py                                     33     10    70%   20, 28, 35, 38-42, 47-48
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/base.py                                         64     16    75%   28, 32, 36, 40, 44, 48, 52, 58-59, 66, 71, 78, 82, 86, 90, 95
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/na.py                                           36     10    72%   15, 26, 29, 32, 37, 41, 44, 52, 56, 66
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/read_only.py                                    21      8    62%   12-14, 17, 20, 23, 26, 31
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/via_disk_folder.py                             111     33    70%   56, 62-63, 70-72, 76-84, 95-101, 137-146, 149-150
.tox/python/lib/python3.8/site-packages/virtualenv/app_data/via_tempdir.py                                  17      5    71%   16-17, 23-24, 27
.tox/python/lib/python3.8/site-packages/virtualenv/config/__init__.py                                        1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/config/cli/__init__.py                                    1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/config/cli/parser.py                                     91     17    81%   21, 35, 39, 60, 80, 82-85, 87, 100, 115-120
.tox/python/lib/python3.8/site-packages/virtualenv/config/convert.py                                        52     27    48%   13, 16, 32-34, 39-41, 49-55, 63-74, 79-83
.tox/python/lib/python3.8/site-packages/virtualenv/config/env_var.py                                        15      7    53%   19-26
.tox/python/lib/python3.8/site-packages/virtualenv/config/ini.py                                            58     25    57%   37-38, 41-47, 49, 52-54, 57-69
.tox/python/lib/python3.8/site-packages/virtualenv/create/__init__.py                                        1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/creator.py                                       131     37    72%   72, 106, 113-114, 128-130, 132, 140, 147, 150, 159, 168-169, 206-208, 212, 216-238
.tox/python/lib/python3.8/site-packages/virtualenv/create/describe.py                                       83      5    94%   55-57, 86, 89
.tox/python/lib/python3.8/site-packages/virtualenv/create/pyenv_cfg.py                                      47     14    70%   25-31, 44-45, 51, 54, 57-58, 61
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/__init__.py                         0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/api.py                             73      5    93%   22, 50, 54-55, 69
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/__init__.py                 0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/builtin_way.py             10      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/__init__.py         1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/common.py          43      9    79%   48-53, 57, 62-64
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/cpython2.py        67     33    51%   21-26, 30, 34, 39, 43, 48-54, 63-70, 82-86, 94-102
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/cpython3.py        55     20    64%   29, 50-52, 56-60, 64, 68-72, 76-81, 84
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/cpython/mac_os.py         184    140    24%   29-34, 37-49, 53-58, 62, 66, 72, 75, 79-83, 87-103, 109, 112, 116-131, 158-168, 172-298
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/pypy/__init__.py            0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/pypy/common.py             36     15    58%   21-24, 28, 37-40, 45-49, 53
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/pypy/pypy2.py              77     32    58%   23, 27-32, 36, 40, 44, 49, 61, 64-71, 79, 87, 91, 95, 99-103, 111, 115, 119-121
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/pypy/pypy3.py              39     13    67%   18, 26, 30, 38, 41, 45-50, 59, 63
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/python2/__init__.py         0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/python2/python2.py         74     41    45%   25-51, 55, 59, 63-77, 81-86, 90-93, 96, 99, 103, 107, 111
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/ref.py                    127     30    76%   45-46, 52, 61-64, 71, 78-83, 87, 91, 93, 109, 120, 132-138, 148, 159, 163, 167, 169, 172
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/builtin/via_global_self_do.py      81     13    84%   42, 44-52, 54, 59, 76, 100
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/store.py                           10      5    50%   7-9, 13-14
.tox/python/lib/python3.8/site-packages/virtualenv/create/via_global_ref/venv.py                            60     36    40%   17-22, 25, 32, 34, 37-43, 46-54, 57-61, 64-69, 73-75, 78-83
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/__init__.py                                     1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/builtin.py                                    116     68    41%   40, 43, 46-47, 57, 62, 70-72, 75-101, 105-115, 120-121, 124, 127-139, 143-151, 156-159
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/cached_py_info.py                              96     49    49%   30-34, 43, 57-58, 60, 69, 71-76, 81-113, 118-119, 122-130, 133-136, 140-141
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/discover.py                                    20      2    90%   18, 36
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/py_info.py                                    326    168    48%   45, 70-71, 97, 115, 119, 136, 144-145, 149, 153, 158, 166-170, 174-183, 194-197, 200, 206-235, 244-247, 254-275, 286-288, 296-298, 302, 305-308, 321-325, 331-332, 346-358, 360, 367-389, 392-411, 417-434, 437-452, 455-463, 466-484, 490
.tox/python/lib/python3.8/site-packages/virtualenv/discovery/py_spec.py                                     90     60    33%   34-67, 72-91, 95, 99-109, 112, 122
.tox/python/lib/python3.8/site-packages/virtualenv/info.py                                                  40      9    78%   39-47
.tox/python/lib/python3.8/site-packages/virtualenv/report.py                                                34      1    97%   50
.tox/python/lib/python3.8/site-packages/virtualenv/run/__init__.py                                          71      5    93%   67, 82, 87-88, 118
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/__init__.py                                    0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/activators.py                                 28      1    96%   34
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/base.py                                       44      3    93%   9, 43, 48
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/creators.py                                   50      7    86%   24, 28, 39-43, 76
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/discovery.py                                  15      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/run/plugin/seeders.py                                    16      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/run/session.py                                           58      4    93%   23, 28, 88, 91
.tox/python/lib/python3.8/site-packages/virtualenv/seed/__init__.py                                          1      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/__init__.py                                    0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/base_embed.py                                 53      3    94%   35, 106, 110
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/pip_invoke.py                                 40     26    35%   15, 18-23, 27-32, 36-56
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/__init__.py                       0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/pip_install/__init__.py           0      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/pip_install/base.py             121      7    94%   30, 38-41, 108, 139-140
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/pip_install/copy.py              26      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/pip_install/symlink.py           44     30    32%   16-18, 22-45, 48-51, 54-56, 59-61
.tox/python/lib/python3.8/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py                 101     27    73%   41, 58-59, 67-71, 95-97, 99-112, 126, 132-133
.tox/python/lib/python3.8/site-packages/virtualenv/seed/seeder.py                                           13      2    85%   30, 39
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/__init__.py                                   4      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/acquire.py                                   59     15    75%   27, 64, 80, 84-96, 118
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/bundle.py                                    31      7    77%   23-26, 38, 47-48
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/__init__.py                            10      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/periodic_update.py                          209     39    81%   33-36, 50, 115, 120, 137, 162, 192, 202, 241, 253, 269-271, 277, 297-300, 305-315, 319-354
.tox/python/lib/python3.8/site-packages/virtualenv/seed/wheels/util.py                                      81     32    60%   20, 40-41, 43, 51-75, 85-92, 111, 115-116
.tox/python/lib/python3.8/site-packages/virtualenv/util/__init__.py                                          6      1    83%   8
.tox/python/lib/python3.8/site-packages/virtualenv/util/error.py                                             8      5    38%   9-13
.tox/python/lib/python3.8/site-packages/virtualenv/util/lock.py                                            122     22    82%   20-23, 52, 62, 66, 71, 76, 103-104, 107, 119-124, 150, 153, 157, 161
.tox/python/lib/python3.8/site-packages/virtualenv/util/path/__init__.py                                     5      0   100%
.tox/python/lib/python3.8/site-packages/virtualenv/util/path/_pathlib/__init__.py                           33     26    21%   12-60
.tox/python/lib/python3.8/site-packages/virtualenv/util/path/_permission.py                                 21     14    33%   10-20, 24-26
.tox/python/lib/python3.8/site-packages/virtualenv/util/path/_sync.py                                       59     13    78%   15-16, 31, 34-39, 69-73, 84
.tox/python/lib/python3.8/site-packages/virtualenv/util/six.py                                              22     16    27%   12-28, 33-50
.tox/python/lib/python3.8/site-packages/virtualenv/util/subprocess/__init__.py                              18      9    50%   9-11, 20-32
.tox/python/lib/python3.8/site-packages/virtualenv/util/zipapp.py                                           24     15    38%   12-15, 19-24, 28-33
.tox/python/lib/python3.8/site-packages/virtualenv/version.py                                                2      0   100%
docs/conf.py                                                                                                 3      0   100%
ptr.py                                                                                                     103     61    41%   26-32, 43-84, 133-134, 142, 152, 161-175, 179-187, 194-209, 213, 219-222
tests/test_ptr.py                                                                                           56      0   100%
--------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                     6784   3111    54%

========================================================================= short test summary info ==========================================================================
FAILED ptr.py::BLACK
====================================================================== 1 failed, 13 passed in 35.91s =======================================================================
ERROR: InvocationError for command /home/tkloczko/rpmbuild/BUILD/pytest-runner-5.2/.tox/python/bin/pytest (exited with code 1)
_________________________________________________________________________________ summary __________________________________________________________________________________
ERROR:   python: commands failed

Declare Setuptools dependency

In #39, it comes to my attention that pytest-runner may be able to declare its minimum (run-time) dependency on setuptools. That seems reasonable to me. Let's try it and see what happens.

.whl file in pypi.org contains underscore instead of dash

See https://pypi.org/project/pytest-runner/5.1/#files

This confuses pip when it is restricted to binary-only installs.

I get an error similar to #41

runner doesn't honor dependency_links

Originally reported by: Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco)

---

When dependencies are resolved, it appears not to honor 'dependency_links'. It would be nice if it did.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/1

Support setup.py test arguments

There are two arguments for setuptools test cmd which would be useful to have in pytest-runner, reducing the need to use --addopts.

They are (https://github.com/pypa/setuptools/blob/master/setuptools/command/test.py):

('test-module=', 'm', "Run 'test_suite' in specified module"),
('test-suite=', 's', "Run single test, case or suite (e.g. 'module.test_suite')"),

While there are a few kinks in converting unittest 'suite' type args into pytest test names, even a very basic approach would be useful.

e.g. Allow --test-suite=tests instead of --addopts="tests/"

Unable to run tests with setuptools_scm >= 1.6.0

Originally reported by: Felix Yan (Bitbucket: felixonmars, GitHub: felixonmars)

---

I am running pytest-runner's tests with "py.test" in its hg repo root folder, and getting the following error:

______________________________________________________________ ERROR collecting docs/conf.py _______________________________________________________________
docs/conf.py:15: in <module>
    version = setuptools_scm.get_version(root='..')
/usr/lib/python3.4/site-packages/setuptools_scm/__init__.py:65: in get_version
    version = version_from_scm(root)
/usr/lib/python3.4/site-packages/setuptools_scm/__init__.py:34: in version_from_scm
    "instead." % root)
E   LookupError: setuptools-scm was unable to detect version for '/tmp'.
E
E   Make sure you're not using GitHub's tarballs (or similar ones), as those don't contain the necessary metadata. Use PyPI's tarballs instead.
================================================================= 1 error in 0.12 seconds ==================================================================

The setuptools_scm dev has replied to my bug report there and points to a bug in the sphinx configuration.

I have tried to change root='..' to root='.' and it works fine afterwards. Any hints?

For reference: pypa/setuptools_scm#44

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/6

Launching python script in test environment

Some of my tests are like the one below:

def test_integration_is_a_tty():
    """Simulate TTY to ensure output is as expected (terminal codes and all)."""

    stdout_wanted = '\x1b[1mcontent\x1b[0m\n'

    # unbuffer disables the output buffering that occurs when program
    # output is redirected from non-interactive programs
    # NOTE: install expect or expect-dev
    command = 'unbuffer python3 -m module.module --options'

    # the following is a rough approximation of unbuffer that should be
    # available on most systems (doesn't require installation of expect)
    # command = "script -eqfc 'python3 -m module.module --options' /dev/null"

    status, stdout = subprocess.getstatusoutput(command)

    assert stdout == stdout_wanted and status is 0

As you can imagine, it fails due to missing dependencies. Does anyone know if it is possible to launch a new interpreter in the test environment?

pytest is causing dependency_links to be ignored

(I originally filed this here, but was asked to move it. I've re-written it a bit now knowing that it might be pytest-runner

When using a setup.cfg, pytest can take over as unit test runner from the built-in stuff.

setup.cfg

[aliases]
test=pytest

When doing this and then when I run python setup.py test, I expect my unit tests to run and dependencies to get installed. Instead, PyPi is getting searched and dependency_links is getting ignored.

setup.py

from setuptools import setup, find_packages

setup(
	name='MyApp',
	version='1.0',
	maintainer='Me',
	maintainer_email='[email protected]',
	url='http://www.somewhere.com',
	packages = find_packages(),
	install_requires=['myotherproject==1.0'],
	setup_requires=['pytest-runner', 'pytest'],
	tests_require=['pytest', 'pytest-asyncio', 'myotherproject==1.0'],
	test_suite='tests',
	dependency_links=['git+ssh://myname@myserver:/git/path/to/stuff/myotherproject@master#egg=myotherproject-1.0'],
	include_package_data=True,
)

When I use the above with python setup.py test, myotherproject gets installed into .eggs.

If I then add a setup.cfg file like so to activate pytest:

setup.cfg

[aliases]
test=pytest

Then python setup.py test ignores dependency_links and attempts to search pypi for the myotherproject (and naturally it can't find it and errors out)

Then I just comment out the lines in setup.cfg and re-run python setup.py test and it works again.

setup.cfg

#[aliases]
#test=pytest

setup.cfg settings for pytest plugins break runner

If you use setup.cfg settings specific for certain plugins (like pytest-pep8), python setup.py pytest will throw errors.

Steps to reproduce:

1. Install deps

    pip install pytest pytest-pep8 pytest-runner
   

2. create a blank setup.py

    import setuptools
   
    setuptools.setup(
        name='pep8failure',
    )
   

3. create an example setup.cfg from the pytest-pep8 example

    [pytest]
    pep8ignore = * ALL
   

4. Run pytest

    py.test
   

   All works.

5. Run pytest-runner

    python setup.py pytest
   

   You get an error message

    running pytest
    error: error in setup.cfg: command 'PyTest' has no such option 'pep8ignore'
   

PyPI releases not showing up

Looks like 2.9 is still the latest release in PyPI, despite the 2.10 and 2.10.1 releases.

Make a new "inactive" release and archive the repository

As discussed in #63, it would probably be a good idea to make a new release to update the Inactive classifier on PyPI, and archive the repository.

home page on pypi is wrong

https://pypi.python.org/pypi/pytest-runner/2.7 points to https://bitbucket.org/pytest-dev/pytest-runner which is incorrect.

Version 5.0 causes pkg_resources.DistributionNotFound

Since release of 5.0, when installing our package on build machines, we get DistributionNotFound. I had to pin to "pytest-runner<5". Otherwise, using:

setup_requires=['pytest-runner'],
tests_require=['pytest', 'pytest-cov', 'coverage', 'requests_mock'],

• pip install '.[dev]'
  Processing /home/vsts/work/1/s
  Complete output from command python setup.py egg_info:
  /usr/share/miniconda/envs/py34/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'use_scm_version'
  warnings.warn(msg)
  warning: install_lib: 'build/lib' does not exist -- no Python modules to install

  zip_safe flag not set; analyzing archive contents...

  Installed /tmp/pip-k_2kf438-build/.eggs/UNKNOWN-0.0.0-py3.4.egg
  Traceback (most recent call last):
  File "", line 1, in
  File "/tmp/pip-k_2kf438-build/setup.py", line 97, in
  'Documentation': [REDACTED]
  File "/usr/share/miniconda/envs/py34/lib/python3.4/distutils/core.py", line 108, in setup
  _setup_distribution = dist = klass(attrs)
  File "/usr/share/miniconda/envs/py34/lib/python3.4/site-packages/setuptools-27.2.0-py3.4.egg/setuptools/dist.py", line 315, in init
  File "/usr/share/miniconda/envs/py34/lib/python3.4/site-packages/setuptools-27.2.0-py3.4.egg/setuptools/dist.py", line 361, in fetch_build_eggs
  File "/usr/share/miniconda/envs/py34/lib/python3.4/site-packages/setuptools-27.2.0-py3.4.egg/pkg_resources/init.py", line 854, in resolve
  pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application

Provide a license file

Originally reported by: Florian Wilhelm (Bitbucket: florianwilhelm, GitHub: florianwilhelm)

---

In setup.py it is stated that pytest-runner is MIT licensed but where is the license file? Normally Python packages provide a LICENSE.txt or LICENSE file.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/8

Versioned extra_requires are always installed

Since the update from 2.9 -> 2.10.1, the conditional dependency enum34 of h2 is always installed when running pytest-runner, but I am running with Python 3.6 and it shouldn't be installed.

The related entry is:

    extras_require={
        ':python_version == "2.7" or python_version == "3.3"': ['enum34>=1.0.4, <2'],
    }

Downgrading pytest-runner to 2.9 without any other changes fixes the behavior for me.

Declarative config breaks installs for old setuptools

Reproduction:

python3 -m venv venv
venv/bin/pip install flake8-mutable

The flake8-mutable project uses pytest-runner in setup_requires (ref), which installs packages via easy_install and not through pip (https://pip.readthedocs.io/en/1.4.1/cookbook.html#controlling-setup-requires), which is probably the reason why our CI is currently crashing:

Collecting flake8-mutable==1.2.0 (from -c python/requirements/constraints.txt (line 43))
  Downloading https://nexus.build-leapyear.com/repository/pypi/packages/97/6a/0fd1d903848fe043c7b6e5283d9def56425754098e69d7683a3ccbbea345/flake8-mutable-1.2.0.tar.gz
    Complete output from command python setup.py egg_info:
    /root/.lybuild/miniconda-4.3.31/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'use_scm_version'
      warnings.warn(msg)
    warning: install_lib: 'build/lib' does not exist -- no Python modules to install
    
    zip_safe flag not set; analyzing archive contents...
    
    Installed /tmp/pip-install-hsfxj0l7/flake8-mutable/.eggs/UNKNOWN-0.0.0-py3.6.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-hsfxj0l7/flake8-mutable/setup.py", line 40, in <module>
        'Programming Language :: Python :: 3.6',
      File "/root/.lybuild/miniconda-4.3.31/lib/python3.6/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/root/src/venv/lib/python3.6/site-packages/setuptools/dist.py", line 315, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "/root/src/venv/lib/python3.6/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
        replace_conflicting=True,
      File "/root/src/venv/lib/python3.6/site-packages/pkg_resources/__init__.py", line 853, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-hsfxj0l7/flake8-mutable/

setup.py test is going to be deprecated

from pytest-dev/pytest#5534

this project also relies on setuptools options that are now considered 'smells':

    setup_requires=[
        'pytest-runner',
    ],
    tests_require=[
        'pytest',
    ],

installer doesn't respect .eggs install dir from setuptools 7.0

Originally reported by: Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco)

---

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/3

Support all [pytest] commands

Originally reported by: Florian Wilhelm (Bitbucket: florianwilhelm, GitHub: florianwilhelm)

---

When defining user options like norecursedirs in setup.cfg or pytest.ini like explained here, pytest-runner can't handle them.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/7

Using setup_requires(['pytest-runner']) gives pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application

We have a simple local package that uses setup_requires(['pytest-runner']) and it has started to fail installation as of the 4.5 release of this package.

Collecting {internal tool}
  Downloading {internal repo}
    Complete output from command python setup.py egg_info:
    {virtualenv}/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'use_scm_version'
      warnings.warn(msg)
    warning: install_lib: 'build/lib' does not exist -- no Python modules to install
    
    zip_safe flag not set; analyzing archive contents...
    
    Installed /tmp/pip-install-mWRuOe/{internal tool}/.eggs/UNKNOWN-0.0.0-py2.7.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-mWRuOe/{internal tool}/setup.py", line 65, in <module>
        '''
      File "{virtualenv}/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "{virtualenv}/lib/python2.7/site-packages/setuptools/dist.py", line 315, in __init__
        self.fetch_build_eggs(attrs['setup_requires'])
      File "{virtualenv}/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
        replace_conflicting=True,
      File "{virtualenv}/lib/python2.7/site-packages/pkg_resources/__init__.py", line 853, in resolve
        raise DistributionNotFound(req, requirers)
    pkg_resources.DistributionNotFound: The 'pytest-runner' distribution was not found and is required by the application

multiple options to addopts

I might be missing something, but it seems the only way to pass multiple options to pytest are by using a single --addopts and wrapping the multiple optoins in quotes e.g.

python setup.py test --addopts '--cov=plugin.pyx --cov-config=.coveragerc'

Is this expected? If so, it might help to tweak the user_options definition to explain this. (As an aside, it wasn't clear to me how to even use one argument: --addopts arg=val or --addopts arg val etc.)

pytest-runner doesn't support dry run option

platform linux -- Python 3.6.3, pytest-3.1.3, py-1.4.34, pluggy-0.4.0

$ ./setup.py --dry-run test 
running pytest
-✂ for brevity ✂-
Traceback (most recent call last):
  File "./setup.py", line 22, in <module>
    'git-lp-propose=gitlptools:git_lp_propose'],
  File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/path/to/project/.eggs/pytest_runner-3.0-py3.6.egg/ptr.py", line 195, in run
  File "/usr/lib/python3.6/distutils/cmd.py", line 103, in __getattr__
    raise AttributeError(attr)
AttributeError: result_code

$ cat setup.cfg
[aliases]
test=pytest

pytest-runner sometimes picks wrong package to install

I'm not 100% sure this is pytest-runner, or something else, but it only happens when using the pytest-runner to run tests so I figured I'd start here. It's also doesn't happen consistently, so it's difficult to reproduce, but I guess starting an issue would be a way to start iterating over how to figure out what's happening.

In a recent build, I had this happen:

build   29-Feb-2016 13:29:51    Searching for pytest
build   29-Feb-2016 13:29:51    Best match: pytest sugar-0.5.1
build   29-Feb-2016 13:29:51    Downloading https://pypi.python.org/packages/source/p/pytest-sugar/pytest-sugar-0.5.1.tar.gz#md5=23bd36427e4df0ea9244662027f42ff1
build   29-Feb-2016 13:29:51    Processing pytest-sugar-0.5.1.tar.gz
build   29-Feb-2016 13:29:51    Writing /tmp/easy_install-G4yHj9/pytest-sugar-0.5.1/setup.cfg
build   29-Feb-2016 13:29:51    Running pytest-sugar-0.5.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-G4yHj9/pytest-sugar-0.5.1/egg-dist-tmp-ar85zH
build   29-Feb-2016 13:29:51    removing '/iad/finn/bamboo-agent-1/xml-data/build-dir/FIAAS-DEPLOYDAEMON-JOB1/.eggs/pytest_sugar-0.5.1-py2.7.egg' (and everything under it)
build   29-Feb-2016 13:29:51    creating /iad/finn/bamboo-agent-1/xml-data/build-dir/FIAAS-DEPLOYDAEMON-JOB1/.eggs/pytest_sugar-0.5.1-py2.7.egg
build   29-Feb-2016 13:29:51    Extracting pytest_sugar-0.5.1-py2.7.egg to /iad/finn/bamboo-agent-1/xml-data/build-dir/FIAAS-DEPLOYDAEMON-JOB1/.eggs
build   29-Feb-2016 13:29:51    
build   29-Feb-2016 13:29:51    Installed /iad/finn/bamboo-agent-1/xml-data/build-dir/FIAAS-DEPLOYDAEMON-JOB1/.eggs/pytest_sugar-0.5.1-py2.7.egg
build   29-Feb-2016 13:29:51    Traceback (most recent call last):
build   29-Feb-2016 13:29:51      File "/iad/finn/bamboo-agent-1/xml-data/build-dir/FIAAS-DEPLOYDAEMON-JOB1/setup.py", line 65, in <module>
build   29-Feb-2016 13:29:51        "fiaas-deploy-daemon = fiaas_deploy_daemon:main"
build   29-Feb-2016 13:29:51      File "/usr/lib/python2.7/distutils/core.py", line 152, in setup
build   29-Feb-2016 13:29:51        dist.run_commands()
build   29-Feb-2016 13:29:51      File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
build   29-Feb-2016 13:29:51        self.run_command(cmd)
build   29-Feb-2016 13:29:51      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
build   29-Feb-2016 13:29:51        cmd_obj.run()
build   29-Feb-2016 13:29:51      File "build/bdist.linux-x86_64/egg/ptr.py", line 67, in run
build   29-Feb-2016 13:29:51      File "/home/finn/.local/share/virtualenvs/fiaas-deploy-daemon/local/lib/python2.7/site-packages/setuptools/dist.py", line 313, in fetch_build_eggs
build   29-Feb-2016 13:29:51        replace_conflicting=True,
build   29-Feb-2016 13:29:51      File "/home/finn/.local/share/virtualenvs/fiaas-deploy-daemon/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 839, in resolve
build   29-Feb-2016 13:29:51        raise DistributionNotFound(req, requirers)
build   29-Feb-2016 13:29:51    pkg_resources.DistributionNotFound: The 'pytest' distribution was not found and is required by the application

When looking for pytest, it actually ends up selecting pytest-sugar, installing that, and then crashing out because pytest isn't installed. I have had a similar issue with flake8 and flake8-print. The next run might fail, and then suddenly it would work on the next run. Incidentally, pytest-sugar has already been installed by the time it gets around to trying to install pytest itself.

The relevant portions of setup.py for the run above:

        setup_requires=['pytest-runner', 'wheel', 'setuptools_git >= 0.3'],
        tests_require=['pytest', 'mock', 'vcrpy', 'pytest-sugar', 'pytest-html', 'pytest-cov'],

Any ideas about how to try and narrow down what's happening here?

Support subprocess launching in dependency context

In pypa/setuptools#794, Setuptools implements a technique for enabling subprocesses to inherit the dependencies installed by the test runner. Pytest-runner should honor that behavior (preferably as a synchronous release).

[Py3.3] AttributeError: type object 'test' has no attribute 'install_dists'

Hello. I'm seeing the following build failure with Python 3.3:

Installed /home/travis/build/twitterdev/twitter-python-ads-sdk/.eggs/pytest_runner-4.0-py3.3.egg
running pytest
Traceback (most recent call last):
  File "setup.py", line 65, in <module>
    **extra_opts
  File "/opt/python/3.3.6/lib/python3.3/distutils/core.py", line 147, in setup
    dist.run_commands()
  File "/opt/python/3.3.6/lib/python3.3/distutils/dist.py", line 930, in run_commands
    self.run_command(cmd)
  File "/opt/python/3.3.6/lib/python3.3/distutils/dist.py", line 949, in run_command
    cmd_obj.run()
  File "/home/travis/build/twitterdev/twitter-python-ads-sdk/.eggs/pytest_runner-4.0-py3.3.egg/ptr.py", line 157, in run
  File "/home/travis/build/twitterdev/twitter-python-ads-sdk/.eggs/pytest_runner-4.0-py3.3.egg/ptr.py", line 118, in install_dists
AttributeError: type object 'test' has no attribute 'install_dists'
The command "python setup.py test" exited with 1.

Looks like it may be related to this change.

Any suggestions on how to resolve this?

Thank you.

extras argument can't be set to False in setup.cfg

Reproduction

See attached setup.cfg file (Rename as setup.cfg)

setup.cfg.example.txt

Description

If I have extras assigned under [pytest] in setup.cfg, it will always pull in extras, regardless of the setting. In the setup.cfg example, I have:

[options.extras_require]
tf = tensorflow
...

[pytest]
extras = False

Tensorflow always gets installed. According to this issue in pyscaffold, pytest-runner reads it as string 'False' which evaluates to True.

Change command name to simply 'pytest'

Originally reported by: Jason R. Coombs (Bitbucket: jaraco, GitHub: jaraco)

---

In discussions with @hpk42, it sounds as if using the command name 'pytest' would be preferable, and that change would likely be sufficient to justify making pytest-runner the recommended technique for invoking pytest under setuptools.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest-runner/issue/2