kondratyev-nv / vscode-python-test-adapter

Python Test Adapter for the VS Code Test Explorer

Python Test Explorer for Visual Studio Code

This extension allows you to run your Python Unittest, Pytest or Testplan tests with the Test Explorer UI.

Getting started

• Install the extension
• Configure Visual Studio Code to discover your tests (see Configuration section and documentation of a test framework of your choice:
  • Unittest documentation
  • Pytest documentation
  • Testplan documentation
• Open Test View sidebar
• Run your tests using the icon in the Test Explorer

Features

• Shows a Test Explorer in the Test view in VS Code's sidebar with all detected tests and suites and their state
• Convenient error reporting during test discovery
• Unittest, Pytest and Testplan debugging
• Shows a failed test's log when the test is selected in the explorer
• Re-run tests on save
• Supports multi-root workspaces
• Supports Unittest, Pytest and Testplan test frameworks and their plugins

Comparison to Python extension's Test View

• Better error reporting during the discovery stage. In case of errors, you will see such tests in an errored state, and by clicking on them, a complete error message would be shown in the Output panel. Python Extension, at best, won't show your tests that contain errors (syntax errors and invalid imports, for example).
• Works better with pytest plugins - Tavern, for example. Python Extension won't discover these tests.
• Based on Test Explorer UI. This fact may be useful if you have a workspace with projects in different languages/frameworks. Test Explorer UI has a lot of plugins, and you can conveniently discover and run tests at the same View.
• Shows you errors and a complete output of your tests just by clicking on a failed test.
• Shows only relevant folders from your workspace. Showing all workspace folders, as the Python Extension is doing, can be an issue when you have multiple workspace folders, but only a couple of them have any tests.
• There might be some configurations when this extension will discover and run your tests, but Python Extension - won't.
• User experience with both extensions is highly subjective. However, you might like the user interface of this extension better. Also, each discovery, test execution, and test cancellation won't require you to select a folder when you have multiple in your workspace.

Configuration

By default the extension uses the configuration from Python extension for Visual Studio Code. To configure Python for your project see Getting Started with Python in VS Code.

However, the test framework used by this extension can be overridden by pythonTestExplorer.testFramework configuration property. Right now, the three available options are unittest, pytest and testplan. When the property is set to null, the configuration from Python extension is used.

Configuring Python test discovery and execution

List of currently used properties:

Property	Description
python.pythonPath	Path to Python.
python.envFile	Path to environment variable definitions file.
python.testing.cwd	Optional working directory for unit tests.
python.testing.unittestEnabled	Whether to enable or disable unit testing using unittest (enables or disables test discovery for Test Explorer).
python.testing.unittestArgs	Arguments used for test discovery (currently only -s and -p arguments are considered).
python.testing.pyTestEnabled	Whether to enable or disable unit testing using pytest (enables or disables test discovery for Test Explorer).
python.testing.pytestPath	Path to pytest executable or a pytest compatible module.
python.testing.pyTestArgs	Arguments passed to the pytest. Each argument is a separate item in the array.
python.testing.autoTestDiscoverOnSaveEnabled	When true tests will be automatically rediscovered when saving a test file.
pythonTestExplorer.testFramework	Test framework to use (overrides Python extension properties python.testing.unittestEnabled and python.testing.pyTestEnabled).
pythonTestExplorer.testplanPath	Relative path to testplan main suite.
pythonTestExplorer.testplanArgs	Arguments passed in. Each argument is a separate item in the array.
pythonTestExplorer.testplanUseLegacyDiscovery	Use old style of test listing for discovery (compatible with older versions of Testplan)
pythonTestExplorer.testplanEnabled	Enable testing using Testplan. Note that Testplan is only supported for Python 3.7+.
pythonTestExplorer.outputs.collectOutputs	Collect test run outputs in separate output channel.
pythonTestExplorer.outputs.showOutputsOnRun	Activate test outputs channel on run.

Configuration supports a placeholders for the workspace folder as ${workspaceFolder} and environment variables in a form of ${env:YOUR_ENVIRONMENT_VARIABLE}.

Configuring Test Explorer UI

The following configuration properties are provided by Test Explorer UI:

Property	Description
testExplorer.onStart	Retire or reset all test states whenever a test run is started
testExplorer.onReload	Retire or reset all test states whenever the test tree is reloaded
testExplorer.codeLens	Show a CodeLens above each test or suite for running or debugging the tests
testExplorer.gutterDecoration	Show the state of each test in the editor using Gutter Decorations
testExplorer.errorDecoration	Show error messages from test failures as decorations in the editor
testExplorer.errorDecorationHover	Provide hover messages for the error decorations in the editor
testExplorer.sort	Sort the tests and suites by label or location. If this is not set (or set to null), they will be shown in the order that they were received from the adapter
testExplorer.showCollapseButton	Show a button for collapsing the nodes of the test tree
testExplorer.showExpandButton	Show a button for expanding the top nodes of the test tree, recursively for the given number of levels
testExplorer.showOnRun	Switch to the Test Explorer view whenever a test run is started
testExplorer.addToEditorContextMenu	Add menu items for running and debugging the tests in the current file to the editor context menu
testExplorer.mergeSuites	Merge suites with the same label and parent
testExplorer.hideEmptyLog	Hide the output channel used to show a test's log when the user clicks on a test whose log is empty
testExplorer.hideWhen	Hide the Test Explorer when no test adapters have been registered or when no tests have been found by the registered adapters. The default is to never hide the Test Explorer (some test adapters only work with this default setting).

See Test Explorer UI documentation for the latest changes in configuration.

Configuring debug

The extension will look for a configuration in launch.json with "type": "python" and "purpose": ["debug-test"] (or "request": "test") to load any of the following options during debugging

• name
• console
• env
• stopOnEntry
• showReturnValue
• redirectOutput
• debugStdLib
• justMyCode
• subProcess
• envFile

For example,

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug test",
            "type": "python",
            "request": "attach",
            "console": "externalTerminal",
            "justMyCode": false,
            "stopOnEntry": true,
            "envFile": "${workspaceFolder}/.env.test",
            "purpose": ["debug-test"]
        }
    ]
}

FAQ

1. Disable duplicated Code Lenses

Test Explorer UI provides a set of Code Lenses to run and debug tests. However, the Python extension also provides a set of its own. As of now, there are no way to disable Code Lenses from the Python extension, see microsoft/vscode-python#10898. If you use only Python Test Explorer to run your tests, you can disable testing functionality by the Python extension with the following settings

{
    // these settings are 'false' by default, so removing those should also work
    "python.testing.unittestEnabled": false, 
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": false,
    "pythonTestExplorer.testFramework": "pytest" // (or unittest)
}

2. How to use this extension to run Django tests?

You can use pytest to discover and run Django tests with this extension. For this, install pytest-django package and follow its Getting started guide.

3. How to re-run tests on save?

Test Explorer UI allows to run tests on saving files in a workspace. To enable autorun, right-click on a test suite and select "Enable autorun". After that, any changes to files in a workspace will re-execute tests when files are saved. See the example below.

You can enable autorun for all tests by clicking on three dots in the Test explorer UI bar.

Troubleshooting

Whether no tests were discovered in the Test Explorer view or anything else doesn't work as expected, you can see logging output selecting Python Test Adapter Log in the Output section.

Questions, issues, feature requests, and contributions

• If you're happy using this extension - star GitHub repo and share your positive feedback in VSCode Marketplace.
• If you have any question or a problem with the extension, please file an issue. Make sure, to include information on Python version, test framework you using, what plugins are installed (mostly for pytest), and what do you see in logs (Python Test Adapter Log in the Output section).
• Contributions are always welcome! Please see contributing guide for more details.

vscode-python-test-adapter's Issues

Update vscode-test-adapter-api package

UI bug: Bad grouping of tests resulting in two main root containers for the same repo

I had this repo open as non-primary root in a multi-root workspace: https://github.com/pyblish/pyblish-qml/
...and in the Test Explorer the UI looks like this:

See how there's 2 of pyblish-qml - All tests? yet their contents are not really duplicated.

One seems to have filenames (with unittest class names under them, and then methods under those) and the other is names of classes in test files (with methods under them.)

Windows import path for unit-tests

Detected an issue regarding a bad formated/joined path for start directory.
This was previously fixed in #29
This fix was removed from latest version of the extension (0.3.0), which leads to an error loading tests.

Modifying line 25 in file placeholderAwareWorkspaceConfiguration.js make it work again.
FROM:
this.resolvePath(original.unittestArguments.startDirectory)
TO:
this.resolvePlaceholders(original.unittestArguments.startDirectory)

See bellow the error logs from 'Python Test Log Adapter' and the test folder path which is badly joined.
Adding manually the fix from above fix to the extension installed on my PC made it work ok.

2019-02-13T14:07:36.554Z info at 'back-end' [pytest runner]: Reading configuration for workspace back-end
2019-02-13T14:07:36.555Z info at 'back-end' [pytest runner]: Pytest test discovery is disabled
2019-02-13T14:07:36.873Z info at 'back-end' [unittest runner]: Environment variables file d:\develop\workarea\repos\mycompany\myproject\back-end.env does not exist
2019-02-13T14:07:36.881Z info at 'back-end' [unittest runner]: Discovering tests using python path "d:\develop\workarea\repos\mycompany\myproject\back-end\venv\Scripts\python.exe" in d:\develop\workarea\repos\mycompany\myproject\back-end with pattern test.py and start directory d:\develop\workarea\repos\mycompany\myproject\back-end\tests
2019-02-13T14:07:37.832Z crit at 'back-end' [unittest runner]: Test loading failed: Process exited with code 1: Traceback (most recent call last):
File "", line 90, in
File "", line 59, in discover_tests
File "C:\Users\myuser\AppData\Local\Programs\Python\Python36\lib\unittest\loader.py", line 338, in discover
raise ImportError('Start directory is not importable: %r' % start_dir)
ImportError: Start directory is not importable: 'd:\develop\workarea\repos\mycompany\myproject\x08ack-end\tests'

Errors in the path consists of numerous issues:
d:\\develop\\workarea\repos\\mycompany\\myproject\x08ack-end\tests

breaks down to a join between:

• d:\\develop\\workarea (double backslash)
• \repos (single backslash)
• \\mycompany\\myproject (double backslash again)
• \x08ack-end\tests (single backslash again and \b being converted to \x08)

test case not discovered when files are under a folder

under vs code i have the following
masterfolder
/dev
/test

if i put the code under dev and test the test cases are not discovered, if i place them under master folder it works fine. Is that by design or is this an environmental problem.

No tests discovered for pytest version >= 4.1.1

Does not filter tests

Thanks for a great extension.

I'm finding that when I try to run a single test the entire suite is run. Only the test that I have launched gets the blue icon but it stays blue for ages while all the tests run.

An I doing something wrong?

Thanks.

Locale variables not defined

Test adapter seems to run with empty environment. That makes some sense, but it's arguably incorrect wrt locales.

Namely, Python 3 < 3.7 assumes US-ASCII locale if none is defined, which is wrong in most cases. E.g., Click refuses to work with this locale.

Test adapter should either pass through LANG and LC_* from the environment, or alternately set a reasonable default (e.g., C.UTF-8 on systems that support it)

Not marking django unittest as passed

Whenever I run the test from the Test Explorer UI, the functional tests using selenium are run and marked as passed, but the Django unittests are not marked as test. The extensions is set to use pytest.

When I run the tests from Terminal using pytest, all tests are passed.

"Process leakage"

Though MS Python extension provides its own discovered tests for TestView, I prefer yours for a couple of reasons. Though it runs smoothely most of time, I found that the more I run tests, the more dangling python3 processes are stay in the system. After a couple of hours it becomes like this and eats all of memory:

It doesn't happen if I don't open TestView at all and since testing in MS Python extension is disabled, I suspect this one.

Tests not discovered by Test Explorer

Hello,

I can run my unit tests in visual studio code with the built-in tester (see image):

But when I click the refresh button on the Test Explorer view, I get the following exception in the Python Test Adapter output:
Especially the "Error: spawn python ENOENT" is unclear to me. Any idea on what I need to change?

2019-02-21T15:13:43.723Z info at 'python-musings' [unittest runner]: Reading configuration for workspace python-musings
2019-02-21T15:13:43.727Z info at 'python-musings' [pytest runner]: Reading configuration for workspace python-musings
2019-02-21T15:13:43.727Z info at 'python-musings' [pytest runner]: Pytest test discovery is disabled
2019-02-21T15:13:43.728Z info at 'python-musings' [unittest runner]: Environment variables file d:\github\python-musings.env does not exist
2019-02-21T15:13:43.730Z info at 'python-musings' [unittest runner]: Discovering tests using python path "python" in d:\github\python-musings with pattern test*.py and start directory d:\github\python-musings\src\01-regular-expressions
2019-02-21T15:13:43.736Z crit at 'python-musings' [unittest runner]: Test loading failed: Error occurred during process execution: Error: spawn python ENOENT

Below, you can see the settings.json file:

{
    "python.unitTest.unittestArgs": [
        "-v",
        "-s",
        "./src/01-regular-expressions",
        "-p",
        "test*.py"
    ],
    "python.unitTest.pyTestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.unittestEnabled": true,
    "python.linting.pylintEnabled": true,
    "python.linting.enabled": true
}

.env variables are not loaded before executing test for Django

I have prepared my Django project and most of the settings are set using environment variables. Since I've made this change the Test Explorer shows test discovery as failed because it cannot find the DJANGO_SETTINGS_MODULE environment variable in the list of available environment variables.

I have given the python.envfile path for the extension to look for environment variables from the postactivate file as shown below.

{

"python.pythonPath": "/Users/<user_name>/.virtualenvs/<folder_name>/bin/python",
"python.envFile": "/Users/<user_name>/.virtualenvs/<folder_name>/bin/postactivate",
"python.unitTest.pyTestArgs": [
    "./<folder_name>/tests"
]

}

If I run the tests manually from the VS Code terminal window (the small run button above test definition), they run properly and all of them pass. However, the test discovery fails with the test explorer.

Therefore I've deducted that the Test Explorer UI doesn't load the environmental variables used by VS Code from postactivate. I see that there is a similar issue that you fixed in the v0.2.4 but I am not sure why this is happening again. Can you please take a look at this? Thank you!

Suite finding doesn't work if the test discovery includes printed text

If you have any sort of code that prints to stdout on import of a test file, it is read as a test suite and added to the tests (and subsequently breaks tests.)

No test report when pytest can't repr() a test failure in --showlocals mode

When using pytest, the test adapter will report a blank result (no success or failure) if a test fails and pytest raises an exception when reporting the test failure.

Here's example_tests.py:

class Thing:
    def __repr__(self):
        raise NotImplementedError

def test_the_thing():
    t = Thing()
    assert t == 3

def test_something_works():
    assert 1 == 1

Expected output: Test Explorer shows one pass, one fail.

Actual output: Test Explorer shows no test results; the icons to the left of the tests in the tree go back to the square-in-circle. Clicking the test nodes does nothing. The Python Test Adapter Log just shows a normal execution that completes.

I'm not completely sure what's happening here because when running pytest on the command line, the tests execute fully, and the exception caused by the __repr__ failure is contained within the output:

$ pytest example_tests.py

========================================= test session starts =========================================
platform darwin -- Python 3.6.8, pytest-4.3.0, py-1.8.0, pluggy-0.9.0
Django settings:settings.pytest (from ini file)
rootdir: /Users/yoz/Work, inifile: pytest.ini
plugins: mock-1.10.1, metadata-1.8.0, jest-0.3.0, django-3.4.8, cov-2.6.1
collected 2 items

example_tests.py F.                                                                             [100%]

============================================== FAILURES ===============================================
___________________________________________ test_the_thing ____________________________________________

    def test_the_thing():
        t = Thing()
>       assert t == 3
E       assert <[NotImplementedError("") raised in repr()] Thing object at 0x10d949d30> == 3

example_tests.py:7: AssertionError
================================= 1 failed, 1 passed in 0.24 seconds ==================================

Attempting to cancel test-running throws an error

If I am running tests and I attempt to abort using the red "X" button, I get this popup notification:

Running the contributed command:'test-explorer.cancel' failed.

Nothing appears in the Python Test Adapter Log.

Tests complete but keep spinning

When I run tests in the test UI, I can see in the Python Test Log that the tests have completed but the icons keep spinning and never change to a completed icon:

After running the tests, if I try to run other tests then they won't even change to a spinning icon to begin with, but the tests still run. The icons just don't change.

Make it easier to contribute

Tonight I was keen to contribute some fixes to this extension, but I couldn't get it working in my local dev environment before bedtime. Some things that would help:

• Any information in the README about setting it up from source
• Anything to set up the Python environment to run the extension tests
• Separation of unit tests (which could be run in the Mocha Test Explorer) from integration tests (which can only be run using cross-env or the debugger, because they require the vscode module)

pytest: find doctest

I tried to follow #24 to enable doctesting in my package.

I added arguments for pyTest

"python.unitTest.pyTestArgs": [
        "--doctest-modules",
    ],

When I run this in a console it test the docstring,
python -m pytest --doctest-modules

What can be the problem ? Thanks a lot.

Support python.unitTest.pyTestPath configuration property

Tests are discovering but not running

Hi, I'm trying to start using Tavern (https://taverntesting.github.io/) over pytest framework. I got my tests discovered in the explorer, but when I try to run it, it instantly stops.
Log show only the following:

2019-08-29T12:21:03.913Z info at 'tools' [pytest runner]: Reading configuration for workspace tools
2019-08-29T12:21:03.914Z info at 'tools' [pytest runner]: Running tests using python path 'C:\Users\user\AppData\Local\Programs\Python\Python37\python.exe' in c:\Users\user\repos\tools
2019-08-29T12:21:03.915Z info at 'tools' [pytest runner]: Environment variables file c:\Users\user\repos\tools.env does not exist
2019-08-29T12:21:06.881Z info at 'tools' [pytest runner]: Test execution completed

The test still displayed as it wasn't run and I get no test logs output.
How to investigate this problem further? Everything works fine using the terminal (I'm running on Windows via Powershell).

Thanks in advance.

.env variables are not loaded before executing test

I have prepared my Django project for deployment on Heroku and most of the settings are set using environment variables. Since I've made this change the Test Explorer shows all the tests as failed, however, if I run the tests manually from the VS Code editor window (the small run button above test definition), they run properly and all of them pass.

Therefore I've deducted that the Test Explorer UI doesn't load the environmental variables used by VS Code from .env. I hope you manage to fix this issue!

Thank you!
Here is my .env file:

DJANGO_SETTINGS_MODULE=superlists.settings
DJANGO_SECRET_KEY='*************'
SITENAME='127.0.0.1'
DATABASE_URL='postgres://********/***'

UI: Add path differentiation for identical file names

I'm working on a project which has multiple folders below the project root, and each folder has its own tests.py. So my Test Explorer looks like this:

... which is not hugely helpful when trying to find the right one.

When dealing with multiple identically-named files, VSCode's practice is to differentiate on path by adding a smaller path description to tab headers:

These smaller, lighter/darker labels (a.k.a. Descriptions) are now available in extension tree views. However, that relies on Test Explorer UI implementation (which I've requested).

In the meantime, how about changing the label to show a differentiating folder name as well as the file name. So, in a project with these files...

• foo/stuff/tests.py
• bar/tests/tests.py
• baz/tests/tests.py

... the TreeItem labels would be:

• Either .../stuff/tests.py or foo/.../tests.py
• bar/.../tests.py
• baz/.../tests.py

Add logging

Icons not showed properly

The icons of tests are not showed properly

Use https://tox.readthedocs.io/en/latest/ (Tox) for testing different flavours of python so windows users can run integration tests and find it easier to contribute

As said above. This should also allow for testing of more versions of python (3.4, 3.5, 3.6 etc) if wanted in the future.

Attempting to debug tests shows an error message and nothing happens

Attempting to debug a test (with the debug button on each test) throws an error:

Error while debugging test: Error: Method not implemented

Test suites that use setUpClass do not work.

I haven't collected the exact reasoning for this one, but from what i can tell any unittest.TestCase that uses setUpClass will not run that function. I'm assuming it's because the tests are run outside of their specific suites.

TeardownClass fixture run multiple times

Hi,

I am testing file access in my code and I have a Unittest written like this.

class TestClass(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        # create files
        return super().setUpClass()
    
    def test_files_access_1(self):
        # test files access
    
    def test_files_access_2(self):
        # test files access
    
    @classmethod
    def tearDownClass(cls):
        # delete files
        return super().tearDownClass()

If I run the test with python -m unittest discover all my tests passes. If I run all the tests from the test explorer sidebar, "test_file_access_2" will fail because the files are deleted. But, everything works if I run my test from the test file view.

This does not happen if I run each test one by one. I can make my test pass, when I do a run all, if I comment the tearDownClass method.

It seems the tearDownClass is run multiple time (once per method test to run?). Could it be possible to have the same behaviour than Unittest discover (tearDownClass run after all test from class are finished)?

No tests detected despite native Python tests discovery working.

Hello!

I love the idea of this extension but I'm unfortunately not having much luck making it work. It installs fine and the panel is empty and neither the Run nor Refresh buttons do anything. No errors that I can see, just nothing happens.

If it's helpful: I'm on Linux CentOS 7.4.

That said, if I run Python: Discover Unit Tests, the standard builtin Python language extension manages to detect them all properly in my workspace.

Could it be because I'm using multi-root workspaces? Does this extension work with those or does it only scan the first workspace path it finds?

Support python.unitTest.pyTestArgs configuration property

Capture output from unittest tests

Now when running unittest tests only result of a test is captured from a TestResult of failed and skipped tests.
For a debug purpose it would be great to capture stdout from tests.

Tests with unittest.skip not marked as skipped for pytest and python 2.7

File with tests

import unittest


@unittest.skip("demonstrating skipping")
def test_simple_check_skipped():
    pass

Configuration

{
    "python.pythonPath": "python2.7",
    "python.unitTest.pyTestEnabled": true
}

Expected behaviour: test_simple_check_skipped marked as skipped.
Actual behaviour: test_simple_check_skipped marked as "not run".

Tests sometimes wont show code lens, nor show up in test explorer UI

Add an additional config variable that can override the test platform of vscode-python just for this addon

I suggest that this adapter should be able to override vscode-python's choice of unit test platform.

We could have a config variable like this

// When config var is set
vscode-python-test-adapter.unitTestPlatform = "unittest";
// Use unittest for python test explorer, vscode-python uses its own settings

// when config var is not set
vscode-python-test-adapter.unitTestPlatform = null;

// deduce unitTestPlatform by vscode-python settings

This would allow people, before this adapter has pytest/nosetest support, to continue using pytest/nosetest in tandem with this adapter if their code happens to have tests that default unittest supports. Personally, i use pytest in my day to day just because the result is that comes out of vscode python's test log is much more useful then unittest or nosetest, despite following unittest conventions.

Test adapter is not discovering dependencies in alternate directories

I am not seeing any tests in the sidebar widget, despite the Python extension recognizing my tests:

(The "Run Tests" button works but nothing happens in the test Explorer). I am seeing output in the log:

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: v:\Binning Code\nearest_neighbor (working)\root\nested, inifile:
collected 66 items
<Package 'v:\\Binning Code\\nearest_neighbor (working)\\root'>
  <Package 'v:\\Binning Code\\nearest_neighbor (working)\\root\\nested\\bluelight_binning'>
    <Module 'test_utilities.py'>
....

My preferences are "pythonTestExplorer.testFramework": null and "python.unitTest.pyTestEnabled": true and "python.unitTest.unittestEnabled": false. I'd really like to get this working but I'm not sure what else I can try. Any suggestions?

Add support for pytest-describe

Hello,

Thanks for great extension and good work with pytest extension.
Could you add support for pytest-describe plugin as well?

I did some debugging and I found that tests are put into groups based on file and classes e.g.

test_file1 (group) -> Test_Class _1 (group) -> test_method_1

I'm not sure how this information is collected by for pytest-describe we have methods inside method. All methods with prefix describe should act like Test Classes and other methods (in my case with prefix it) should act like test methods e.g.

test_file_1 (group) -> describe_method_1 (group) -> it_test_method_1

In current state all methods with describe prefix are skipped and execution is incorrect because run command look like as follow: pytest test_file_1::it_test_method_1 where it should look like follow pytest test_file_1::describe_method_1::it_test_method_1

It would be great to get support for this plugin because it has native support for all pytest features and organize python tests in the best way (in my opinion).

You can find more about this plugin at page: https://github.com/ropez/pytest-describe
I'm using latest vscode editor and latest version of plugins.

Thank you in advance for help.
In case of any help I can produce more examples or execute some tests with real code.

Best Regards and Happy new Year
Pawel

Support for pytest tests

Support more placeholders for configuration

List of all placeholders from VS Code https://code.visualstudio.com/docs/editor/variables-reference.
List of placeholders considered by Python Extension https://github.com/DonJayamanne/pythonVSCode/blob/527af23a25d10725cb9086cc385137649c8a9233/src/client/common/variables/systemVariables.ts#L93.

Support unittest subTest method

Sample test case

class SubtractionTests(unittest.TestCase):
    def test_minus_operations(self):
        with self.subTest("Two minus one is one"):
            print("checking 2 - 1")
            self.assertEqual(1, 2 - 1)
        with self.subTest("Two minus two is one"):
            print("checking 2 - 2")
            self.assertEqual(1, 2 - 2)
        with self.subTest("Two minus zero is two"):
            print("checking 2 - 0")
            self.assertEqual(2, 2 - 0)

Right now the test method test_minus_operations is discovered but not showing state on run.

Tests not executed when unittest args has -s parameter

See skipped tests should run test from suite with start folder in config and should run suite with start folder in config.

The explorer is not show me the tests

If i want to run the tests but the icons are not showing up and always want to install pytest via pip but its already installed on my computer.
The tests are running but i do not see them in the explorer.
It says TEST DISCOVERY FAILED.
I use POP_OS(Linux(ubunu))

Test Discovery failed:
Error: ImportError while loading conftest '/home/user/Documents/Project1/tests/conftest.py'.
tests/conftest.py:5: in
from flaskr import *
E ModuleNotFoundError: No module named 'flaskr' <-------" its FLASKR for a reason "

Traceback (most recent call last):
File "/home/user/.vscode/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/run_adapter.py", line 16, in
main(tool, cmd, subargs, toolargs)
File "/home/user/.vscode/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/main.py", line 90, in main
parents, result = run(toolargs, **subargs)
File "/home/user/.vscode/extensions/ms-python.python-2019.6.22090/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 34, in discover
raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 4)

pytest: collect and run doctests

When pytest is run with the --doctest-modules option, it'll collect doctests like so:

<Package '/Users/yoz/Code/MyPeeps/core'>
  <DoctestModule 'models.py'>
    <DoctestItem 'core.models.Person.instance_type.__str__'>
    <DoctestItem 'core.models.Person.instance_type.display_name'>
    <DoctestItem 'core.models.Person.instance_type.next_anniversary'>
    <DoctestItem 'core.models.Person.instance_type.split_name'>
...

Cannot debug unittest

When I try to debug a test using unittest I get
"Error while debugging test: Error: Unittest debugging is not supported at the time."

Do you have any plans for releasing debugging with unittest?

Workaround is to use pytest.

Python Test Explorer does not show test if Test Framework set to null

If the Test framework is left to null the extension is not able to show the test even if the python extension settings are set to Unittest.

If I set Test Framework to Unittest the extension is able to show the tests in the side pane.

I think this is happening because of the change in the settings of the Python extension settings name for the test framework option.

Support for Multi-root Workspaces

pythonPath not working properly if it contains ..

My pythonPath is: ${workspaceRoot}/../bin/python
It has to be relative to fix an issue regarding the debugger, and everything else is working properly. However, when discovering tests, I get:

2019-01-17T00:25:30.775Z info at 'src' [unittest runner]: Discovering tests using python path "/home/matt/Projects/test/src/bin/python" in /home/matt/Projects/test/src with pattern *test*.py and start directory .
2019-01-17T00:25:30.801Z crit at 'src' [unittest runner]: Test loading failed: Error occurred during process execution: Error: spawn /home/matt/Projects/test/src/bin/python ENOENT

The .. is ignored. The path should be /home/matt/Projects/test/bin/python

Broken if you have variables in your configured Python/Pytest executable path

I have a multi-root workspace and each root has a settings.json which looks like:

{
    "python.pythonPath": "${workspaceFolder}/.rez-python.sh",
    "python.unitTest.pyTestPath": "${workspaceFolder}/.rez-pytest.sh",
    "python.linting.pylintPath": "${workspaceFolder}/.rez-pylint.sh",
    "python.linting.pylintUseMinimalCheckers": false
}

I do this because of the setup at work, we use Rez:
https://github.com/nerdvegas/rez
to manage our environments, so I made some utility scripts which put me in the correct Python environment (via Rez) for that tool, be it python itself or pytest or pylint.

When I have such a configuration, in the Test Explorer Output log, if I try to run a test, I got:

Error occurred during process execution: Error: spawn ${workspaceFolder}/.rez-python.sh ENOENT

Normally ENOENT = no such directory, so perhaps it's just a matter of the "${variables}" not being resolved prior to spawning?

For reference, VSCode supports variables in its settings:
https://code.visualstudio.com/docs/editor/variables-reference

Heads up: vscode-python team working on similar project

Not an issue as such, just a note that the Microsoft's vscode-python team is now working on their own Test Explorer, with much of the same functionality: microsoft/vscode-python#193 (comment)

I told them about your work, but I suspect they want to implement it a different way than the "Test Explorer UI" extension.

No "Show source" button for unittest methods

This is a great extension. I would love to see deep linking (i.e. double click a test in the test explorer and go to that method in the test file). Thanks for all of the great work.

Multiple variable setters in .env file

Hello!
I have a $PYTHONPATH in .env which contains multiple paths.
If I write it like this:

PYTHONPATH="<path1>:<path2>:<path3>"

everything works as expected. But it looks messy so I tried to rewrite it like this:

PYTHONPATH="${PYTHONPATH}:<path1>"
PYTHONPATH="${PYTHONPATH}:<path2>"
PYTHONPATH="${PYTHONPATH}:<path3>"

Python autocomplete feature finds this paths and handle them correctly, but it seems that PythonTestAdapter cannot parse .env, it shows no tests at all. If I put them back in single line, tests are shown again.

UX suggestion: sort the groups alphabetically

With a lot of multiple roots in a workspace, it can get a bit difficult to scroll considering it's not sorted alphabetically.

Would it be hard to add an alphabetic sort of the treeview items?