microsoft / vscode-python-tools-extension-template

Template for creating VS Code extensions for python tools.

Template for VS Code python tools extensions

This is a template repository to get you started on building a VS Code extension for your favorite python tool. It could be a linter, formatter, or code analysis, or all of those together. This template will give you the basic building blocks you need to build a VS Code extension for it.

Programming Languages and Frameworks

The extension template has two parts, the extension part and language server part. The extension part is written in TypeScript, and language server part is written in Python over the pygls (Python language server) library.

For the most part you will be working on the python part of the code when using this template. You will be integrating your tool with the extension part using the Language Server Protocol. pygls currently works on the version 3.16 of LSP.

The TypeScript part handles working with VS Code and its UI. The extension template comes with few settings pre configured that can be used by your tool. If you need to add new settings to support your tool, you will have to work with a bit of TypeScript. The extension has examples for few settings that you can follow. You can also look at extensions developed by our team for some of the popular tools as reference.

Requirements

1. VS Code 1.64.0 or greater
2. Python 3.8 or greater
3. node >= 18.17.0
4. npm >= 8.19.0 (npm is installed with node, check npm version, use npm install -g [email protected] to update)
5. Python extension for VS Code

You should know to create and work with python virtual environments.

Getting Started

1. Use this template to create your repo.
2. Check-out your repo locally on your development machine.
3. Create and activate a python virtual environment for this project in a terminal. Be sure to use the minimum version of python for your tool. This template was written to work with python 3.8 or greater.
4. Install nox in the activated environment: python -m pip install nox.
5. Add your favorite tool to requirements.in
6. Run nox --session setup.
7. Optional Install test dependencies python -m pip install -r src/test/python_tests/requirements.txt. You will have to install these to run tests from the Test Explorer.
8. Open package.json, look for and update the following things:
   1. Find and replace <pytool-module> with module name for your tool. This will be used internally to create settings namespace, register commands, etc. Recommendation is to use lower case version of the name, no spaces, - are ok. For example, replacing <pytool-module> with pylint will lead to settings looking like pylint.args. Another example, replacing <pytool-module> with black-formatter will make settings look like black-formatter.args.
   2. Find and replace <pytool-display-name> with display name for your tool. This is used as the title for the extension in market place, extensions view, output logs, etc. For example, for the black extension this is Black Formatter.
9. Install node packages using npm install.
10. Go to https://marketplace.visualstudio.com/vscode and create a publisher account if you don't already have one.
    1. Use the published name in package.json by replacing <my-publisher> with the name you registered in the marketplace.

Features of this Template

After finishing the getting started part, this template would have added the following. Assume <pytool-module> was replaced with mytool, and <pytool-display-name> withMy Tool:

1. A command My Tool: Restart Server (command Id: mytool.restart).
2. Following setting:
   • mytool.args
   • mytool.path
   • mytool.importStrategy
   • mytool.interpreter
   • mytool.showNotification
3. Following triggers for extension activation:
   • On Language python.
   • On File with .py extension found in the opened workspace.
4. Following commands are registered:
   • mytool.restart: Restarts the language server.
5. Output Channel for logging Output > My Tool

Adding features from your tool

Open bundled/tool/lsp_server.py, here is where you will do most of the changes. Look for TODO comments there for more details.

Also look for TODO in other locations in the entire template:

• bundled/tool/lsp_runner.py : You may need to update this in some special cases.
• src/test/python_tests/test_server.py : This is where you will write tests. There are two incomplete examples provided there to get you started.
• All the markdown files in this template have some TODO items, be sure to check them out as well. That includes updating the LICENSE file, even if you want to keep it MIT License.

References, to other extension created by our team using the template:

• Protocol reference: https://microsoft.github.io/language-server-protocol/specifications/specification-3-16/
• Implementation showing how to handle Linting on file open, save, and close. Pylint
• Implementation showing how to handle Formatting. Black Formatter
• Implementation showing how to handle Code Actions. isort

Building and Run the extension

Run the Debug Extension and Python configuration form VS Code. That should build and debug the extension in host window.

Note: if you just want to build you can run the build task in VS Code (ctrl+shift+B)

Debugging

To debug both TypeScript and Python code use Debug Extension and Python debug config. This is the recommended way. Also, when stopping, be sure to stop both the Typescript, and Python debug sessions. Otherwise, it may not reconnect to the python session.

To debug only TypeScript code, use Debug Extension debug config.

To debug a already running server or in production server, use Python Attach, and select the process that is running lsp_server.py.

Logging and Logs

The template creates a logging Output channel that can be found under Output > mytool panel. You can control the log level running the Developer: Set Log Level... command from the Command Palette, and selecting your extension from the list. It should be listed using the display name for your tool. You can also set the global log level, and that will apply to all extensions and the editor.

If you need logs that involve messages between the Language Client and Language Server, you can set "mytool.server.trace": "verbose", to get the messaging logs. These logs are also available Output > mytool panel.

Adding new Settings or Commands

You can add new settings by adding details for the settings in package.json file. To pass this configuration to your python tool server (i.e, lsp_server.py) update the settings.ts as need. There are examples of different types of settings in that file that you can base your new settings on.

You can follow how restart command is implemented in package.json and extension.ts for how to add commands. You can also contribute commands from Python via the Language Server Protocol.

Testing

See src/test/python_tests/test_server.py for starting point. See, other referred projects here for testing various aspects of running the tool over LSP.

If you have installed the test requirements you should be able to see the tests in the test explorer.

You can also run all tests using nox --session tests command.

Linting

Run nox --session lint to run linting on both Python and TypeScript code. Please update the nox file if you want to use a different linter and formatter.

Packaging and Publishing

1. Update various fields in package.json. At minimum, check the following fields and update them accordingly. See extension manifest reference to add more fields:
   • "publisher": Update this to your publisher id from https://marketplace.visualstudio.com/.
   • "version": See https://semver.org/ for details of requirements and limitations for this field.
   • "license": Update license as per your project. Defaults to MIT.
   • "keywords": Update keywords for your project, these will be used when searching in the VS Code marketplace.
   • "categories": Update categories for your project, makes it easier to filter in the VS Code marketplace.
   • "homepage", "repository", and "bugs" : Update URLs for these fields to point to your project.
   • Optional Add "icon" field with relative path to a image file to use as icon for this project.
2. Make sure to check the following markdown files:
   • REQUIRED First time only: CODE_OF_CONDUCT.md, LICENSE, SUPPORT.md, SECURITY.md
   • Every Release: CHANGELOG.md
3. Build package using nox --session build_package.
4. Take the generated .vsix file and upload it to your extension management page https://marketplace.visualstudio.com/manage.

To do this from the command line see here https://code.visualstudio.com/api/working-with-extensions/publishing-extension

Upgrading Dependencies

Dependabot yml is provided to make it easy to setup upgrading dependencies in this extension. Be sure to add the labels used in the dependabot to your repo.

To manually upgrade your local project:

1. Create a new branch
2. Run npm update to update node modules.
3. Run nox --session setup to upgrade python packages.

Troubleshooting

Changing path or name of lsp_server.py something else

If you want to change the name of lsp_server.py to something else, you can. Be sure to update constants.ts and src/test/python_tests/lsp_test_client/session.py.

Also make sure that the inserted paths in lsp_server.py are pointing to the right folders to pick up the dependent packages.

Module not found errors

This can occurs if bundled/libs is empty. That is the folder where we put your tool and other dependencies. Be sure to follow the build steps need for creating and bundling the required libs.

Common one is pygls module not found.

TODO: The maintainer of this repo has not yet edited this file

Repo Owner Make sure you update this. As a repository owner you will need to update this file with specific instructions for your extension.

vscode-python-tools-extension-template's Issues

Extension fails to import `bundled` module

Hi, I cloned the template and followed the instructions to replace all the various bits for my own module. However, when I try to run/debug the extension, it fails to start correctly because it can't find/import the bundled module from server.py. I tried adding the root of the extension to sys.path, but it still can't find the module.

Relevant section of extension logs:

Name: µfmt
Module: ufmt
[DEBUG 2022-7-5 16:12:47.701]: Configuration: {"name":"µfmt","module":"ufmt"}
Python extension loading
Waiting for interpreter from python extension.
Python extension loaded
[INFO 2022-7-5 16:12:48.146]: Server run command: /Users/jreese/workspace/trailrunner/.venv/bin/python /Users/jreese/workspace/vscode-ufmt/bundled/tool/_debug_server.py
[INFO 2022-7-5 16:12:48.154]: Server: Start requested.
[DEBUG 2022-7-5 16:12:48.154]: Server State: Starting
Traceback (most recent call last):
  File "/Users/jreese/workspace/vscode-ufmt/bundled/tool/_debug_server.py", line 43, in <module>
    runpy.run_path(SERVER_PATH, run_name="__main__")
  File "/Users/jreese/.pyenv/versions/3.10.4/lib/python3.10/runpy.py", line 269, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/Users/jreese/.pyenv/versions/3.10.4/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/Users/jreese/.pyenv/versions/3.10.4/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/Users/jreese/workspace/vscode-ufmt/bundled/tool/server.py", line 40, in <module>
    from bundled.tool.utils import RunResult
ModuleNotFoundError: No module named 'bundled'
[Info  - 4:12:53 PM] Connection to server got closed. Server will restart.

Full log output: https://gist.github.com/jreese/72f9ee83a0cd6ef63918026d8a26e0d3

Extension repo: https://github.com/omnilib/vscode-ufmt

How to change the extension name in `editor.defaultFormatter`.

Hi, The extension namely yapf I published, but the name in editor.defaultFormatter is like {publisher}.yapf. How can I get a sample name instead of it.

Update vscode-languageclient to 8.0 after `pygls` starts supporting LSP 3.17

Do not attempt to launch any servers in case when Python is not installed

Something like:

public async hasInterpreters(): Promise<boolean> {
    const onAddedToCollection = createDeferred();
    // Watch for collection changed events.
    this.api.onChanged(async (e: PythonEnvCollectionChangedEvent) => {
        if (e.new) {
            onAddedToCollection.resolve();
        }
    });
    const initialEnvs = this.api.getEnvs();
    if (initialEnvs.length > 0) {
        return true;
    }
    await Promise.race([onAddedToCollection.promise, this.api.getRefreshPromise()]);
    return this.api.getEnvs().length > 0;
}

can be built off using the proposed discovery API.

Handle out of workspace files.

Server does not restart

The first time LSP starts up, it works correctly, but after running command Restart Server (or editing Settings), it stops in the following areas in the VSCode OUTPUT:

Server Command [RUN]: c:\Users...python.exe c:\Users...server.py
Server Command [DEBUG]: c:\Users...python.exe c:\Users..._debug_server.py
Exception ignored in atexit callback: <function ProcessManager.stop_all_processes at 0x0000029B83B57760>
TypeError: ProcessManager.stop_all_processes() missing 1 required positional argument: 'self'

After that, the server does not start forever and no output will be produced.

I found that this issue is caused by #48
( ada4b5ab9f1552e0cc5408302e1c559057f25ea4) and can be fixed by reverting the createServer() code in src/common/server.ts. The OUTPUT will be as follows.

Exception ignored in atexit callback: <function ProcessManager.stop_all_processes at 0x000001AFFD633250>
TypeError: ProcessManager.stop_all_processes() missing 1 required positional argument: 'self'
CWD Server: c:\Users...

missing example of a vs code command that is implemented in python

would love to see an example of how a simple command is registered in package.json and implemented in lsp_server.py using pygls. when trying to implement it myself based on this template, I had the need to pass the currently edited document URI to the command function.

to the best of my understanding, this requires some extra work under the activate function in extenstion.ts.
something along the lines of

context.subscriptions.push(
        onDidChangePythonInterpreter(async () => {
            await runServer();
        }),
        onDidChangeConfiguration(async (e: vscode.ConfigurationChangeEvent) => {
            if (checkIfConfigurationChanged(e, serverId)) {
                await runServer();
            }
        }),
        registerCommand(`${serverId}.restart`, async () => {
            await runServer();
        }),
        // new code here:
        registerCommand(`my_new_fancy_command`, () => {
            // some code to get the active lsp client
            if (lsClient) {
                if (vscode.window.activeTextEditor) {
                    const docUri = vscode.window.activeTextEditor.document.uri.toString();
                    // pass the active doc uril to the command
                    lsClient.sendRequest(`my_new_fancy_command`, [docUri]);
                }
            }
        }),
    );

the problem here is that in the current state of things, I cannot get a hold of the active LSP client. this is although it is a public global variable here. it is always undefined.

would love for help/ ideas here.
thanks for the great template!

Add documentation to show how to debug a production server.

Any template extension fails to activate when no workspace is opened

vscode-python-tools-extension-template/src/common/settings.ts

Lines 19 to 29 in 620e863

	export async function getExtensionSettings(namespace: string, includeInterpreter?: boolean): Promise<ISettings[]> {
	const settings: ISettings[] = [];
	const workspaces = getWorkspaceFolders();
	for (const workspace of workspaces) {
	const workspaceSetting = await getWorkspaceSettings(namespace, workspace, includeInterpreter);
	settings.push(workspaceSetting);
	}
	return settings;
	}

can return an empty array if no workspaces are opened.

Consumers assume atleast one element exists in the array:

vscode-python-tools-extension-template/src/extension.ts

Line 38 in 620e863

setLoggingLevel(settings[0].logLevel);

Causes issues like microsoft/vscode-black-formatter#124.

Change the `TODO` in the license file

It currently says:

vscode-python-tools-extension-template/LICENSE

Line 3 in 53d1c62

# TODO: Copyright (c) Microsoft Corporation.

But that is actually correct for the code coming from the template. That TODO should be moved to its own line for folks to add their own names to the license while retaining the Microsoft copyright for the code they got from this repo.

`npm install` shows `1 high severity vulnerability`

Similar to microsoft/vscode-isort#255

npm install output

npm WARN deprecated [email protected]: vsce has been renamed to @vscode/vsce. Install using @vscode/vsce instead.

added 375 packages, and audited 376 packages in 4s

74 packages are looking for funding
  run `npm fund` for details

1 high severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.

atexit.register decorator causes exception

I submitted a PR to fix an issue I noticed in the Ruff extension for VS Code.

Later I realized the code is from this template.

If I opened a PR to make the same change here, would it be merged?

Thanks for any insight!

Numpy error when using bundled libs from different python version

Hello team,

thanks for this template!
I am building an extension using this template, see the repo.
And it works, but only if the python version is the same as the python version used when it was built.

If it is not the case, when the user's python (python 3.12) has not the same version as the one that built it (python 3.11). I get the following error from numpy:

 File "/home/inigo/.vscode-server/extensions/codecarbon.codecarbon-0.1.0/bundled/libs/codecarbon/core/cpu.py", line 13, in <module>
    import pandas as pd
  File "/home/inigo/.vscode-server/extensions/codecarbon.codecarbon-0.1.0/bundled/libs/pandas/__init__.py", line 19, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

What do you think I should do? Build an extension per python version and then detect which one to load?

Thanks!

Test labels

Update recommended extensions to contain prettier

extensions.json seems to be missing prettier: esbenp.prettier-vscode. But the pipelines of say Black extension, seems to be checking linting which relies on prettier.

Add a way to inject debugger into LSP process without requiring attach to process.

1. Add a compound launch that starts both the extension and python debugger.
2. Add an environment variable to with path to debugger to config to "Debug Extension and Server".
3. Add code in server.py to load debugger if env variable is present.
4. Add code to server.ts to pass the environment variables to the Language server.
5. Update other debug configs

Adopt `@vscode/python-extension` to fetch Python extension API in all extensions which use the template

We now have a npm package which carries the Python extension api types, which makes it easier to track changed api types and also has a helper function which can be used to access the API:

Instructions to adopt can be found here: https://github.com/microsoft/vscode-python/blob/main/pythonExtensionApi/README.md

Extension debugging (TS) missing breakpoints, not clear it's updating on re-run

Hey folks, thanks for the great template!

I've had a fair bit of trouble trying to debug the extension.

When I run it, or both the extension and Python, it starts a new instance of VSCode and has the extension seemingly present; however, I cannot add breakpoints, and I don't believe changing the source of the extension is having any impact.

I may be misunderstanding the scope of the TX/extension debugging features. Alternatively, some cache of the production extension is stuck somewhere, and I'm still running that.

I would appreciate any ideas to debug!

Grab a tool from the selected environment if installed there

See astral-sh/ruff-vscode#17 .

Is this a template to implement code extension using Python?

Hi,

I am a little bit confused if this template will work with any type of document? For example, can I write a vscode extension which handle .csv file using Python?

Do users of the extension created with this need to install Python?

Thanks!

Ensure unique debug sessions for each server restart when debugging.

How to handle bundling native deps

I built the µfmt extension following this template, which results in bundling µfmt and its dependencies in the vsix file, as expected.

However, some of these dependencies have native extensions (eg, LibCST for parsing Python code). LibCST in particular will be requiring these native extensions in the future, with no fallback to pure-python, as that would greatly reduce performance and increase the ongoing maintenance cost of the package.

I've used the vscode-platform-specific-sample project to add builds for multiple platforms, but this does not seem to be a complete solution for bundling native Python extensions.

The problem is that these native extensions are specific to the major version of the Python runtime, and (afaict) VS Code does not seem to expose the runtime baked into the application in such a way that I can target my vsix builds to the appropriate Python versions, making it difficult to guarantee that I'm building a compatible extension to use with the bundled version of µfmt.

As of right now, it seems that the nox --session setup command simply builds/bundles a native extension based on whatever version of Python my nox binary was installed with:

Installed nox with 3.9.9:

(venv) amethyst@mordin ~/workspace/vscode-ufmt platforms± » which nox
/Users/amethyst/workspace/vscode-ufmt/venv/bin/nox
(venv) amethyst@mordin ~/workspace/vscode-ufmt platforms± » python -V
Python 3.9.9
(venv) amethyst@mordin ~/workspace/vscode-ufmt platforms± » find bundled/libs | grep .so$
bundled/libs/libcst/_parser/parso
bundled/libs/libcst/native.cpython-39-darwin.so

Installed nox with 3.10.6:

(venv) amethyst@mordin ~/workspace/vscode-ufmt platforms± » python -V
Python 3.10.6
(venv) amethyst@mordin ~/workspace/vscode-ufmt platforms± » find bundled/libs | grep .so$
bundled/libs/libcst/_parser/parso
bundled/libs/libcst/native.cpython-310-darwin.so

The .so files installed into bundled/libs/ are the only ones that make it into the vsix file when I finally run vsce package, but this means that if I pick the wrong runtime (that doesn't match the major version used by VSCode), then users of the vsix (that opt to use the bundled version of µfmt) will not be able to import/load the native extension, resulting in worse performance, and eventually break the bundled version of µfmt when LibCST removes the pure python implementation.

Is there any way to know/detect what version of the runtime is bundled with VS Code? Is there a way to use that bundled runtime to build a virtualenv and/or install nox? Is that something that can be done from a Github action? Or do I just need to depend on users having a local runtime/virtualenv and drop support for bundling a build of µfmt altogether?

Thank you!