RFE: Support Remote Interpreter about vscode-python

[DonJayamanne]

This is a great suggestion. The architecture of VSCode and the extension supports this. However this would require a lot of effort, primarily in two parts. One would be to sync the two environments and the other would be the creation of a proxy component that sits on the server to manage all of the comms and surely enough this would have to be developed in python.

However, this would require weeks if not longer to build. But something that I will most definitely explore as a side project.

[DonJayamanne]

From @satyenr on May 18, 2016 11:51

Thanks for accepting this RFE. I was trying to add this feature myself, but the lack of knowledge of VSCode related stuff and other priorities forced me to put that project aside for the time being.

[DonJayamanne]

Todo: Add support for docker containers as per #179

[DonJayamanne]

Suggestions for #183

[DonJayamanne]

@vbem, @satyenr , @satyenr, @nffdiogosilva
Hi everyone, I would like to re-visit this to understand the requirements and the constraints.

Here are some of the environments (usages):

• Docker/vagrant containers
• Remove servers (linux, mac, windows)

Requirements:

• Linting (code analysis)
• Code navigation (find references, go to definition, etc)
• Refactoring
• Debugging (done, this works today, but SSH is to be looked at - separate issue)

One possible solution I'm thinking of is:

• Install nodejs on the remote server
• Install a npm package (nodejs module)
• Start a socket server on this target server
• VS Code will communicate with the remote server over a socket connection

This is the easiest solution and quickest to implement.
However, this requires the installation of:

• nodejs on the server
• an npm package

Would this be acceptable?

The only other solution is the following one. However this would take a lot longer to develop and implment:

• install a python package on the server

[DonJayamanne]

From @satyenr on October 4, 2016 9:12

When I logged the ticket, I had the following use case in mind:

• Remote server has a fully configured Python environment, which is more or less identical to the deployment setup.
• The developer workstation may have Python installed, but it doesn't have the fully configured environment. While it is possible to build a virtual environment containing the required modules in many cases, some modules don't work cross-platform, or are hard to compile. In any case, it is always preferable to development using an environment that is similar, if not identical, to the deployment environment.
• The development and deployment environments may even be running different operating systems.

As a result, linting, code-completion etc don't work as expected and end up causing more trouble than helping. A workaround is to run VSCode in a VNC session, or using X forwarding. In both cases, responsiveness of the UI is a function of network connectivity/load.

Originally, I had imagined that it will be implemented by running commands on the remote machine over SSH - that is what PyCharm does. Not sure if they maintain the connection as long as the IDE is open.

Using SSH ensures that you don't have to worry about things like authentication, managing a daemon on the remote machine etc. But using SSH means assuming that the client has a working SSH client - something you can't assume about Windows clients. I suspect PyCharm may be using its own SSH client - something I don't think is trivial to implement.

Your idea of using a socket server on remote machine seems like a good one. I don't particularly care for which language it is implemented in. Python would be ideal, since we are anyway assuming that python is installed on the remote machine. But I am fine with NodeJS as well. As long as the protocol is open, someone can always implement a python version later.

install a python package on the server

I assume you meant - install a python package on the remote which is then run as a daemon listening for requests from VSCode. Let me know if that is not the case.

[DonJayamanne]

@satyenr

I assume you meant - install a python package on the remote which is then run as a daemon listening for requests from VSCode. Let me know if that is not the case.

Yes, you are right. Will try to start with the node daemon, and later move it to a python daemon.

Finally, an open protocol does exist today:
https://github.com/Microsoft/language-server-protocol

Again, implementing this will be done at a later stage.

• Phase 1: Custom protocol with solution developed in nodejs
• Phase 2: microsoft language server protocol
• Phase 3: Develop this in Python

[DonJayamanne]

From @satyenr on October 4, 2016 9:47

Sounds like a good plan to me. I can help with the Python part if you want.

I don't have any experience with NodeJS, unfortunately.

[DonJayamanne]

From @nffdiogosilvadc on November 11, 2016 23:50

Hello =),

How is the current state of this issue?

Thank you

[DonJayamanne]

@nffdiogosilvadc
no work has started on this, as there isn't much interest in this feature. Except possibly for the two of you.

[DonJayamanne]

From @aventurella on November 28, 2016 14:24

I'll raise my hand for this. Our whole dev team formerly relied on vagarant and currently relies on docker to do all of our development. Specifically during development, we use docker-compose to ensure all of the devs are running the same environment.

Today we use Anaconda + Sublime. I have been exploring Visual Studio Code for Python development and something similar is a must. This looks to be where it will originate from if it is to be. I can't thank you enough for even discussing it.

In our docker development case we just start up a new container that has the anaconda remote server on it. It gets the volumes from the django application, so it has access to all of the code currently running:

volumes:
  - ./django:/usr/app
  - /usr/local/lib/python3.5

We also deploy on docker, so if this were a: We need to debug a running server we would end up doing something similar, just attaching the container and disposing of it when done. From that point of view I have absolutely no issue with this workflow:

- Install nodejs on the remote server
- Install a npm package (nodejs module)
- Start a socket server on this target server

I am also aware that not everyone uses Docker. I only mention the Docker use case here for those that may also depend on it. Naturally every deployment would differ and have different needs.

[DonJayamanne]

From @mgallagher on December 3, 2016 5:2

+1
this would be amazing

[DonJayamanne]

From @nffdiogosilva on December 7, 2016 15:26

Now that it seems that more people are interested in this feature, will it return to roadmap again?

[DonJayamanne]

Agreed, looks like there's enough interest here to get this into the roadmap now.
There are two ways to go about this. I'll post both options, so as to get the feedback.

[DonJayamanne]

There are two ways to implement this feature:

Option 1: Add dependency on node.js on the remote server

Benefit: Easier to develop
This is the easiest option for me (less python code), more node.js (javascript) code - i'm familiar with this

[DonJayamanne]

There are two ways to implement this feature:

Option 2: Add dependency on node.js on the remote server

Benefit: No dependency on node.js (except some other python libraries that I'll have to write)
Will require more work as I'll need to write everything from scratch using Python (this isn't my language of choice).

[DonJayamanne]

FYI, I'll commence the work with the node.js approach (Option 1).

[DonJayamanne]

From @satyenr on December 8, 2016 3:47

Option 2: Add dependency on node.js on the remote server

You meant, "Add a Python module on the remote server", methinks.

[DonJayamanne]

From @SierraN on December 29, 2016 12:25

For my team it is mandatory functionality without which we can not work, so fingers crossed :)

[DonJayamanne]

From @ianfitzpatrick on January 3, 2017 5:47

Hoping for this too. This is single feature is what's keeping me from switching to Visual Studio Code. Would be amazing if this lands!

[DonJayamanne]

From @wasinski on January 17, 2017 16:50

Another word of encouragement: this will be an absolute killer feature!

[DonJayamanne]

From @peymanmortazavi on January 20, 2017 23:46

👍 I'll switch to VSCode at work if this feature becomes available.

[DonJayamanne]

From @gulshan on January 25, 2017 9:55

https://github.com/palantir/python-language-server
Will this be any help?

[DonJayamanne]

Thanks, that just might work

[DonJayamanne]

From @adlermedrado on February 13, 2017 16:2

+1 to this feature. It'll be great.

[DonJayamanne]

From @satyenr on February 13, 2017 16:32

Seems like Anaconda (Sublime Text) has implemented this feature using a similar approach. A daemon (written in Python) listens on a given TCP port in the remote host. I have tried it - and my initial impression is that it works well enough.

@DonJayamanne If you haven't started working on this already, it may be worth looking at Anaconda to see if the server can be reused here. The code is available under GPLv3.

For what it is worth, PyCharm maintains an SSH connection with the remote host to achieve this.

[DonJayamanne]

From @YAmikep on March 10, 2017 18:12

👍 Hoping for this too. Would be amazing if this lands! Thanks

[DonJayamanne]

From @liqing-ustc on March 20, 2017 5:20

Hoping for this feature!!!

[DonJayamanne]

From @tony-garcia on March 23, 2017 19:31

+1

I'm using VSCode for most things these days but for a few projects the dev server is in a python Docker container so I'm still using PyCharm for those (since it allows associating a project to a python interpreter in a Docker container or docker-compose service). Would be cool to just use VSCode for everthing.

[DonJayamanne]

From @johnnywell on April 2, 2017 2:13

+1
I recently heard about this extension on Talk Python to me and switched to VS Code since then. I'm really looking forward to seeing this in action!

[DonJayamanne]

From @jaymickey on April 8, 2017 5:43

+1. I develop mostly on a Windows machine, and had planned to utilise a Linux Docker container to develop a flask based web application, but was disappointed to see that there is currently no way to do this while still having access to code completion (intellisense), linting, etc.

[DonJayamanne]

From @wsgeek12 on May 5, 2017 19:50

This would be great yes yes yes +1

[DonJayamanne]

From @IBestuzhev on May 11, 2017 9:31

+1. It would be a nice feature as I often use Docker (and sometimes Vagrant) to set-up the environment.

I can share what I use now.

For pylint it's quite easy, but does not work too fast unfortunately. I used a custom path for pylint and point it to my custom .pylint.py (gist).

I was trying to do the same with jedi, but I did not manage to pass the stdin/stdout connection. I modified python.jediPath to point to jedi.py file from that gist. Unfortunately due to some stdin/stdout issues I was able to do it only if I modify completion.py (the one in the docker) to process one request per time. I just added a break to watch method. Also there is something in docker that prevents from running it under exec, only run works.

This jedi setup is working but it's really slow to use. Obviously it's a terrible idea to deploy the container every time, it was just proof of concept. I think replacing the communication to tcp will speed it up.

So ideally is to have TCP communication between client/providers/jediProxy and pythonFiles/completion. In this case it will be easy to port-forward the connection to remote host.

Or as workaround it can be TCP communication between jedi.py from python.jediPath to modified completion.py that runs on remote. But this way is dirtier, as custom jedi.py file should block on import and prevent execution of some code from original completion.py.

Another thing that is needed here is path mapping.

But I hope someone may find this ideas useful and can extend it.

[DonJayamanne]

@IBestuzhev wow, thanks, will try to get started on this. busy with a lot of things.
But this will surely make things easier, at least for docker.

[DonJayamanne]

From @IBestuzhev on May 11, 2017 12:50

@DonJayamanne Thanks!

My example is for docker, but I think it's not bound to docker that much. For TCP connection there is ssh port forwarding, so it should not really matter if it connects to some local listener, or listener under docker with --port option, or listener on remote server with ssh -L tunnel.

For running pylint it can be ssh again, it support to execute command remotely instead of opening shell. The problem here is Windows developers, who will use not default implementation, but putty, bash -c "ssh ..." or cygwin. But it should not introduce problems, well, I started to use it with .bat file on Windows first.

It would be nice if plugin can detect Docker or Vagrant, as Anaconda does, but as for me it should be in future releases.

I checked the Language Server implementation. It implements both communications. The language server can be started locally as process and use stdin/stdout. Or it can be connect to TCP, and server should be started manually to accept JSON RPC requests. But the implementation of TCP server is not "production ready", I'd say that project is in alpha state and it often fails with segfaults.

[DonJayamanne]

Closed in favor of #79