Performance of Code Completion for LARGE files about vscode-python

[brettcannon]

@MilkyHearts @wiredprairie I can't make any promises about stability, accuracy, etc., but if you go into your settings and put "python.jediEnabled": false that will prompt you to restart VS Code and then it will download our experimental language server. It's written in C# and taken from our Python workload for Visual Studio, so it should be much faster.

We are also working on it regularly, so if you end up wanting to try newer bits later before we release a public preview, just go into the extension's directory and delete the Analysis directory; the extension will download the latest bits when you restart VS Code again.

But as I said, this is not even public preview for a reason, so no promises about anything. 😁

[MilkyHearts]

@brettcannon OMG THANK YOU! the autocomplete is instant after setting jediEnabled to false im so happy.

[brettcannon]

In the new year we're hoping to start looking into our intellisense performance and quality so we can begin to address the issues you and others are running into with it.

[brettcannon]

[ @MilkyHearts I edited your ✉️ ⬆️ to come off as more 😃 and less 😠 ]

[wiredprairie]

Jedi is unfortunately slow. I just confirmed that with a simple test.

import jedi
with open('example.py', 'r') as example:
    e=example.read()

script = jedi.Script(e)
completions = script.completions()  # <<< WAIT 

This may not be an adequate comparison or usage, but using the code above, it takes less than 5 seconds to build the list of completions.

We'd definitely want Intellisense. The classes wouldn't be usable without Intellisense.

One issue is that it the penalty for extracting the completion data from a Python file appears to be each time Intellisense is requested, rather than being cached.

However, I'm not sure if users would accept a 20 second delay, even once per VS code launch for a file. (And a 5 second delay if that's the fastest it could be due to Jedi performance probably isn't acceptable either more than once a session.)

Ideally, the Jedi results would be cached until they've become stale or invalidated. As the files I'm talking about are generated, and very infrequently changing, a one time Jedi-to-cache file result would be a very effective way of reducing the time taken by further usage of Intellisense. Further, in our case, it would be great if the cache files could be deployed to a cache directory or as side-car files so that we'd Jedi-compile them once per Python file generation and then no user's VS Code instance would need to locally process the file, ever.

If the results were to be cached, especially for larger Python files, I would expect many Python extension users in VS Code would benefit.

[MikhailArkhipov]

This may be improved by fix for #152 (no longer immediately loading method docs, just names)

[brettcannon]

Unfortunately 40,000 lines of Python code is simply a large file to process. We use Jedi for our intellisense and it seems to not be handling a file of that magnitude that well. I'm assuming you still want the intellisense, just faster? Or do you want to just turn it off to avoid the penalty?

Otherwise we have discussed trying to share the intellisense engine between us and PTVS (would probably be a new one, not the one it currently has), but it would require downloading .NET and no concrete plans beyond "it's a possiblity" have been had yet.

[wiredprairie]

@brettcannon -- any thoughts about this issue?

[MikhailArkhipov]

@wiredprairie - is it better in 0.9?

[wiredprairie]

@MikhailArkhipov - unfortunately, it still takes 20+ seconds.

[brettcannon]

@wiredprairie would it be possible to upload a test file for us to benchmark against?

(BTW we are upgrading Jedi to 0.12.0; it probably won't resolve this issue but you never know 😉 ).

[wiredprairie]

@brettcannon This is still a good example.

[brettcannon]

@wiredprairie Oops, sorry for not noticing the hyperlink in the initial message!

[MilkyHearts]

Its been almost a year and IntelliSense is still so slow...
This is really holding me back from using VSCode.

[brettcannon]

Closing as this is an upstream issue which we don't have direct control or influence over.