Autocomplete for modules on PYTHONPATH about vscode-python

[DonJayamanne]

I cannot set an explicit path using python.autoComplete.extraPaths because the path changes.

You can specify relative paths in this setting. Please try that and let me know.
here's an example:

"python.autoComplete.extraPaths": [
    "${workspaceRoot}/custom/paths"   
],

Where ${workspaceRoot} is the workspace directory.

[ablakey]

Hi Don, thanks for responding.

I can do as you suggested and it works as intended, but that doesn't resolve my issue. I don't think I conveyed my issue well. I won't go into why (ROS workspaces), but suffice it to say that the various Python packages I want to autocomplete are found in a number of places that change throughout my day. I cannot reasonably find them with absolute or relative paths.

However, the compilation steps modifies my environment's PYTHONPATH envvar. This means that I can always find these packages in a Python REPL.

I think my issue can be succinctly collapsed into:

Is it possible and/or reasonable to expect that if I can open a REPL from an environment and import a Python package, then if I open VSCode from the same environment, I should be able to autocomplete that package?

I think this means appending the paths found in PYTHONPATH to the list of lookup locations for Autocomplete. I'm not sure this happens or if there's a way to do so through configuration today.

I appreciate your time.

[DonJayamanne]

then if I open VSCode from the same environment, I should be able to autocomplete that package?

Yes this should work. Please quit all instances if vscode (double check it isn't running in the background), then launch vscode from the terminal using code .
That should ensure the PYTHONPATH is inherited.

[ablakey]

Thanks, that helps narrow the scope a lot. If I should expect PYTHONPATH to be inherited then I'll experiment more with that specifically.

I did as you suggested, but it didn't work. While recording an example video of this (here) I noticed that my laptop's fan was trying to launch it into orbit. There's a few processes that eat up my resources until I kill VSCode.

I can cause the same processes to appear simply by starting VSCode and trying to autocomplete a custom package.

Maybe I'm killing Autocomplete with too many packages or some bad package.
I'm going to try to reproduce this to varying degrees using a fresh environment of only a few custom packages.

Note: I reported the wrong OS. So used to saying 16.04. This is on macOS Sierra.

[DonJayamanne]

Is there any chance you could share the source code? I'd like to check this out at my end.

[ablakey]

Let me drill down a little first. At the very least, I can replicate the issue with a much smaller set of code and share that with you. Right now a considerable amount of what's in my PYTHONPATH cannot be shared external to my company.

[ablakey]

Okay, so the CPU usage thing wasn't related. I had run code . in my home directory. I left it for half an hour and it finally calmed down. I'm sure it was indexing everything in my home dir or whatnot.

I dug deeper and I found a few interesting tidbits about how a ROS Catkin workspace links Python files. I cannot provide commentary on why but this is what I found:

$ printenv PYTHONPATH
/Users/ablakey/ws/devel/lib/python2.7/site-packages

$ ls -lh /Users/ablakey/ws/devel/lib/python2.7/site-packages/chronos_runtime
lrwxr-xr-x  1 ablakey  staff    96B 10 Nov 15:25 __init__.py@ -> /Users/ablakey/ws/devel/.private/chronos/lib/python2.7/site-packages/chronos_runtime/__init__

# Interesting, it's a symlink. Let's follow it:
$ cat /Users/ablakey/ws/devel/.private/chronos/lib/python2.7/site-packages/chronos_runtime/__init__.py

# -*- coding: utf-8 -*-
# generated from catkin/cmake/template/__init__.py.in
# keep symbol table as clean as possible by deleting all unnecessary symbols

from os import path as os_path
from sys import path as sys_path

from pkgutil import extend_path

__extended_path = "/Users/ablakey/ws/src/chronos/src".split(";")
for p in reversed(__extended_path):
    sys_path.insert(0, p)
    del p
del sys_path

__path__ = extend_path(__path__, __name__)
del extend_path

__execfiles = []
for p in __extended_path:
    src_init_file = os_path.join(p, __name__ + '.py')
    if os_path.isfile(src_init_file):
        __execfiles.append(src_init_file)
    else:
        src_init_file = os_path.join(p, __name__, '__init__.py')
        if os_path.isfile(src_init_file):
            __execfiles.append(src_init_file)
    del src_init_file
    del p
del os_path
del __extended_path

for __execfile in __execfiles:
    with open(__execfile, 'r') as __fh:
        exec(__fh.read())
    del __fh
    del __execfile
del __execfiles

So it looks like Catkin does a little import magic to link python source files in its devel space to the Python in the source directory, rather than symlinking every python source file found in src.

I'm going to bet that Autocomplete finds the package chronos_runtime with an __init__.py but nothing else. I'm going to bet that it only statically analyzes files, so if they're only discoverable at runtime, it will fail to discover them. Am I close?

If this is accurate, then I think this is a closed issue and I'll find an alternative way to handle my ever-mutating PYTHONPATH -> devel -> src files story.

[DonJayamanne]

I'm going to bet that it only statically analyzes files
Yes, this is correct.
In light of this and considering your findings, I'm choosing to close this issue.

Thanks for digging deeper into the issue.

[ablakey]

Thanks for taking the time to help me with this! I saw a closed feature request for wildcards in extraPaths. That would help address my issue. I will consider re-raising that feature request in another issue if there's no reasonable alternative.