Proto install-global not working on windows? about proto

[rotu]

The input has context.tool_dir.path="/proto/tools/python/latest" which is not a directory. Even if the pinned global version is 3.11.6, the path still shows "latest". This may or may not be a red herring 🕵️

[TRACE 13:56.192] warpgate::plugin:install_global Calling plugin function install_global plugin="python" input={"context":{"tool_dir":{"path":"/proto/tools/python/latest","virtual_prefix":"/proto","real_prefix":"C:\Users\dan\.proto"},"version":"latest"},"dependency":"pdm","globals_dir":{"path":"/userhome/.local/bin","virtual_prefix":"/userhome","real_prefix":"C:\Users\dan"}}

[milesj]

@rotu Both of those fail with "program not found". Can you verify that PATH is correct?

For python, it would be using the pip shim: https://github.com/moonrepo/python-plugin/blob/master/src/proto.rs#L176
And node the npm shim: https://github.com/moonrepo/node-plugin/blob/master/crates/common/src/commands.rs#L6

I just tested proto install-global node rimraf myself and it worked. So maybe a Windows env var issue?

[rotu]

My path starts with C:\Users\dan\.proto\shims;C:\Users\dan\.proto\bin;. If I add a file called pip.exe (either to my shims folder or anywhere later in my path) then it tries to run that instead of returning "program not found".

I don't know how you expect this code to work - it calls exec_command which expects an executable, so I'm not sure how the .cmd or .ps1 shims could get invoked at all!

[milesj]

Does rust not use PATHEXT?!?! 😣

Let me dig deeper.

Edit: Omg it doesn't: https://github.com/rust-lang/rust/blob/master/library/std/src/sys/windows/process.rs#L417 And some context: rust-lang/rust#37381

Let me patch this then.

[rotu]

Re: #298

Can't we just skip the middleman and have proto install-global call commands::run::run itself? It seems a wee bit silly to open a shell, which opens the python(|.cmd|.ps1) shim, which wraps a second invocation of proto, which calls run().

[milesj]

proto doesn't know what processes/commands are going to be ran, so no, it cannot use run itself.

It was built this way so that installing/uninstalling a global can be more than just a child process in the wasm code. If they want, they can run fs operations as well, or modify env vars, etc.

[milesj]

can you try 0.23.2

[rotu]

proto doesn't know what processes/commands are going to be ran, so no, it cannot use run itself.

It was built this way so that installing/uninstalling a global can be more than just a child process in the wasm code. If they want, they can run fs operations as well, or modify env vars, etc.

Gotcha. I can see that the top-level code can't do this. But the call to exec_command is in python-plugin, which knows exactly how to invoke pip since it created the shim in the first place!

can you try 0.23.2

Roger! It installs something but the paths seem all incorrect:

proto install-global python pdm

...
Installing collected packages: pdm
━━━━━━ Installing pdm for Python
WARNING: The script pdm.exe is installed in 'C:\Users\dan\AppData\Roaming\Python\Python312\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pdm-2.10.3
WARNING: There was an error checking the latest version of pip.
[ INFO 2023-11-20 16:45:43] proto::commands::install_global:install_global Installed pdm to ~/.local/bin!

proto list-global python

Error: × Main thread panicked.
├─▶ at crates\cli\src\commands\list_global.rs:54:54
╰─▶ called Result::unwrap() on an Err value: Os { code: 3, kind: NotFound, message: "The system cannot find the path specified." }
help: set the RUST_BACKTRACE=1 environment variable to display a backtrace.

proto install-global node rimraf

...
━━━ Installing rimraf for Node.js
added 41 packages in 1s
[ INFO 2023-11-20 16:51:14] proto::commands::install_global:install_global Installed rimraf to ~.proto/tools/node/globals/bin!

But the directory ~\.proto/tools/node/globals/bin doesn't exist. The files rimraf, rimraf.cmd, rimraf.ps1 are all in ~\.proto/tools/node/globals/.

proto list-global node

No global packages installed

[rotu]

BTW, great job on proto tool info python and proto tool info! These make it so much easier to get oriented!

[milesj]

The problem is that we only account for unix paths in the python plugin. Need to add C:\Users\dan\AppData\Roaming\Python\Python312\Scripts also (windows is annoying lol).

But the directory ~.proto/tools/node/globals/bin doesn't exist. The files rimraf, rimraf.cmd, rimraf.ps1 are all in ~.proto/tools/node/globals/.

Looks like another case of windows doing things differently -_-

[rotu]

Looks like another case of windows doing things differently -_-

Yeh. Also the Python paths I think are relative to the version in use and the node path is not. I don't know which is the better behavior for proto, but it seems like it should be one or the other.

[milesj]

Do you know if theres an arg/env var that we can use for pip to override the install dir? Similar to what we do for node.

[milesj]

Maybe fix for node: moonrepo/node-plugin#22 Will need to test more.

[rotu]

Do you know if theres an arg/env var that we can use for pip to override the install dir? Similar to what we do for node.

I think this should do it:
pip config --site set global.prefix $PROTO_HOME/tools/python/globals

[rotu]

And maybe also add this to $PROTO_HOME/tools/python/<version>/install/sitecustomize.py:

import site, sys, os
site.addsitedir(os.path.join(sys.exec_prefix, '../../tools/python/globals/Lib/site-packages'))

[milesj]

@rotu

This should fix it: moonrepo/python-plugin#7 But I'll also need to update the proto side to resolve a version so it's not "latest".

[milesj]

#300

[rotu]

@rotu

This should fix it: moonrepo/python-plugin#7 But I'll also need to update the proto side to resolve a version so it's not "latest".

Sweet! FYI, I'm not sure how to update my proto to integrate the new plugin version. Guessing I'll have to wait for a wasm build to be uploaded and can't do something like:

plugins.python = "github:moonrepo/python-plugin@fix-windows-globals"

[milesj]

Yeah I'll have to publish a release. Gonna test on my windows machine first just to ensure its right.

[rotu]

Yeah I'll have to publish a release. Gonna test on my windows machine first just to ensure its right.

Might not be a bad idea to upload artifacts in the PR CI since we're building it anyway!

[milesj]

@rotu Yeah I want to figure out a better solution for all this.

You could always clone the repo, build locally, and test it too :P

[milesj]

Testing on my windows machine. We can probably just drop --user, since it'll install into the tools dir.

But I'm not sure how to fix PATH resolving, since this path (and --user path) will change between python versions.

Maybe this is a "don't manage python globals in proto" situation.

PS C:\Users\Miles\Projects\proto> cargo run -- run python -- -m pip install pdm
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
     Running `target\debug\proto.exe run python -- -m pip install pdm`
[DEBUG 2023-11-21 21:43:11] proto  Running proto v0.23.2  args=["target\\debug\\proto.exe", "run", "python", "--", "-m", "pip", "install", "pdm"]
[TRACE 43:11.779] starbase::app  Running startup phase with 1 systems
[TRACE 43:11.779] starbase::app  Running analyze phase with 1 systems
[TRACE 43:11.779] starbase::app  Running execute phase with 2 systems
[DEBUG 43:11.781] proto_core::tool_loader:run  Traversing upwards to find a configured plugin  tool="python"
[DEBUG 43:11.781] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:11.781] starbase_utils::fs:run  Reading file  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:11.785] warpgate::loader:run  Creating plugin loader  cache_dir="C:\\Users\\Miles\\.proto\\plugins"
[TRACE 43:11.785] warpgate::loader:run  Loading plugin python  plugin="python"
[TRACE 43:11.785] warpgate::loader:run  Plugin already downloaded and cached  plugin="python" path="C:\\Users\\Miles\\.proto\\plugins\\python-06bc5da02073099597305a8691894325ec8d0a86f0772886be138331f73fba19.wasm"
[DEBUG 43:11.785] proto_core::tool_loader:run  Loading WASM plugin  source="C:\\Users\\Miles\\.proto\\plugins\\python-06bc5da02073099597305a8691894325ec8d0a86f0772886be138331f73fba19.wasm"
[TRACE 43:11.786] proto_core::tool_loader:run  Storing tool identifier  id="python"
[TRACE 43:11.786] proto_core::tool_loader:run  Storing user configuration  config={"auto_clean":false,"auto_install":false,"node_intercept_globals":true}
[TRACE 43:11.786] proto_core::tool_loader:run  Storing proto environment  env={"arch":"x64","os":"windows","home_dir":{"path":"/userhome/","virtual_prefix":"/userhome","real_prefix":"C:\\Users\\Miles"},"proto_dir":{"path":"/proto/","virtual_prefix":"/proto","real_prefix":"C:\\Users\\Miles\\.proto"}}
[DEBUG 43:11.786] proto_core::tool:run  Creating tool python and instantiating plugin
[DEBUG 43:11.786] proto_core::tool_manifest:run:load  Loading manifest.json  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:11.786] starbase_utils::fs:run:load  Opening file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:11.786] starbase_utils::fs_lock:run:load  Locking file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:11.787] starbase_utils::fs_lock:run:load  Unlocking file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[DEBUG 43:12.675] proto_core::tool:run  Created tool python and its WASM runtime
[TRACE 43:12.675] warpgate::plugin:run  Calling plugin function register_tool  plugin="python" input={"id":"python"}
[TRACE 43:12.676] warpgate::plugin:run  Called plugin function register_tool  plugin="python" output={"inventory":{},"name":"Python","plugin_version":"0.2.0","self_upgrade_commands":[],"type":"Language"}
[TRACE 43:12.677] proto_core::version_detector:run  Attempting to find local version from config files  tool="python"
[TRACE 43:12.677] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles\\Projects\\proto"
[DEBUG 43:12.678] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:12.678] starbase_utils::fs:run  Reading file  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:12.678] warpgate::plugin:run  Calling plugin function detect_version_files  plugin="python" input={}
[TRACE 43:12.679] warpgate::plugin:run  Called plugin function detect_version_files  plugin="python" output={"files":[".python-version"]}
[TRACE 43:12.679] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles\\Projects\\proto"
[TRACE 43:12.679] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles\\Projects"
[TRACE 43:12.680] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles\\Projects"
[TRACE 43:12.680] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles"
[TRACE 43:12.680] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles"
[TRACE 43:12.680] proto_core::version_detector:run  Attempting to use global version from manifest  tool="python"
[DEBUG 43:12.680] proto_core::version_detector:run  Detected global version from manifest  tool="python" version="3.12.0" file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[DEBUG 43:12.681] proto_core::tool:run  Resolving a semantic version or alias  tool="python" initial_version="3.12.0"
[DEBUG 43:12.681] proto_core::tool:run  Resolved to 3.12.0 (without validation)  tool="python" version="3.12.0"
[DEBUG 43:12.681] proto_core::tool:run  Checking if tool is installed  tool="python" install_dir="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0"
[DEBUG 43:12.681] proto_core::tool:run  Tool has already been installed, locating binaries and shims  tool="python" install_dir="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0"
[DEBUG 43:12.681] proto_core::tool:run  Locating executable for tool  tool="python"
[TRACE 43:12.682] warpgate::plugin:run  Calling plugin function locate_executables  plugin="python" input={"context":{"tool_dir":{"path":"/proto/tools/python/3.12.0","virtual_prefix":"/proto","real_prefix":"C:\\Users\\Miles\\.proto"},"version":"3.12.0"}}
[TRACE 43:12.683] warpgate::plugin:run  Called plugin function locate_executables  plugin="python" output={"globals_lookup_dirs":["$HOME/.local/bin","$APPDATA/Roaming/Python312/Scripts","$APPDATA/Python312/Scripts"],"primary":{"exe_path":"install/python.exe","parent_exe_name":null},"secondary":{"pip":{"no_bin":true,"parent_exe_name":null,"shim_before_args":"-m pip"}}}
[DEBUG 43:12.683] proto_core::tool:run  Found an executable  tool="python" exe_path="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0\\install/python.exe"
[DEBUG 43:12.684] proto::commands::run:run  Running Python  bin="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0\\install/python.exe" args=["-m", "pip", "install", "pdm"]
Collecting pdm
  Obtaining dependency information for pdm from https://files.pythonhosted.org/packages/51/bb/78a45cddd416cc23302c0b0698f8288de07ad6b883462857818e055e1fdc/pdm-2.10.3-py3-none-any.whl.metadata
  Downloading pdm-2.10.3-py3-none-any.whl.metadata (10 kB)
Collecting blinker (from pdm)
  Obtaining dependency information for blinker from https://files.pythonhosted.org/packages/fa/2a/7f3714cbc6356a0efec525ce7a0613d581072ed6eb53eb7b9754f33db807/blinker-1.7.0-py3-none-any.whl.metadata
  Downloading blinker-1.7.0-py3-none-any.whl.metadata (1.9 kB)
Collecting certifi (from pdm)
  Obtaining dependency information for certifi from https://files.pythonhosted.org/packages/64/62/428ef076be88fa93716b576e4a01f919d25968913e817077a386fcbe4f42/certifi-2023.11.17-py3-none-any.whl.metadata
  Downloading certifi-2023.11.17-py3-none-any.whl.metadata (2.2 kB)
Collecting packaging!=22.0,>=20.9 (from pdm)
  Obtaining dependency information for packaging!=22.0,>=20.9 from https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl.metadata
  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting platformdirs (from pdm)
  Obtaining dependency information for platformdirs from https://files.pythonhosted.org/packages/31/16/70be3b725073035aa5fc3229321d06e22e73e3e09f6af78dcfdf16c7636c/platformdirs-4.0.0-py3-none-any.whl.metadata
  Downloading platformdirs-4.0.0-py3-none-any.whl.metadata (11 kB)
Collecting rich>=12.3.0 (from pdm)
  Obtaining dependency information for rich>=12.3.0 from https://files.pythonhosted.org/packages/be/be/1520178fa01eabe014b16e72a952b9f900631142ccd03dc36cf93e30c1ce/rich-13.7.0-py3-none-any.whl.metadata
  Downloading rich-13.7.0-py3-none-any.whl.metadata (18 kB)
Collecting virtualenv>=20 (from pdm)
  Obtaining dependency information for virtualenv>=20 from https://files.pythonhosted.org/packages/7f/1b/433ebb5530c48d70bebe1bf10ec7591af9f66847e34c4fcbca16d5eaaa0c/virtualenv-20.24.7-py3-none-any.whl.metadata
  Downloading virtualenv-20.24.7-py3-none-any.whl.metadata (4.5 kB)
Collecting pyproject-hooks (from pdm)
  Downloading pyproject_hooks-1.0.0-py3-none-any.whl (9.3 kB)
Collecting requests-toolbelt (from pdm)
  Downloading requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB ? eta 0:00:00
Collecting unearth>=0.12.1 (from pdm)
  Obtaining dependency information for unearth>=0.12.1 from https://files.pythonhosted.org/packages/c7/8b/1a8403d6323ef4477b61289c4aac75c7eac5724227509c8fd07255f997ee/unearth-0.12.1-py3-none-any.whl.metadata
  Downloading unearth-0.12.1-py3-none-any.whl.metadata (3.9 kB)
Collecting findpython<1.0.0a0,>=0.4.0 (from pdm)
  Obtaining dependency information for findpython<1.0.0a0,>=0.4.0 from https://files.pythonhosted.org/packages/51/6d/6347172d549ae6d9aca0406f29b4146b09b951a54524fb8bb253cb19be90/findpython-0.4.0-py3-none-any.whl.metadata
  Downloading findpython-0.4.0-py3-none-any.whl.metadata (5.3 kB)
Collecting tomlkit<1,>=0.11.1 (from pdm)
  Obtaining dependency information for tomlkit<1,>=0.11.1 from https://files.pythonhosted.org/packages/6e/43/159750d32481f16e34cc60090b53bc0a14314ad0c1f67a9bb64f3f3a0551/tomlkit-0.12.3-py3-none-any.whl.metadata
  Downloading tomlkit-0.12.3-py3-none-any.whl.metadata (2.7 kB)
Collecting shellingham>=1.3.2 (from pdm)
  Obtaining dependency information for shellingham>=1.3.2 from https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl.metadata
  Downloading shellingham-1.5.4-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting python-dotenv>=0.15 (from pdm)
  Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)
Collecting resolvelib>=1.0.1 (from pdm)
  Downloading resolvelib-1.0.1-py2.py3-none-any.whl (17 kB)
Collecting installer<0.8,>=0.7 (from pdm)
  Downloading installer-0.7.0-py3-none-any.whl (453 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 453.8/453.8 kB 9.6 MB/s eta 0:00:00
Collecting cachecontrol[filecache]>=0.13.0 (from pdm)
  Obtaining dependency information for cachecontrol[filecache]>=0.13.0 from https://files.pythonhosted.org/packages/1d/e3/a22348e6226dcd585d5a4b5f0175b3a16dabfd3912cbeb02f321d00e56c7/cachecontrol-0.13.1-py3-none-any.whl.metadata
  Downloading cachecontrol-0.13.1-py3-none-any.whl.metadata (3.0 kB)
Collecting truststore (from pdm)
  Obtaining dependency information for truststore from https://files.pythonhosted.org/packages/20/56/7811d5439b6a56374f274a8672d8f18b4deadadeb3a9f0c86424b98b6f96/truststore-0.8.0-py3-none-any.whl.metadata
  Downloading truststore-0.8.0-py3-none-any.whl.metadata (4.0 kB)
Collecting requests>=2.16.0 (from cachecontrol[filecache]>=0.13.0->pdm)
  Obtaining dependency information for requests>=2.16.0 from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting msgpack>=0.5.2 (from cachecontrol[filecache]>=0.13.0->pdm)
  Obtaining dependency information for msgpack>=0.5.2 from https://files.pythonhosted.org/packages/dc/c1/63903f30d51d165e132e5221a2a4a1bbfab7508b68131c871d70bffac78a/msgpack-1.0.7-cp312-cp312-win_amd64.whl.metadata
  Downloading msgpack-1.0.7-cp312-cp312-win_amd64.whl.metadata (9.4 kB)
Collecting filelock>=3.8.0 (from cachecontrol[filecache]>=0.13.0->pdm)
  Obtaining dependency information for filelock>=3.8.0 from https://files.pythonhosted.org/packages/81/54/84d42a0bee35edba99dee7b59a8d4970eccdd44b99fe728ed912106fc781/filelock-3.13.1-py3-none-any.whl.metadata
  Downloading filelock-3.13.1-py3-none-any.whl.metadata (2.8 kB)
Collecting markdown-it-py>=2.2.0 (from rich>=12.3.0->pdm)
  Obtaining dependency information for markdown-it-py>=2.2.0 from https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl.metadata
  Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
Collecting pygments<3.0.0,>=2.13.0 (from rich>=12.3.0->pdm)
  Obtaining dependency information for pygments<3.0.0,>=2.13.0 from https://files.pythonhosted.org/packages/97/9c/372fef8377a6e340b1704768d20daaded98bf13282b5327beb2e2fe2c7ef/pygments-2.17.2-py3-none-any.whl.metadata
  Downloading pygments-2.17.2-py3-none-any.whl.metadata (2.6 kB)
Collecting distlib<1,>=0.3.7 (from virtualenv>=20->pdm)
  Obtaining dependency information for distlib<1,>=0.3.7 from https://files.pythonhosted.org/packages/43/a0/9ba967fdbd55293bacfc1507f58e316f740a3b231fc00e3d86dc39bc185a/distlib-0.3.7-py2.py3-none-any.whl.metadata
  Downloading distlib-0.3.7-py2.py3-none-any.whl.metadata (5.1 kB)
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=12.3.0->pdm)
  Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm)
  Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/b6/7c/8debebb4f90174074b827c63242c23851bdf00a532489fba57fef3416e40/charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata
  Downloading charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl.metadata (34 kB)
Collecting idna<4,>=2.5 (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm)
  Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB ? eta 0:00:00
Collecting urllib3<3,>=1.21.1 (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm)
  Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/96/94/c31f58c7a7f470d5665935262ebd7455c7e4c7782eb525658d3dbf4b9403/urllib3-2.1.0-py3-none-any.whl.metadata
  Downloading urllib3-2.1.0-py3-none-any.whl.metadata (6.4 kB)
Downloading pdm-2.10.3-py3-none-any.whl (243 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 243.8/243.8 kB 7.5 MB/s eta 0:00:00
Downloading findpython-0.4.0-py3-none-any.whl (20 kB)
Downloading packaging-23.2-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB ? eta 0:00:00
Downloading rich-13.7.0-py3-none-any.whl (240 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 240.6/240.6 kB 15.4 MB/s eta 0:00:00
Downloading shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Downloading tomlkit-0.12.3-py3-none-any.whl (37 kB)
Downloading unearth-0.12.1-py3-none-any.whl (42 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.9/42.9 kB 2.0 MB/s eta 0:00:00
Downloading virtualenv-20.24.7-py3-none-any.whl (3.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.8/3.8 MB 26.7 MB/s eta 0:00:00
Downloading platformdirs-4.0.0-py3-none-any.whl (17 kB)
Downloading blinker-1.7.0-py3-none-any.whl (13 kB)
Downloading certifi-2023.11.17-py3-none-any.whl (162 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.5/162.5 kB 9.5 MB/s eta 0:00:00
Downloading truststore-0.8.0-py3-none-any.whl (16 kB)
Downloading distlib-0.3.7-py2.py3-none-any.whl (468 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 28.7 MB/s eta 0:00:00
Downloading filelock-3.13.1-py3-none-any.whl (11 kB)
Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 4.8 MB/s eta 0:00:00
Downloading msgpack-1.0.7-cp312-cp312-win_amd64.whl (223 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 223.3/223.3 kB 13.3 MB/s eta 0:00:00
Downloading pygments-2.17.2-py3-none-any.whl (1.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 37.7 MB/s eta 0:00:00
Downloading requests-2.31.0-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.6/62.6 kB ? eta 0:00:00
Downloading cachecontrol-0.13.1-py3-none-any.whl (22 kB)
Downloading charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl (100 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100.4/100.4 kB ? eta 0:00:00
Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB ? eta 0:00:00
Installing collected packages: resolvelib, distlib, urllib3, truststore, tomlkit, shellingham, python-dotenv, pyproject-hooks, pygments, platformdirs, packaging, msgpack, mdurl, installer, idna, filelock, charset-normalizer, certifi, blinker, virtualenv, requests, markdown-it-py, findpython, unearth, rich, requests-toolbelt, cachecontrol, pdm
  WARNING: The script dotenv.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pygmentize.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script normalizer.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script virtualenv.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script markdown-it.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script findpython.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script unearth.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script doesitcache.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pdm.exe is installed in 'C:\Users\Miles\.proto\tools\python\3.12.0\install\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed blinker-1.7.0 cachecontrol-0.13.1 certifi-2023.11.17 charset-normalizer-3.3.2 distlib-0.3.7 filelock-3.13.1 findpython-0.4.0 idna-3.4 installer-0.7.0 markdown-it-py-3.0.0 mdurl-0.1.2 msgpack-1.0.7 packaging-23.2 pdm-2.10.3 platformdirs-4.0.0 pygments-2.17.2 pyproject-hooks-1.0.0 python-dotenv-1.0.0 requests-2.31.0 requests-toolbelt-1.0.0 resolvelib-1.0.1 rich-13.7.0 shellingham-1.5.4 tomlkit-0.12.3 truststore-0.8.0 unearth-0.12.1 urllib3-2.1.0 virtualenv-20.24.7

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
[DEBUG 43:21.953] proto_core::tool_manifest:save  Saving manifest  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:21.954] starbase::app  Running shutdown phase with 1 systems
[TRACE 43:21.955] starbase_utils::fs:save  Creating file without truncating  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:21.956] starbase_utils::fs_lock:save  Locking file exclusively  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:21.956] starbase_utils::fs_lock:save  Writing file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:21.956] starbase_utils::fs_lock:save  Unlocking file exclusively  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
PS C:\Users\Miles\Projects\proto> cargo run -- run python -- -m pip install pdm --user
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s
     Running `target\debug\proto.exe run python -- -m pip install pdm --user`
[DEBUG 2023-11-21 21:43:46] proto  Running proto v0.23.2  args=["target\\debug\\proto.exe", "run", "python", "--", "-m", "pip", "install", "pdm", "--user"]
[TRACE 43:46.854] starbase::app  Running startup phase with 1 systems
[TRACE 43:46.854] starbase::app  Running analyze phase with 1 systems
[TRACE 43:46.854] starbase::app  Running execute phase with 2 systems
[DEBUG 43:46.856] proto_core::tool_loader:run  Traversing upwards to find a configured plugin  tool="python"
[DEBUG 43:46.856] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:46.856] starbase_utils::fs:run  Reading file  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:46.860] warpgate::loader:run  Creating plugin loader  cache_dir="C:\\Users\\Miles\\.proto\\plugins"
[TRACE 43:46.860] warpgate::loader:run  Loading plugin python  plugin="python"
[TRACE 43:46.860] warpgate::loader:run  Plugin already downloaded and cached  plugin="python" path="C:\\Users\\Miles\\.proto\\plugins\\python-06bc5da02073099597305a8691894325ec8d0a86f0772886be138331f73fba19.wasm"
[DEBUG 43:46.861] proto_core::tool_loader:run  Loading WASM plugin  source="C:\\Users\\Miles\\.proto\\plugins\\python-06bc5da02073099597305a8691894325ec8d0a86f0772886be138331f73fba19.wasm"
[TRACE 43:46.861] proto_core::tool_loader:run  Storing tool identifier  id="python"
[TRACE 43:46.861] proto_core::tool_loader:run  Storing user configuration  config={"auto_clean":false,"auto_install":false,"node_intercept_globals":true}
[TRACE 43:46.861] proto_core::tool_loader:run  Storing proto environment  env={"arch":"x64","os":"windows","home_dir":{"path":"/userhome/","virtual_prefix":"/userhome","real_prefix":"C:\\Users\\Miles"},"proto_dir":{"path":"/proto/","virtual_prefix":"/proto","real_prefix":"C:\\Users\\Miles\\.proto"}}
[DEBUG 43:46.861] proto_core::tool:run  Creating tool python and instantiating plugin
[DEBUG 43:46.861] proto_core::tool_manifest:run:load  Loading manifest.json  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:46.862] starbase_utils::fs:run:load  Opening file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:46.862] starbase_utils::fs_lock:run:load  Locking file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:46.862] starbase_utils::fs_lock:run:load  Unlocking file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[DEBUG 43:47.692] proto_core::tool:run  Created tool python and its WASM runtime
[TRACE 43:47.693] warpgate::plugin:run  Calling plugin function register_tool  plugin="python" input={"id":"python"}
[TRACE 43:47.693] warpgate::plugin:run  Called plugin function register_tool  plugin="python" output={"inventory":{},"name":"Python","plugin_version":"0.2.0","self_upgrade_commands":[],"type":"Language"}
[TRACE 43:47.694] proto_core::version_detector:run  Attempting to find local version from config files  tool="python"
[TRACE 43:47.694] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles\\Projects\\proto"
[DEBUG 43:47.694] proto_core::tools_config:run  Loading .prototools  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:47.695] starbase_utils::fs:run  Reading file  file="C:\\Users\\Miles\\Projects\\proto\\.prototools"
[TRACE 43:47.695] warpgate::plugin:run  Calling plugin function detect_version_files  plugin="python" input={}
[TRACE 43:47.695] warpgate::plugin:run  Called plugin function detect_version_files  plugin="python" output={"files":[".python-version"]}
[TRACE 43:47.696] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles\\Projects\\proto"
[TRACE 43:47.696] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles\\Projects"
[TRACE 43:47.696] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles\\Projects"
[TRACE 43:47.696] proto_core::version_detector:run  Checking directory  tool="python" dir="C:\\Users\\Miles"
[TRACE 43:47.696] proto_core::tool:run  Attempting to detect a version from directory  tool="python" dir="C:\\Users\\Miles"
[TRACE 43:47.697] proto_core::version_detector:run  Attempting to use global version from manifest  tool="python"
[DEBUG 43:47.697] proto_core::version_detector:run  Detected global version from manifest  tool="python" version="3.12.0" file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[DEBUG 43:47.697] proto_core::tool:run  Resolving a semantic version or alias  tool="python" initial_version="3.12.0"
[DEBUG 43:47.697] proto_core::tool:run  Resolved to 3.12.0 (without validation)  tool="python" version="3.12.0"
[DEBUG 43:47.697] proto_core::tool:run  Checking if tool is installed  tool="python" install_dir="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0"
[DEBUG 43:47.698] proto_core::tool:run  Tool has already been installed, locating binaries and shims  tool="python" install_dir="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0"
[DEBUG 43:47.698] proto_core::tool:run  Locating executable for tool  tool="python"
[TRACE 43:47.698] warpgate::plugin:run  Calling plugin function locate_executables  plugin="python" input={"context":{"tool_dir":{"path":"/proto/tools/python/3.12.0","virtual_prefix":"/proto","real_prefix":"C:\\Users\\Miles\\.proto"},"version":"3.12.0"}}
[TRACE 43:47.699] warpgate::plugin:run  Called plugin function locate_executables  plugin="python" output={"globals_lookup_dirs":["$HOME/.local/bin","$APPDATA/Roaming/Python312/Scripts","$APPDATA/Python312/Scripts"],"primary":{"exe_path":"install/python.exe","parent_exe_name":null},"secondary":{"pip":{"no_bin":true,"parent_exe_name":null,"shim_before_args":"-m pip"}}}
[DEBUG 43:47.699] proto_core::tool:run  Found an executable  tool="python" exe_path="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0\\install/python.exe"
[DEBUG 43:47.700] proto::commands::run:run  Running Python  bin="C:\\Users\\Miles\\.proto\\tools\\python\\3.12.0\\install/python.exe" args=["-m", "pip", "install", "pdm", "--user"]
Requirement already satisfied: pdm in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (2.10.3)
Requirement already satisfied: blinker in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.7.0)
Requirement already satisfied: certifi in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (2023.11.17)
Requirement already satisfied: packaging!=22.0,>=20.9 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (23.2)
Requirement already satisfied: platformdirs in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (4.0.0)
Requirement already satisfied: rich>=12.3.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (13.7.0)
Requirement already satisfied: virtualenv>=20 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (20.24.7)
Requirement already satisfied: pyproject-hooks in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.0.0)
Requirement already satisfied: requests-toolbelt in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.0.0)
Requirement already satisfied: unearth>=0.12.1 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.12.1)
Requirement already satisfied: findpython<1.0.0a0,>=0.4.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.4.0)
Requirement already satisfied: tomlkit<1,>=0.11.1 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.12.3)
Requirement already satisfied: shellingham>=1.3.2 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.5.4)
Requirement already satisfied: python-dotenv>=0.15 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.0.0)
Requirement already satisfied: resolvelib>=1.0.1 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (1.0.1)
Requirement already satisfied: installer<0.8,>=0.7 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.7.0)
Requirement already satisfied: cachecontrol[filecache]>=0.13.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.13.1)
Requirement already satisfied: truststore in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from pdm) (0.8.0)
Requirement already satisfied: requests>=2.16.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from cachecontrol[filecache]>=0.13.0->pdm) (2.31.0)
Requirement already satisfied: msgpack>=0.5.2 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from cachecontrol[filecache]>=0.13.0->pdm) (1.0.7)
Requirement already satisfied: filelock>=3.8.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from cachecontrol[filecache]>=0.13.0->pdm) (3.13.1)
Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from rich>=12.3.0->pdm) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from rich>=12.3.0->pdm) (2.17.2)
Requirement already satisfied: distlib<1,>=0.3.7 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from virtualenv>=20->pdm) (0.3.7)
Requirement already satisfied: mdurl~=0.1 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from markdown-it-py>=2.2.0->rich>=12.3.0->pdm) (0.1.2)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\miles\.proto\tools\python\3.12.0\install\lib\site-packages (from requests>=2.16.0->cachecontrol[filecache]>=0.13.0->pdm) (2.1.0)

[notice] A new release of pip is available: 23.2.1 -> 23.3.1
[notice] To update, run: python.exe -m pip install --upgrade pip
[TRACE 43:48.474] starbase::app  Running shutdown phase with 1 systems
[DEBUG 43:48.474] proto_core::tool_manifest:save  Saving manifest  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:48.475] starbase_utils::fs:save  Creating file without truncating  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:48.476] starbase_utils::fs_lock:save  Locking file exclusively  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:48.476] starbase_utils::fs_lock:save  Writing file  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
[TRACE 43:48.476] starbase_utils::fs_lock:save  Unlocking file exclusively  file="C:\\Users\\Miles\\.proto\\tools\\python\\manifest.json"
PS C:\Users\Miles\Projects\proto>

[rotu]

But I'm not sure how to fix PATH resolving, since this path (and --user path) will change between python versions.

I don’t think we should use the --user flag - the files are site-specific.

That’s why I suggested —-prefix=/path/to/globals (though you’re probably right to put this in the invocation of pip rather than the ini), which lets you select the direct parent of the scripts folder.

The docs mention that packages so installed are tied to the version of python that it pip-installed under. I think that’s what we want!

[milesj]

Is this still an issue after the latest patch?

[rotu]

Nope! this issue is fixed!

[milesj]

Niiice