Expected Behavior
The resource marker files in ament_python
packages do not interfere with other Python tools.
Actual Behavior
On Windows, if I use colcon build --merge-install --symlink-install
to build a workspace with ament_python
packages and source the workspace's setup.bat
, the package's resource
folder is picked up as an importable Python package.
This causes errors in other packages.
(humble) c:\Code\dbgdump\test_ws>python -c "import resource; print(resource.__spec__)"
ModuleSpec(name='resource', loader=<_frozen_importlib_external._NamespaceLoader object at 0x0000020E1B72EC10>, submodule_search_locations=_NamespacePath(['c:\\Code\\dbgdump\\test_ws\\build\\test_pkg\\resource']))
Note that the imported module is in the workspace's build folder:
c:\\Code\\dbgdump\\test_ws\\build\\test_pkg\\resource
The resource
package is supposed to be a Unix-specific Python package that I guess is in the standard library.
Several other Python packages test only for an ImportError
for the resource
package as a test for non-Unix systems. For example:
However, if there is an importable resource
package that isn't https://docs.python.org/3/library/resource.html, it raises an AttributeError
instead.
I first encountered this issue because Jupyter notebooks seem to use the resource
module, which means I can't properly install and use Jupyter notebooks in a fully-sourced ROS2 environment on Windows ROS2.
See jupyter/notebook#5817 (comment) for earlier details.
Detailed Reproduction
I did a step-by-step reproduction in an otherwise empty workspace below.
The problematic outcome is annotated with ❌... the errors annotated with ✅ represent expected behavior on Windows.
It's specifically the use of --symlink-install
that triggers the behavior.
(humble) c:\Code\dbgdump\test_ws>cd src
(humble) c:\Code\dbgdump\test_ws\src>ros2 pkg create test_pkg --build-type ament_python
(humble) c:\Code\dbgdump\test_ws\src>tree /f
Folder PATH listing for volume Windows-SSD
Volume serial number is BC74-9D52
C:.
└───test_pkg
│ package.xml
│ setup.cfg
│ setup.py
│
├───resource
│ test_pkg
│
├───test
│ test_copyright.py
│ test_flake8.py
│ test_pep257.py
│
└───test_pkg
__init__.py
(humble) c:\Code\dbgdump\test_ws\src>cd ..
(humble) c:\Code\dbgdump\test_ws>colcon build --merge-install --symlink-install
# Before we source the workspace, we get a ModuleNotFoundError, which is fine
# This is considered a sign that we're on Windows for other packages looking for the resource module
# like https://github.com/prometheus/client_python
(humble) c:\Code\dbgdump\test_ws>python -c "import resource"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'resource' ✅
# however, when we source the workspace, we have an issue
(humble) c:\Code\dbgdump\test_ws>install\setup.bat
(humble) c:\Code\dbgdump\test_ws>python -c "import resource"
# nothing happens here, because it exists! What is it? The resource folder from test_pkg
(humble) c:\Code\dbgdump\test_ws>python -c "import resource; print(resource.__path__)"
_NamespacePath(['c:\\Code\\dbgdump\\test_ws\\build\\test_pkg\\resource'])
# Packages that test for ImportError as a platform selector will now fail.
# We get an AttributeError if we use methods that exist in the Unix resource module
(humble) c:\Code\dbgdump\test_ws>python -c "import resource; print(resource.getpagesize())"
Traceback (most recent call last):
File "<string>", line 1, in <module>
AttributeError: module 'resource' has no attribute 'getpagesize' ❌
# We're fine if we just use colcon build
(humble) c:\Code\dbgdump\test_ws>rd /s /q build install log
(humble) c:\Code\dbgdump\test_ws>colcon build
(humble) c:\Code\dbgdump\test_ws>install\setup.bat
(humble) c:\Code\dbgdump\test_ws>python -c "import resource; print(resource.getpagesize())"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'resource' ✅
# It seems like colcon build --merge-install is fine as well
(humble) c:\Code\dbgdump\test_ws>rd /s /q build install log
(humble) c:\Code\dbgdump\test_ws>colcon build --merge-install
(humble) c:\Code\dbgdump\test_ws>install\setup.bat
(humble) c:\Code\dbgdump\test_ws>python -c "import resource; print(resource.getpagesize())"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'resource' ✅
Machine Info
I have ROS2 installed via Robostack
Edition Windows 11 Home
Version 22H2
Installed on 5/12/2022
OS build 22622.601
Python 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:50:36) [MSC v.1929 64 bit (AMD64)] on win32
(humble) c:\Code\dbgdump\test_ws>mamba list | findstr ament-index
ros-humble-ament-index-cpp 1.4.0 py39he8739fe_1 robostack-humble
ros-humble-ament-index-python 1.4.0 py39h4c0ac80_1 robostack-humble
I'm considering some upstream PRs to fix the tests in the affected packages too, but since resource
is a standard library package, it seems pretty valid to presume that no one would shadow its name in Windows unless their intent was to provide the same API for a special case.
I have not reproduced this on all my Windows machines, so I'll report back if I find that this is something peculiar about this machine and ROS2 installation.
Any advice for workarounds is appreciated. For now I'm just going to locally hack the offending Python libraries to test for AttributeError
as well.