python / importlib_metadata

Backport of the importlib.metadata module

Library to access the metadata for a Python package.

This package supplies third-party access to the functionality of importlib.metadata including improvements added to subsequent Python versions.

Compatibility

New features are introduced in this third-party library and later merged into CPython. The following table indicates which versions of this library were contributed to different versions in the standard library:

importlib_metadata	stdlib
7.0	3.13
6.5	3.12
4.13	3.11
4.6	3.10
1.4	3.8

Usage

See the online documentation for usage details.

Finder authors can also add support for custom package installers. See the above documentation for details.

Caveats

This project primarily supports third-party packages installed by PyPA tools (or other conforming packages). It does not support:

• Packages in the stdlib.
• Packages installed without metadata.

Project details

• Project home: https://github.com/python/importlib_metadata
• Report bugs at: https://github.com/python/importlib_metadata/issues
• Code hosting: https://github.com/python/importlib_metadata
• Documentation: https://importlib-metadata.readthedocs.io/

For Enterprise

Available as part of the Tidelift Subscription.

This project and the maintainers of thousands of other packages are working with Tidelift to deliver one enterprise subscription that covers all of the open source you use.

Learn more.

importlib_metadata's Issues

resolve() doesn't support bare-module entry points

In GitLab by @jaraco on Nov 21, 2018, 15:51

Consider this routine that loads entry points for keyring. It allows loading the entry point with an optional callable indicated.

The resolve function in importlib_metadata doesn't support this use-case because it requires that the : be supplied, whereas in pkg_resources and entrypoints, the callable is optional (and the module is returned).

importlib_metadata should match this expectation.

Final review before 0.8 release

In GitLab by @jaraco on Dec 30, 2018, 16:49

In order to keep things moving, I've gone ahead and merged the remaining merge requests on the 0.8 milestone (and resolved the conflicts).

I believe this is ready to go, although there may be some editorial or other concerns that Barry would like to address before making the changes public. Please feel free to review the merge requests or resulting code and make whatever changes you see fit. When you're ready to cut the release, either of us can just tag the master with 0.8 and the release should happen automatically.

Readme and API still exposes Distribution class

In GitLab by @jaraco on Sep 16, 2018, 03:06

The README still presents Distribution.from_named_module as a recommended way to resolve a __version__ from inside a module... and there's no simple function that implements that behavior.

My goal in that section of the readme and the example usage was to provide the best recommended way to set __version__ based on (something in) globals() in a top-level module.

Maybe that's unnecessary... as the version() function does it more elegantly (and bypasses the module_name -> module -> __dist_name__) traversal (by requiring the developer to supply the distribution name or module directly). I guess I'd prefer if one could use simply __name__, such that one paste in the boilerplate:

__version__ = importlib_metadata.version(__name__)

Rather than having to supply documentation explaining to a maintainer what value to supply for version:

__version__ = importlib_metadata.version(distribution_name_of_your_project)

If we don't need for __name__ to resolve to a module or dist_name, it may be possible to also remove the whole bit about __dist_name__. Perhaps all operations should require that the distribution name be supplied and never resolve through an imported or named module.

Add mechanical releases

In GitLab by @jaraco on Sep 10, 2018, 02:34

Let's find a way to make tagged releases automatically released to PyPI.

KeyError on import - There is no item named 'importlib_metadata-0.6.dist-info/METADATA' in the archive

In GitLab by @fizwit on Dec 5, 2018, 22:01

built importlib_metadata from source and it fails to run.

$ python
Python 3.6.7 (default, Dec  5 2018, 09:25:16)
[GCC 5.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/site-packages/importlib_metadata-0.6-py3.6.egg/importlib_metadata/__init__.py", line 17, in <module>
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/site-packages/importlib_metadata-0.6-py3.6.egg/importlib_metadata/api.py", line 105, in version
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/site-packages/importlib_metadata-0.6-py3.6.egg/importlib_metadata/api.py", line 86, in distribution
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/site-packages/importlib_metadata-0.6-py3.6.egg/importlib_metadata/api.py", line 48, in from_name
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/site-packages/importlib_metadata-0.6-py3.6.egg/importlib_metadata/_hooks.py", line 126, in find_distribution
  File "/app/easybuild/software/Python/3.6.7-foss-2016b-fh1/lib/python3.6/zipfile.py", line 1304, ingetinfo
    'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'importlib_metadata-0.6.dist-info/METADATA' in the archive"

Remove `read_text()`

In GitLab by @warsaw on Dec 31, 2018, 20:21

Given that files() returns the list of files in a distribution, and each file is represented by a PackagePath instance, and that instance already has a read_text() method on it, I don't think we need the top-level read_text() method. IMHO, that extra convenience isn't worth it. @jaraco what do you think?

Support for package discovery/enumeration

In GitLab by @jaraco on Nov 21, 2018, 15:42

Although importlib_metadata supplies support for things like entry_points, each of these calls requires as input the name of the distribution supplying that metadata. But many use-cases require loading all entry points of a particular name from all installed distributions, and without a mechanism to list all installed distributions, it's not possible to support this use-case.

Should entry_points() return the actual callables?

In GitLab by @warsaw on Sep 11, 2018, 14:59

Right now, entry_points() returns a ConfigParser loaded from the entry_points.txt file. Maybe it would be more useful to resolve the entry points and return the actual callables or raise ImportError or AttributeError if those can't be found.

The other option would be to have another API such as load_entry_points() which does this, but OTOH, I think the above behavior is more useful (and it's always possible to do essentially what entry_points() currently does; it's ~4 lines of actual Python code.

Test runs fail on second invocation

In GitLab by @jaraco on Dec 5, 2018, 24:46

Running tests the first time works fine:

importlib_metadata feature/file-enum $ tox -e py37-cov -r
py37-cov recreate: /Users/jaraco/code/public/importlib_metadata/.tox/py37-cov
py37-cov installdeps: coverage>=4.5, importlib_resources
py37-cov develop-inst: /Users/jaraco/code/public/importlib_metadata
py37-cov installed: You are using pip version 10.0.1, however version 18.1 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,coverage==4.5.2,-e git+gl://python-devs/importlib_metadata@59f0f3ad4c32d2b739d6f32658e9e4e04bf14ab9#egg=importlib_metadata,importlib-resources==1.0.2,zipp==0.3.2
py37-cov run-test-pre: PYTHONHASHSEED='2863425022'
py37-cov runtests: commands[0] | python -m coverage run --rcfile=/Users/jaraco/code/public/importlib_metadata/coverage.ini -m unittest discover
............/Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/lib/python3.7/site-packages/pip/_vendor/urllib3/util/selectors.py:14: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import namedtuple, Mapping
/Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/lib/python3.7/site-packages/pip/_vendor/urllib3/_collections.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping, MutableMapping
/Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/lib/python3.7/site-packages/pip/_vendor/pyparsing.py:3245: DeprecationWarning: Using or importing the ABCs from 'collections'instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  elif isinstance( exprs, collections.Iterable ):
.........
----------------------------------------------------------------------
Ran 21 tests in 0.434s

OK
py37-cov runtests: commands[1] | python -m coverage combine --rcfile=/Users/jaraco/code/public/importlib_metadata/coverage.ini
py37-cov runtests: commands[2] | python -m coverage html --rcfile=/Users/jaraco/code/public/importlib_metadata/coverage.ini
py37-cov runtests: commands[3] | python -m coverage xml --rcfile=/Users/jaraco/code/public/importlib_metadata/coverage.ini
py37-cov runtests: commands[4] | python -m coverage report -m --rcfile=/Users/jaraco/code/public/importlib_metadata/coverage.ini --fail-under=100
Name                                  Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------------
importlib_metadata/__init__.py            5      0      0      0   100%
importlib_metadata/_hooks.py             58      0     12      0   100%
importlib_metadata/api.py                68      0      8      0   100%
importlib_metadata/docs/__init__.py       0      0      0      0   100%
---------------------------------------------------------------------------------
TOTAL                                   131      0     20      0   100%
________________________________________________________________________________________ summary ________________________________________________________________________________________
  py37-cov: commands succeeded
  congratulations :)

Following the acceptance of !9, running the tests again, the tests fail when tox tries to get the project name:

importlib_metadata feature/file-enum $ tox -e py37-cov -v
using tox.ini: /Users/jaraco/code/public/importlib_metadata/tox.ini
using tox-3.5.3 from /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tox/__init__.py
py37-cov reusing: /Users/jaraco/code/public/importlib_metadata/.tox/py37-cov
  /Users/jaraco/code/public/importlib_metadata$ /Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/bin/python /Users/jaraco/code/public/importlib_metadata/setup.py --name
Traceback (most recent call last):
  File "/Users/jaraco/code/public/importlib_metadata/setup.py", line 4, in <module>
    import setuptools_scm
ModuleNotFoundError: No module named 'setuptools_scm'
  /Users/jaraco/code/public/importlib_metadata$ /Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/bin/python -m pip freeze >/Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/log/py37-cov-5.log
py37-cov installed: You are using pip version 10.0.1, however version 18.1 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,coverage==4.5.2,-e git+gl://python-devs/importlib_metadata@bbea8c68fef5bd32bffe48f4580743b8af7ed483#egg=importlib_metadata,importlib-resources==1.0.2,zipp==0.3.2
________________________________________________________________________________________ summary ________________________________________________________________________________________
ERROR:   py37-cov: InvocationError for command '/Users/jaraco/code/public/importlib_metadata/.tox/py37-cov/bin/python /Users/jaraco/code/public/importlib_metadata/setup.py --name' (exited with code 1)

This issue is the same as the one I reported with tox a few days ago. I'm going to renew that conversation.

Document usage

In GitLab by @warsaw on Sep 11, 2018, 24:36

Coverage tests fail to run if space in CWD

In GitLab by @jaraco on Feb 24, 2019, 15:35

Yesterday, Steven Ma was running tests using tox on Windows with a space in the current working directory. When doing so, we discovered that the coverage tests will fail to pass the parameters correctly and fail to run.

expose `distributions()` in `importlib_metadata`?

In GitLab by @asottile on Feb 13, 2019, 04:01

Currently it is present in importlib_metadata.api, and seems like a useful api but it isn't exposed in __init__

Backport to Python 2.7

In GitLab by @jaraco on May 23, 2018, 14:10

Revamp entry point support

In GitLab by @jaraco on Dec 7, 2018, 20:37

I realized today that the entry point functionality exposed by importlib_metadata isn't that useful because it focuses on supplying entry points only for one package.

The whole purpose of entry points is the "inversion of control" - that suppliers of entry points can come from any number of packages and consumers of entry points can also be in various packages, but generally they define the expectation for that named "group" of entry points.

Therefore, the interface provided by entrypoints and pkg_resources is really the interface that clients need - a way to query for all entry points of a given group and resolve/load them.

In my opinion, importlib_metadata should (in my order of preference):

• provide a compatible interface to entry points that obviates entrypoints and pkg_resources (for entry points),
• provide a low-level interface for enumerating all entry points over all packages, leaving it to another package to provide a usable interface for entry points,
• get out of the business of doing anything about entry points.

The reason I don't like the second option is it doesn't really help wean users off of the third-party packages. Entry points would still be unusable in the Python 3.8 stdlib without another third-party package. I would go for that, though, if that were Takluyver's preference.

I don't think there's much value in providing the current entry_points function as it is, because most use cases won't find it usable in that form (without clumsily first enumerating all packages and then filtering out the ones that you don't need).

I'll plan to draft an implementation of what I have in mind for the first option. In the meantime, what are the motivations for not providing an interface that obviates the need for entrypoints and pkg_resources.iter_entry_points?

Support wheels

In GitLab by @warsaw on Sep 11, 2018, 24:31

I'm opening this as a separate ticket. We don't currently support wheels, but like with importlib_resources we should. I'll do this after adding the 2.7 support, because... fun!

Distribution lookup is case-sensitive

In GitLab by @jaraco on Nov 1, 2018, 20:12

~ $ rwt -q importlib_metadata==0.6 -- -c "import importlib_metadata; print(importlib_metadata.version('PyJWT'))"
1.6.4
~ $ rwt -q importlib_metadata==0.6 -- -c "import importlib_metadata; print(importlib_metadata.version('pyjwt'))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/rwt-_mwk2v6o/importlib_metadata/api.py", line 105, in version
    return distribution(package).version
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/rwt-_mwk2v6o/importlib_metadata/api.py", line 86, in distribution
    return Distribution.from_name(package)
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/rwt-_mwk2v6o/importlib_metadata/api.py", line 52, in from_name
    raise PackageNotFoundError(name)
importlib_metadata.api.PackageNotFoundError: pyjwt

Probably the search should be case-insensitive.

Supply entry points?

In GitLab by @jaraco on May 18, 2018, 19:56

As this package supplies metadata, it's also well-suited as a foundation for serving entry points. Is that something this module could/should support? It surely would be nice to have in the stdlib as well? Otherwise, a library like entrypoints will need to duplicate functionality in this lib or rely on it.

Perhaps that latter option is the best course of action - to keep entrypoint support separate for now, but and consider incorporating that into stdlib as a separate action and maybe separate module (importlib.entrypoints?).

Thoughts?

Do we need to expose the `Distribution` class?

In GitLab by @warsaw on Sep 14, 2018, 23:14

As I've been working on bpo-34632 it dawns on me that maybe we don't need to actually expose the Distribution class or the distribution() function. They essentially provide functionality that we can hide behind the entry_points(), metadata(), and version() functions. I'm inclined to keep the API simple and turn the former two into internal API, meaning that I wouldn't add documentation for them to Python 3.8, nor would I add them to the __all__.

@jaraco What do you think?

Add TWINE_USERNAME and TWINE_PASSWORD to group CI/CD settings

In GitLab by @jaraco on Sep 11, 2018, 20:06

While working on #8, I've identified a way to pass credentials to twine and upload the dists. However, for the process to work, PyPI credentials need to be supplied.

Unlike Travis CI, GitLab CI does not have support for encrypted variables (variables encrypted on an individual's computer decryptable only using a key private to the repository that's inaccessible even to the devs on the project).

Instead, GitLab CI has secret variables, which provide some level of secrecy, but are readily exposed to other members with the same access.

I have access to configure these variables for importlib_metadata but probably not for other projects in python-devs, which is probably where these credentials should be installed.

While working out this ticket, I found some really buggy behavior on that page. In fact, I think I was able to access that page even though I'm possibly not supposed to be (I can't see the menu, but I can set and retrieve values, if I first navigate through another group where I have access).

WheelMetadataFinder is searching wrong name

In GitLab by @jaraco on Nov 4, 2018, 18:33

Consider:

draft $ pip download --no-deps path.py
Collecting path.py
  Using cached https://files.pythonhosted.org/packages/ea/82/d710e0727980ac1dccd2887f866e84c614a68c0a8d0c3aee606997e5eeeb/path.py-11.5.0-py2.py3-none-any.whl
  Saved ./path.py-11.5.0-py2.py3-none-any.whl
Successfully downloaded path.py
draft $ env PYTHONPATH=./path.py-11.5.0-py2.py3-none-any.whl rwt importlib_metadata -- -S
Collecting importlib_metadata
  Using cached https://files.pythonhosted.org/packages/d5/e4/49da554657aebdfe3e26beefc605fabd5dfa054b1864f8a2c527537a3dba/importlib_metadata-0.6-py2.py3-none-any.whl
Installing collected packages: importlib-metadata
Successfully installed importlib-metadata-0.6
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28)
[Clang 6.0 (clang-600.0.57)] on darwin
>>> import importlib_metadata
>>> importlib_metadata.distribution('path.py')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/rwt-gk1ox5sl/importlib_metadata/api.py", line 86, in distribution
    return Distribution.from_name(package)
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/rwt-gk1ox5sl/importlib_metadata/api.py", line 52, in from_name
    raise PackageNotFoundError(name)
importlib_metadata.api.PackageNotFoundError: path.py

The metadata isn't found if the distribution name is used. It seems the WheelMetadataFinder is expecting a module name:

>>> importlib.distribution('path')
>>> importlib_metadata.distribution('path')
<importlib_metadata._hooks.WheelDistribution object at 0x102ad8780>

That's inconsistent with the behavior of the MetadataPathFinder, and violates the user's expectations (that metadata should be readable without importing any modules).

Entry points can't include extras

In GitLab by @jaraco on Dec 11, 2018, 02:49

In pkg_resources and entrypoints, entry points are allowed to specify extras. Perhaps these should also be supported. For example:

my entrypoint = mymodule:attr [extra1, extra2]

This package should at least support parsing those entry points.

Make Distribution class public

In GitLab by @warsaw on Dec 19, 2018, 22:47

Add support for file enumeration and discovery

In GitLab by @jaraco on Sep 18, 2018, 12:02

I was recently helping a colleague understand the relationship between a distribution and the files it installed, and I realized there's not really a good, program-friendly answer on how to find files related to a (distribution) package.

This project is probably a good place to present that functionality.

I'd like to have a handful of functions that:

1. Given a module or script or data file as installed by a Python package, return the distribution for that package.
2. Enumerate all files in a distribution package.

Entry points can't resolve multiple attributes

In GitLab by @jaraco on Dec 11, 2018, 02:46

In pkg_resources and entrypoints, an entry point object is allowed to specify multiple attributes. For example:

my entrypoint = mymodule:Class.attr

This package should support resolving attr from mymodule.Class.

Entry points fail to parse with indented definition on Python 2.7

In GitLab by @jaraco on Dec 11, 2018, 03:23

In this job, I learned that the Python 2.7 ConfigParser fails to parse entry points as defined in Jinja2 due to excess indentation of the definition:

Traceback (most recent call last):
  File "importlib_metadata/tests/test_api.py", line 39, in test_entry_points
    scripts = importlib_metadata.entry_points()['console_scripts']
  File "importlib_metadata/api.py", line 202, in entry_points
    ordered = sorted(eps, key=by_group)
  File "importlib_metadata/api.py", line 200, in <genexpr>
    dist.entry_points for dist in distributions())
  File "importlib_metadata/api.py", line 146, in entry_points
    return EntryPoint._from_text(self.read_text('entry_points.txt'))
  File "importlib_metadata/api.py", line 70, in _from_text
    config.readfp(io.BytesIO(text))
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 324, in readfp
    self._read(fp, filename)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 512, in _read
    raise MissingSectionHeaderError(fpname, lineno, line)
MissingSectionHeaderError: File contains no section headers.
file: <???>, line: 2
'    [babel.extractors]\n'

Add support for reading dependencies

In GitLab by @jaraco on Sep 26, 2018, 24:57

I recently stumbled onto another case where pkg_resources supplies some important metadata, the dependencies for a package.

~ $ pkg_resources.get_distribution('jaraco.functools').requires()
[Requirement.parse('more-itertools')]

I believe importlib_metadata should probably provide at least a list of requirements if not also possibly parsing those requirements (although the latter might only work if the third-party 'packaging' library is available).

Local distribution not found for package using the 'src' layout

In GitLab by @jaraco on Jan 29, 2019, 01:42

In the Distribution.find_local method, I assumed, apparently incorrectly, that performing a setup.py develop or pip install -e . on a package would generate the metadata in . as it does on projects I've maintained.

I recently inherited the configparser project, which uses the 'src' layout, and I discover that in that project, setuptools generates the egg-info metadata in 'src', and in this condition, find_local gets a ValueError because there's no metadata found in .:

configparser master $ python setup.py egg_info                                                                                                                                           
running egg_info
creating src/configparser.egg-info
writing src/configparser.egg-info/PKG-INFO
writing dependency_links to src/configparser.egg-info/dependency_links.txt
writing requirements to src/configparser.egg-info/requires.txt
writing top-level names to src/configparser.egg-info/top_level.txt
writing manifest file 'src/configparser.egg-info/SOURCES.txt'
reading manifest file 'src/configparser.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/configparser.egg-info/SOURCES.txt'
configparser master $ pip-run -q importlib_metadata==0.8                                                                                                                                 
Python 3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) 
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib_metadata
>>> importlib_metadata.api.local_distribution()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/importlib_metadata/api.py", line 326, in local_distribution
    return Distribution.find_local()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/importlib_metadata/api.py", line 186, in find_local
    dist, = dists
ValueError: not enough values to unpack (expected 1, got 0)
>>> 

I'm not sure what to do... because in the general case, any package can decide to have its metadata built wherever it likes (there's nothing special about 'src').

I think the recommendation from PEP 517 and PEP 518 is that tools that need metadata for a package that's on the file system, the way to do that is to call the hooks. I think importlib_metadata should probably do this rather than expecting the metadata to be in ..

I'm also tempted to find metadata in . or src as a short-term, hacky workaround. But before I do that, I'll try to see if I can make the pep517 package help out in this regard.

flufl.flake8 failing with TypeError

In GitLab by @jaraco on Dec 5, 2018, 19:36

I've added a few commits to master as a basis for #29 (4434eaa, 78bc056, and c546d9d), and now the qa tests are crashing:

importlib_metadata master $ tox -e qa -r
qa recreate: /Users/jaraco/code/public/importlib_metadata/.tox/qa
qa installdeps: mypy, flake8, flufl.flake8
qa develop-inst: /Users/jaraco/code/public/importlib_metadata
qa installed: You are using pip version 10.0.1, however version 18.1 is available.,You should consider upgrading via the 'pip install --upgrade pip' command.,flake8==3.6.0,flufl.flake8==0.9,-e git+gl://python-devs/importlib_metadata@c546d9d40cb3eea26fe6939b33f8271f25d685be#egg=importlib_metadata,mccabe==0.6.1,mypy==0.641,mypy-extensions==0.4.1,pycodestyle==2.4.0,pyflakes==2.0.0,typed-ast==1.1.0
qa run-test-pre: PYTHONHASHSEED='1556744124'
qa runtests: commands[0] | python -m flake8 importlib_metadata
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/__main__.py", line 4, in <module>
    cli.main()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/main/cli.py", line 16, in main
    app.run(argv)
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/main/application.py", line 412, in run
    self._run(argv)
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/main/application.py", line 400, in _run
    self.run_checks()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/main/application.py", line 318, in run_checks
    self.file_checker_manager.run()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/checker.py", line 340, in run
    self.run_serial()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/checker.py", line 324, in run_serial
    checker.run_checks()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/checker.py", line 612, in run_checks
    self.run_ast_checks()
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flake8/checker.py", line 520, in run_ast_checks
    for (line_number, offset, text, check) in runner:
  File "/Users/jaraco/code/public/importlib_metadata/.tox/qa/lib/python3.7/site-packages/flufl/flake8/imports.py", line 141, in run
    if last_import.module > record.module:
TypeError: '>' not supported between instances of 'str' and 'NoneType'
ERROR: InvocationError for command '/Users/jaraco/code/public/importlib_metadata/.tox/qa/bin/python -m flake8 importlib_metadata' (exited with code 1)

I suspect an issue in flufl.flake8.

Create a DistributionFinder abstract base class

In GitLab by @jaraco on Feb 20, 2019, 14:36

In this comment, @brettcannon suggests that the "DistributionFinder" behavior be documented in an ABC. I had the same instinct. Let's see if we can't create such an ABC and adapt these docs to reference that ABC.

Review implementation and cut first experimental release

In GitLab by @jaraco on May 23, 2018, 13:19

I'd like to cut a release and start experimenting gathering feedback on real-world feedback. I plan to start with an 0.1 release and iterate quickly with an explicit disclaimer about not guaranteeing any stable interface until 1.0.

Before I do that, Barry suggested I get feedback from @brettcannon on the basic implementation. Brett, will you have time to review?

• Review from Brett

The README has a brief overview of the supported API and limitations, probably a good place to start.

@warsaw Anything else you recommend before proceeding with the first release?

Tests not run on Python 2

In GitLab by @jaraco on Dec 5, 2018, 19:16

Why doesn't CI run the tests against Python 2?

Cannot get metadata from .egg-info files

In GitLab by @chrisjbillington on Feb 10, 2019, 23:18

I'm using importlib_metadata 0.8 on Python 3.7 on Arch Linux.

The following example shows that pyserials metadata can't be found by importlib_metadata, even though pkg_resources has it in the working_set:

import pkg_resources
import importlib_metadata

for dist in pkg_resources.working_set:
    if dist.project_name == 'pyserial':
        print(dist.path)
        print(dist.version)

importlib_metadata.version('pyserial')

this gives:

/usr/lib/python3.7/site-packages/pyserial-3.4-py3.7.egg-info
3.4
Traceback (most recent call last):
  File "196.py", line 9, in <module>
    importlib_metadata.version('pyserial')
  File "/usr/lib/python3.7/site-packages/importlib_metadata/api.py", line 345, in version
    return distribution(package).version
  File "/usr/lib/python3.7/site-packages/importlib_metadata/api.py", line 310, in distribution
    return Distribution.from_name(package)
  File "/usr/lib/python3.7/site-packages/importlib_metadata/api.py", line 158, in from_name
    raise PackageNotFoundError(name)
importlib_metadata.api.PackageNotFoundError: pyserial

The thing about pyserial is that its import name ('serial') does not match its project name ('pyserial'), which might have something to do with it. importlib_metadata.version gives the same exception if I input 'serial', however.

KeyError when importlib_metadata is loaded as an egg

In GitLab by @jaraco on Oct 15, 2018, 10:20

In this job, you can see the introduction of import importlib_metadata to a project that uses setup.py test will result in an exception:

running egg_info
creating mongo_connector.egg-info
writing mongo_connector.egg-info/PKG-INFO
writing entry points to mongo_connector.egg-info/entry_points.txt
writing requirements to mongo_connector.egg-info/requires.txt
writing dependency_links to mongo_connector.egg-info/dependency_links.txt
writing top-level names to mongo_connector.egg-info/top_level.txt
writing manifest file 'mongo_connector.egg-info/SOURCES.txt'
reading manifest file 'mongo_connector.egg-info/SOURCES.txt'
writing manifest file 'mongo_connector.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 140, in <module>
    **extra_opts
  File "/opt/python/3.4.6/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/python/3.4.6/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/python/3.4.6/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/command/test.py", line 211, in run
    self.run_tests()
  File "/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/command/test.py", line 234, in run_tests
    **exit_kwarg
  File "/opt/python/3.4.6/lib/python3.4/unittest/main.py", line 92, in __init__
    self.parseArgs(argv)
  File "/opt/python/3.4.6/lib/python3.4/unittest/main.py", line 139, in parseArgs
    self.createTests()
  File "/opt/python/3.4.6/lib/python3.4/unittest/main.py", line 146, in createTests
    self.module)
  File "/opt/python/3.4.6/lib/python3.4/unittest/loader.py", line 157, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/opt/python/3.4.6/lib/python3.4/unittest/loader.py", line 157, in <listcomp>
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/opt/python/3.4.6/lib/python3.4/unittest/loader.py", line 128, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/home/travis/virtualenv/python3.4.6/lib/python3.4/site-packages/setuptools/command/test.py", line 43, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/opt/python/3.4.6/lib/python3.4/unittest/loader.py", line 116, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
  File "/home/travis/build/yougov/mongo-connector/tests/test_command_replication.py", line 26, in <module>
    from mongo_connector.doc_managers.doc_manager_base import DocManagerBase
  File "/home/travis/build/yougov/mongo-connector/mongo_connector/doc_managers/doc_manager_base.py", line 19, in <module>
    from mongo_connector.connector import get_mininum_mongodb_version
  File "/home/travis/build/yougov/mongo-connector/mongo_connector/connector.py", line 34, in <module>
    from mongo_connector import config, constants, errors, util
  File "/home/travis/build/yougov/mongo-connector/mongo_connector/config.py", line 20, in <module>
    from mongo_connector.constants import __version__
  File "/home/travis/build/yougov/mongo-connector/mongo_connector/constants.py", line 16, in <module>
    import importlib_metadata
  File "<frozen importlib._bootstrap>", line 2237, in _find_and_load
  File "<frozen importlib._bootstrap>", line 2226, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1191, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1161, in _load_backward_compatible
  File "/home/travis/build/yougov/mongo-connector/.eggs/importlib_metadata-0.6-py3.4.egg/importlib_metadata/__init__.py", line 17, in <module>
  File "/home/travis/build/yougov/mongo-connector/.eggs/importlib_metadata-0.6-py3.4.egg/importlib_metadata/api.py", line 105, in version
  File "/home/travis/build/yougov/mongo-connector/.eggs/importlib_metadata-0.6-py3.4.egg/importlib_metadata/api.py", line 86, in distribution
  File "/home/travis/build/yougov/mongo-connector/.eggs/importlib_metadata-0.6-py3.4.egg/importlib_metadata/api.py", line 48, in from_name
  File "/home/travis/build/yougov/mongo-connector/.eggs/importlib_metadata-0.6-py3.4.egg/importlib_metadata/_hooks.py", line 126, in find_distribution
  File "/opt/python/3.4.6/lib/python3.4/zipfile.py", line 1084, in getinfo
    'There is no item named %r in the archive' % name)
KeyError: "There is no item named 'importlib_metadata-0.6.dist-info/METADATA' in the archive"
The command "python setup.py test" exited with 1.

Unable to load non-ASCII metadata on Python 2

In GitLab by @jaraco on Dec 5, 2018, 18:32

Attempting to load non-ascii metadata on Python 2.7 fails with a UnicodeEncodeError and this traceback:

    desc = self._find_local_distribution().metadata['Description']
.tox/python/lib/python2.7/site-packages/importlib_metadata/api.py:71: in metadata
    self.read_text('METADATA') or self.read_text('PKG-INFO')
/opt/python/2.7.15/lib/python2.7/email/__init__.py:57: in message_from_string
    return Parser(*args, **kws).parsestr(s)
/opt/python/2.7.15/lib/python2.7/email/parser.py:82: in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
E   UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 905: ordinal not in range(128)

Discovered this on the portend project in this job, which is using pytest-checkdocs to ensure the long-description is valid ReStructuredText.

Drop dependency on pip and importlib_metadata in tests

In GitLab by @jaraco on Feb 23, 2019, 23:59

Don't use glob.iglob()

In GitLab by @warsaw on May 24, 2018, 22:21

I don't have the full details in a publicly disclosable format, but we've discovered a really bad interaction between glob.iglob() and threads in entrypoints. It really comes down to regexps not releasing the GIL (which of course makes sense), and needing to glob for the metadata directory.

We should rewrite the directory discovery code to not rely on glob. It's perhaps a bit too bad that pip doesn't keep a registry of installed packages and their metadata directories.

Hooks for other loaders

In GitLab by @jaraco on May 18, 2018, 20:06

With pkg_resources, it was possible for a package provider (database, zip, executablother) to register a Provider for metadata, keyed on the loader which loads the packages, to provide metadata. This hook would allow any arbitrary third-party provider such as pyinstaller to register their hook and thus supply metadata or entry points or similar.

Does this project wish to provide that sort of extensibility? If so, early or deferred?

make the Distribtion._path attribute public?

In GitLab by @anthrotype on Nov 12, 2018, 11:39

It would be nice to be able to get the dist-info or egg-info metadata directory path for a given python package name.

For example, py2app would need to know where this metadata is on the filesystem in order to copy it inside the generated .app bundle

https://bitbucket.org/ronaldoussoren/py2app/issues/256/fs-module-not-fully-working-from-app

Currently one could do that with importlib_metadata via the Distribution._path attribute, however the leading underscore suggests that that is not part of the api.

Could/should it be made public (read-only perhaps)?

Add Python 3.7 to CI/CD and get the GitLab pipeline succeeding

In GitLab by @jaraco on May 23, 2018, 14:10

ReadTheDocs out of date

In GitLab by @jaraco on Feb 20, 2019, 15:42

The current "latest" docs published on RTD are not up-to-date with the latest release of importlib_metadata.

Complete port to CPython

In GitLab by @jaraco on Feb 20, 2019, 15:00

bpo-34632 describes the addition of the importlib_metadata functionality to CPython.

In python 9327, an early port was attempted, but the underlying implementation has since changed dramatically, so another port is probably the best approach.

Here's what I suggest for a port: Either CPython or this package should be the canonical, authoritative implementation. Currently, this implementation is that one, but at some point in the future, perhaps I suspect the CPython one may be. Regardless, one repo or another should contain the changes the comprise the port, the commits that transform the code from the canonical code base to the other (such that one could merge a contribution made in the canonical codebase into the other).

Test and support Python 3.4 and 3.5

In GitLab by @warsaw on Sep 17, 2018, 22:17

Can setuptools rely on importlib_metadata instead of pkg_resources?

In GitLab by @jaraco on Dec 24, 2018, 16:54

As an academic exercise, I'd like to survey the setuptools source code and see what functions within pkg_resources that setuptools (the Python package) relies on that could be supported by importlib_resources. I think it would be useful to know what functionality might be used there that's not implemented here.

Merge with importlib_resources

In GitLab by @jaraco on Sep 7, 2018, 13:18

In this comment, @warsaw suggests the project might merge with importlib_resources.

I do have a goal to have this functionality released a couple of weeks before the PyPA summit in late October, so ideally before Oct 1.

So this raises some concerns - can this project release the planned 0.1 and 0.2 releases and consider the merge with importlib_resources later, or do we wish to avoid any releases of this functionality prior to the merge with importlib_resources?

Also, what are the advantages of merging with importlib_resources? My instinct would be not to merge the two projects unless they have significant overlap in implementation.

Fix README so it renders on PyPI

In GitLab by @brettcannon on Nov 24, 2018, 01:57

Currently https://pypi.org/project/importlib-metadata/ is unrendered reStructuredText. Either something isn't specified in setup.py or there's a syntax error in the README.

Distribution lookup fails with dashes in the name

In GitLab by @jaraco on Nov 4, 2018, 09:36

Similar to #20, I've found that if a package has dashes in the name, the distribution lookup will fail with a PackageNotFoundError.

>>> import importlib_metadata
>>> importlib_metadata.version('elastic2-doc-manager')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jaraco/code/yg/public/elastic2-doc-manager/.tox/python/lib/python3.7/site-packages/importlib_metadata/api.py", line 105, in version
    return distribution(package).version
  File "/Users/jaraco/code/yg/public/elastic2-doc-manager/.tox/python/lib/python3.7/site-packages/importlib_metadata/api.py", line 86, in distribution
    return Distribution.from_name(package)
  File "/Users/jaraco/code/yg/public/elastic2-doc-manager/.tox/python/lib/python3.7/site-packages/importlib_metadata/api.py", line 52, in from_name
    raise PackageNotFoundError(name)
importlib_metadata.api.PackageNotFoundError: elastic2-doc-manager
>>> ^D
elastic2-doc-manager feature/auto-version $ ls -d .tox/python/lib/python3.7/site-packages/elastic*.dist-info
.tox/python/lib/python3.7/site-packages/elastic2_doc_manager-0.4.1.dev4+g5558283.dist-info

I suspect that to support resolving version numbers from filenames and disambiguating a dash in the name from the dash that separates the name from the version, some tool is "normalizing" dashes to underscores... so to look up the metadata for a distribution, the name needs to be normalized this way.

I might have thought it would go the other way, because pypi normalizes the other way.

Discover distribution for local package under development

In GitLab by @jaraco on Nov 27, 2018, 16:56

Consider the case where a library wishes to test the Description (or Name or other metadata) of the local project under test. importlib_metadata can return the Description, but only if it knows the Distribution name. One wishes to solicit the metadata for "this" project.

Currently, one can get the name of the current project by way of setuptools by invoking python setup.py --name, but as PEP 517/518 become more popular, that convention becomes less reliable. Moreover, ideally it should be possible to retrieve the information via an API without invoking a subprocess.

I'm thinking something like:

>>> importlib_metadata.metadata(importlib_metadata.local_name())['Description']
'...'

If the Distribution class were more public, I'd suggest:

>>> importlib_metadata.Distribution.local().metadata()['Description']

`importlib_metadata.entry_points` api is surprising

In GitLab by @asottile on Feb 13, 2019, 03:40

what I expected:

entry_points('pytest') => return me a mapping of pytest's entry points

reality:

entry_points('pytest') => ignore the passed in name argument (never referenced in function) and return all entrypoints for all distributions

Add type annotations

In GitLab by @warsaw on Sep 11, 2018, 24:35

For Python 3 only.

Wrong file location when loading metadata from egg-info

In GitLab by @felixonmars on Jan 1, 2019, 11:08

When loading metadata from an egg-info dir, it seems the top level dir was misplaced in the locate() output.

For example for the file
/usr/lib/python3.7/site-packages/pip/_internal/vcs/bazaar.py

The metadata in pip.egginfo points to src/pip/_internal/vcs/bazaar.py, so importlib_metadata.files("pip")[...].locate() returns a non-exist file:
/usr/lib/python3.7/site-packages/src/pip/_internal/vcs/bazaar.py

I wonder what should be done to fix it? Does setuptools need to generate a better (but incompatible) egg-info or maybe importlib_metadata should not trust the location of SOURCES from egg-info?