brettcannon / modutil

A library for working with Python modules

modutil

A library for working with Python modules. The highlights are:

• lazy_import() provides a way to do lazy import for large CLI apps.
• calc___all__() allows you to no longer have to manually maintain a module's :attr:__all__.
• filtered_dir() has dir() only show the relevant attributes of your module.

For these and other features of the library, please visit the documentation.

modutil's Issues

Add a smart, lazy __all__

The idea being that maintaining all is annoying, so why not automate it?

It would need to be a part of __getattr__. That probably means implementing a fall_through argument so that you can stack __getattr__ functions (e.g. this with lazy_import().

Provide a dynamic __dir__

Would want a couple of things:

• Leave out modules.
• Leave out common module attributes (e.g. __name__, __path__).
• Leave out anything starting with a _ but not ending in one.
• Provide a way to forcibly add something?
• Provide a way to forcibly mask something out?

Support for Python versions <3.7?

Not sure if this would violate your principles or not, but this hack would let you make this work on all versions of Python, I think :-)

Document how to set dir() to __all__

Since using __all__ for dir() is just __dir__ = lambda: __all__, there's no need for code to handle that case. But it should at least be documented so people know that if they come looking for it. If people want common module names as well then it simply becomes __dir__ = lambda: frozenset(__all__) + modutil.COMMON_MODULE_ATTRS (once again, not complicated enough to need code, but perhaps docs to cover the idea).

Inspired by @glyph and his 'publication' project.

Provide an `everything()` function

If either #1 or #2 happen, then it might be worth providing an everything() function which simply provides everything turned on. E.g.:

mod, __dir__, __getattr__ = modutil.everything(__name__, {'A'})

There wouldn't be any option to override default argument values as everything else will be constructed to allow for appropriate composition so that any special needs can be done by hand.

Provide a static __all__ generator

It really doesn't change after the module is finished being populated with attributes. The real trick is that it would have to be defined at the end of the module rather than at the top as a near-post import step.

Unlike __dir__, probably all attributes that start with a _ should be left out.

Provide a __dir__ that pulls from __all__

This would only be useful in the case where someone statically defined __all__ and wanted a simple __dir__ that returned it. Although since that's:

__all__ = [...]
__dir__ = lambda: __all__

Maybe it isn't worth it? Otherwise that would just become dir_from___all__(__name__) and that doesn't seem that much more helpful.

Feature request: lazy loading classes within module

Hi @brettcannon - I was wondering if it would be possible to implement some sort of lazy loading of the symbols from within a module/package. Specifically I'm thinking of loading a bunch of classes from different modules into the package __init__.py for ease of use for users in a Jupyter notebook, however to do so is currently very expensive for certain classes as they have many dependencies and so it would be great if the user could do something like the following:

# internally in mypkg, MyClass is imported from mypkg.sub
from mypkg import MyClass  # ideally lazy load so symbol is available but module not actually loaded

# load class and module on first use
MyClass()

Proxy module attributes through to __spec__

I.e. remove __file__, __cached__, __package__, and __loader__ from a module and proxy their accesses through to the underlying __spec__ object. Provides another way to minimize the attributes exposed on a module.

Deleting __name__ and __path__ is probably a bad idea since too much code out there assuming those attributes are set at the global level of a module.