Coder Social home page Coder Social logo

lazy-imports's Introduction

Lazy-Imports

Apache-2.0 License Contributor Covenant Python Version pypi
pytest Static Code Checks GitHub issues

This is a Python tool to support lazy imports. Likewise, the actual initialization of the module does not occur until usage time to postpone ModuleNotFoundErrors to the time of the actual usage of the module. This is useful when using various optional dependencies which might not all be installed or which have high load times and/or ressource consumption.

Table of Contents

Maintainers

One Conversation
This project is maintained by the One Conversation team of Deutsche Telekom AG. It is based on _LazyModule from HuggingFace and try_import() from the Optuna framework. Many thanks to HuggingFace for your consent and to Optuna for your consent to publish it as a standalone package 🤗 ♥.

Installation

Lazy-Imports is available at the Python Package Index (PyPI). It can be installed with pip:

$ pip install lazy-imports

Usage & Example for LazyImporter

A good and easy to understand example of how to use Lazy-Imports can be found in the __init__.py file of the HPOflow package. It is printed here:

import sys
from typing import TYPE_CHECKING

from lazy_imports import LazyImporter

from hpoflow.version import __version__


_import_structure = {
    "mlflow": [
        "normalize_mlflow_entry_name",
        "normalize_mlflow_entry_names_in_dict",
        "check_repo_is_dirty",
    ],
    "optuna": ["SignificanceRepeatedTrainingPruner"],
    "optuna_mlflow": ["OptunaMLflow"],
    "optuna_transformers": ["OptunaMLflowCallback"],
    "utils": ["func_no_exception_caller"],
}

# Direct imports for type-checking
if TYPE_CHECKING:
    from hpoflow.mlflow import (  # noqa: F401
        check_repo_is_dirty,
        normalize_mlflow_entry_name,
        normalize_mlflow_entry_names_in_dict,
    )
    from hpoflow.optuna import SignificanceRepeatedTrainingPruner  # noqa: F401
    from hpoflow.optuna_mlflow import OptunaMLflow  # noqa: F401
    from hpoflow.optuna_transformers import OptunaMLflowCallback  # noqa: F401
    from hpoflow.utils import func_no_exception_caller  # noqa: F401
else:
    sys.modules[__name__] = LazyImporter(
        __name__,
        globals()["__file__"],
        _import_structure,
        extra_objects={"__version__": __version__},
    )

Usage & Example for try_import

try_import is a context manager that can wrap imports of optional packages to defer exceptions. This way you don't have to import the packages every time you call a function, but you can still import the package at the top of your module. The context manager defers the exceptions until you actually need to use the package. You can see an example below:

from lazy_imports import try_import

with try_import() as optional_package_import:  # use try_import as a context manager
    import optional_package  # optional package that might not be installed

# other non optional functions here

def optional_function():  # optional function that uses the optional package
    optional_package_import.check()  # check if the import was ok or raise a meaningful exception

    optional_package.some_external_function()  # use the optional package here

Support and Feedback

The following channels are available for discussions, feedback, and support requests:

Reporting Security Vulnerabilities

This project is built with security and data privacy in mind to ensure your data is safe. We are grateful for security researchers and users reporting a vulnerability to us, first. To ensure that your request is handled in a timely manner and non-disclosure of vulnerabilities can be assured, please follow the below guideline.

Please do not report security vulnerabilities directly on GitHub. GitHub Issues can be publicly seen and therefore would result in a direct disclosure.

Please address questions about data privacy, security concepts, and other media requests to the [email protected] mailbox.

Contribution

Our commitment to open source means that we are enabling - in fact encouraging - all interested parties to contribute and become part of our developer community.

Contribution and feedback is encouraged and always welcome. For more information about how to contribute, as well as additional contribution information, see our Contribution Guidelines.

Code of Conduct

This project has adopted the Contributor Covenant as our code of conduct. Please see the details in our Contributor Covenant Code of Conduct. All contributors must abide by the code of conduct.

Licensing

Copyright (c) 2021 Philip May, Deutsche Telekom AG
Copyright (c) 2020, 2021 The HuggingFace Team
Copyright (c) 2018 Preferred Networks, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

lazy-imports's People

Contributors

philipmay avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

lazy-imports's Issues

Issue with type checkers

Hello!

When I use your package, apparently mypy and PyLance are not happy with it. They raises warning saying:

Stub file not found for "lazy_imports"

Indeed some of your datatypes cannot be properly understood by the type checkers because of this missing file (see https://mypy.readthedocs.io/en/stable/stubs.html). Any chance you can fix this at some point ?

Thanks :)

Attribute import support

Does the package support, or plan to support attribute-from-module imports?

From peps docs:

There are also third-party packages such as demandimport. These provide a “lazy module object” which delays its own import until first attribute access. This is not sufficient to make all imports lazy: imports such as from foo import a, b will still eagerly import the module foo since they immediately access an attribute from it. It also imposes noticeable runtime overhead on every module attribute access, since it requires a Python-level getattr or getattribute implementation.

Question about which packages this can be used with

I'm a little confused about where I can use this. Does the package author have to incorporate lazy-imports into their code in order to lazily load it? Or, could I import pandas, for example, lazily without having to modify the pandas source code?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.