rsokl / custom_inherit

A Python package that provides tools for inheriting docstrings in customizable ways.

Hello! I do work in the areas of machine learning, physics, and software development. I also work on improving methods for testing scientific/research software, and am a maintainer for the Hypothesis testing library. I am passionate about education, and created the CogWorks course at the MIT Beaver Works Summer Institute as well as the website Python Like You Mean It.

Libraries for accelerating and improving ML research

• hydra-zen: Making Hydra more pythonic and easier to use at-scale for ML workflows and expriments
• responsible-ai-toolbox: PyTorch-centric library for evaluating and enhancing the robustness of AI technologies.

Other open source projects

• MyGrad: Drop-in automatic differentiation for NumPy
• noggin: A simple tool for logging and plotting metrics in real time
• custom_inherit: inheriting and merging docstrings in customizable ways (my first ever open source project!)

Tutorials

• Property-based testing tutorial (with co-author Zac Dodds)

custom_inherit's Issues

Problem when merging with flake8

I don't know if this is more of an issue to do to flake8, but I try.

Still with the .py from #46:

"""This is a test."""
import sklearn
import numpy
from typing import Optional
from custom_inherit import doc_inherit


class MyDecisionTreeClassifier(sklearn.tree.DecisionTreeClassifier):
    @doc_inherit(sklearn.tree.DecisionTreeClassifier.predict_proba, style="google_with_merge")
    def predict_proba(
        self,
        X: numpy.ndarray,
        check_input: Optional[bool] = True,
        my_option: Optional[bool] = False,
    ) -> numpy.ndarray:
        """Predict class probabilities of the input samples X.

        Args:
            my_option: If True, this is happening. If False, this other thing is happening and I
                need two lines to explain this option.

        Returns:
            numpy.ndarray: the result

        """
        print(my_option)
        super.predict_proba(X, check_input=check_input)

When you flake8 it with flake8 src/concrete/ml/sklearn/d.py --config flake8_src.cfg with

[flake8]
max-line-length = 100
per-file-ignores = __init__.py:F401
extend-ignore = E203

I get

src/concrete/ml/sklearn/d.py:18:1: DAR101 Missing parameter(s) in Docstring: - X
src/concrete/ml/sklearn/d.py:18:1: DAR101 Missing parameter(s) in Docstring: - check_input
src/concrete/ml/sklearn/d.py:22:1: DAR202 Excess "Returns" in Docstring: + return

Then, I have to add a # noqa: DAR101 in the docstring, to make flake8 happy. But it is a bit ugly in the html then:

Documentation

I was copy'n'pasting the examples and found some mistakes:

• the colons are sometimes missing:
  • class Parent(metaclass=DocInheritMeta(style="numpy"))should be class Parent(metaclass=DocInheritMeta(style="numpy")):
  • class Parent(object) as well

The indentation is not always correct:

• the line with raise NotImplementedError to the left
• the indentations are inconsistent (but valid) in the section inheriting Docstrings using a decorator

Allow docstring of __init__ to also inherit from the parent class

Hi all,

I noticed that, all methods starting and ending by "__" was kept off the docstring inheritance when the metaclass DocInheritMeta is used.

What is the reason for that? I would like the __init__ docstring also inherit from the parent class, could we imagine to have that as an option? is there any reason we should not always include it in the inheritance process?

Google-style functions using "Args:" are inherited with "Parameters:"

This is a minor quibble, but I thought it would be good to note.

Multiple inheritance

When having multiple inheritance (without properly using ABC or mixins #20 I found a strange behavior :

from custom_inherit import DocInheritMeta


class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
    """Parent.

    Notes
    -----
    Blah
    """

    def meth(self, x, y=None):
        """
        Raises
        ------
        NotImplementedError"""
        raise NotImplementedError


class Child1(Parent):
    """Child1. 

    Methods
    -------
    meth: does something
    """

    def meth(self, x, y=None):
        """Method description. """
        return 0


class Child2(Parent):
    """Child 2.

    Methods
    -------
    blah : does not much
    """
    def blah(self):
        """Method blah."""
        pass

class GrandChild(Child1, Child2):
    """Grand Child."""
    pass

>>> GrandChild.__doc__
'Grand Child.\n\nMethods\n-------\nblah : does not much\nmeth: does something\n\nNotes\n-----\nBlah\nBlah\nBlah\nBlah'

The doc of the grand parent class gets repeated. Is that the correct behavior ?

Show-stopping bug

Will fix tonight -___-

Fix _Store

_Store needs to implement a custom update

http://stackoverflow.com/questions/2060972/subclassing-python-dictionary-to-override-setitem

Conda-Forge Package

I'm curious if you would be interested in making this package available on Anaconda distributions via Conda-Forge. Seeing as this package has no dependencies, packaging it there would be a breeze, allowing for users to leverage it within Anaconda/Miniconda environments without relying on pip.

I can guide you through the process if this helps.

It does not work with multiple inheritance (mixins)

I have set of mixins to be used together with a base class:

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy")):
    """
    Parent class.

    Attributes
    ------------
    foo : str
        Foo attribute.
    """

class Mixin:
    """
    This is mixin which does something.
    """

class Child(Mixin, Parent):
    """
    Child description.
    """

>>> Child.__doc__
'Child description.'

Instead of expected:

>>> Child.__doc__
'Child description.\n\nAttributes\n----------\nfoo : str\n    Foo attribute.'

Now, the workaround is to put Mixin after Parent in the inheritance, but this goes against the logic I need (mixin has to override some things from parent).

Transferring ownership

Hi all - I am going to be transferring ownership of this project to my active GitHub account: @rsokl . I simply don't use this GitHub account anymore.

Inherit docstrings from Zope Interfaces

Feature request: inherit docstrings from zope.interface. For example:

from custom_inherit import DocInheritMeta
from zope.interface import implementer, Interface

class IClass(Interface):
    x = Attribute("Variable x.  Insert this description to class documentation.")
    def __init__(x):
        """Constructor.  Sets `x`"""
    def power(p):
        """Return `x**p`"""

@implementer(IClass)
class Class(metadata=DocInheritMeta(style="numpy")):
    def __init__(self, x):
        self.x = x
    def power(self, p):
        return self.x**p

c = Class(x=5)

Not sure if anything else is needed from an interface perspective: I think DocInheritMeta can probably just check to see if Class has the appropriate attributes (set by the implementer decorator) and then extract the documentation using duck typing.

If you think this fits with the design of custom_inherit, I can look at submitting a PR.

Make readthedocs page

Whitespace stripped with numpy style causes trouble

Part of the inheritance merging (for numpy style) involves stripping as much whitespace as possible. This is expected behaviour from the docstrings

custom_inherit/src/custom_inherit/_doc_parse_tools/numpy_parse_tools.py

Lines 142 to 143 in 7e08197

	Any whitespace that can be uniformly removed from a docstring's second line and onwards is
	removed. Sections will be separated by a single blank line.

and the tests

custom_inherit/tests/inheritance_test.py

Lines 79 to 80 in 7e08197

	GrandChild.__doc__
	== "Grand Child.\n\nMethods\n-------\nblah : does not much\nmeth: does something\n\nNotes\n-----\nBlah"

However, from the example README.md it's not expected (I've taken the liberty to shorten the example):

  """Method description

      Parameters
      ----------
      x: int
         blah-x
       ..."""

As this will be changed to

"Method description.\n\nParameters\n----------\nx: int\n blahx\n..."

However, this causes some trouble with other tools which use the whitespace differentiate between the parameter name and description to parse the docstrings. For instance, the parser in mkdocstrings and pytkdocs can no longer function correctly with the whitespace stripped.

Would the behaviour be able to be changed to keep the indentation? To a user this isn't expected from reading the package documentation or by assuming how it will behave.

I believe it might not be too hard a change to make but don't know this package well enough to be certain. I'm happy to make a PR if this is agreeable and you point me in the direction of where would be a good place to start

Cannot remove parameters

In my case I remove all arguments accepted in parent class in child's class method. It seems there is no way to also remove them from a docstring and it is automatically copied over? So I would like to remove the whole section. It seems I have to define an otherwise empty "Parameters" section.

(Working in numpy style.)

Merging a google docstring with a numpy docstring

Hey. Great tool, thanks a lot!

I was wondering if we can merge docstring of different kinds.

"""This is a test."""
import sklearn
import numpy
from typing import Optional
from custom_inherit import doc_inherit


class MyDecisionTreeClassifier(sklearn.tree.DecisionTreeClassifier):
    @doc_inherit(sklearn.tree.DecisionTreeClassifier.predict_proba, style="google_with_merge")
    def predict_proba(
        self,
        X: numpy.ndarray,
        check_input: Optional[bool] = True,
        my_option: Optional[bool] = False,
    ) -> numpy.ndarray:
        """Predict class probabilities of the input samples X.

        Args:
            my_option: If True, this is happening. If False, this other thing is happening and I
                need two lines to explain this option.

        Returns:
            numpy.ndarray: the result

        """
        print(my_option)
        super.predict_proba(X, check_input=check_input)

Here, I'd like to merge scikit docstring (which is numpy) with mine (which is google). It more or less works already:

but you can see that there is a bad thing happening because my_option explanation is too long. More precisely, it cuts the line in two, and writes "option. (need two lines to explain this) –" as a fake new option.

I wonder if it can be fixed easily. If not, it is already a great tool thanks

Within-Field Docstring Merging

I'm wondering if there is any way to handle a case where parameters or comments are composed from multiple sections.

It's always easier to understand with an example:

def some_operation(func):
    @doc_inherit(parent=func, style="numpy_napoleon")
    def wrapper( ... ):
        """
        Parameters
        ---------------
        x : str
          some string
        y : str
          some other string
        """
        < stuff happens >
        return
    return wrapper

@some_operation
def foo( ... ):
    """
    Parameters
    ---------------
    z : int
      some number
    """
    < stuff happens >

Then on inspection we get something resembling:

    Parameters
    ---------------
    x : str
      some string
    y : str
      some other string
    z : int
      some number

I can understand how technically challenging this could be but I'm wondering if this is a desired feature. Does this seem feasible?

class docstring with 'numpy_merge' section order

I found out that section order matters for 'numpy_merge' in class docstring :

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
   """Parent summary.

   Methods
   -------
   meth
   """
   pass

class Child(Parent):
    """Child summary.
    
    Attributes
    ----------
    None: so far

    """
    pass

c = Child()
print(c.__doc__)

will return

Child summary.

Attributes
----------
None: so far

while, inverting the two sections

from custom_inherit import DocInheritMeta

class Parent(metaclass=DocInheritMeta(style="numpy_with_merge")):
   """Parent summary.

   Attributes
   ----------
   None: so far
   """
   pass

class Child(Parent):
    """Child summary.
    
    Methods
    -------
    meth
    """
    pass

c = Child()
print(c.__doc__)

will return the expected output :

Child summary.

Methods
-------
meth

Attributes
----------
None: so far

Is that the correct behavior ?

custom_inherit 3.0 will drop support for Python < 3.5

This will lead to considerable cleanups in the code base. I also plan to create a readthedocs page and to improve the test suite for this package.