douglasdavis / numpydoc.el

Insert NumPy style docstrings in Python functions.

numpydoc.el

An Emacs Lisp package to automatically insert NumPy style docstrings in Python function definitions.

Calling numpydoc-generate parses the function at point (the cursor can be anywhere in the function body). The parsing detects argument names, type hints, exceptions, and the return type hint. This information is used to generate a docstring.

The default behavior is to prompt the user (in the minibuffer) for a short & long description of the function, a description for each function argument, a description for each possible exception, and a description for the return. It's also possible to either disable the minibuffer prompt or use yasnippet insertion. See customization for more information. You'll also find a few examples below. See the NEWS file to see recent changes.

Setup

Pick your favorite method of Emacs Lisp package setup:

;; use-package with :ensure t to intall from MELPA.
(use-package numpydoc
  :ensure t)

;; use the straight.el package manager.
(straight-use-package 'numpydoc)

;; clone the git respository and require
(add-to-list 'load-path "/path/to/numpydoc.el")
(require 'numpydoc)

;; or point use-package to the local clone
(use-package numpydoc
  :load-path "/path/to/numpydoc.el")

The C-c C-n binding is vacant (not used in python.el, as of writing this), so you may want to give yourself a convenient shortcut:

;; with use-package
(use-package numpydoc
  :ensure t
  :bind (:map python-mode-map
              ("C-c C-n" . numpydoc-generate)))

;; without
(add-to-list 'load-path "/path/to/numpydoc.el")
(require 'numpydoc)
(define-key python-mode-map (kbd "C-c C-n") #'numpydoc-generate)

Customization

View customizations without leaving Emacs via M-x customize-group RET numpydoc

numpydoc-insertion-style

The method used to insert components of the docstring (default is 'prompt).

• 'prompt will trigger a request for each description in the minibuffer.
• 'yas (requires yasnippet to be installed) will generate a template and call yas-expand-snippet, providing an insertion method familiar to yasnippet users.
• nil will disable any interactive insertion (template text will be inserted).

numpydoc-quote-char

Quote character to use (the default is a double quote, ?\", used throughout the numpydoc docstring guide and the black formatting tool).

numpydoc-insert-examples-block

If t (the default) an Examples block will be added to the docstring.

numpydoc-insert-parameter-types

If t (the default) type hints from the function signature will be used to add a type to each argument in the Parameters block of the docstring.

numpydoc-insert-raises-block

If t (the default) a Raises bock will be added to the docstring if exceptions are detected in the function body.

numpydoc-insert-return-without-typehint

If t a Returns block will be inserted in the absence of a return type hint.

numpydoc-template-short

Template text that will be used as the short description if numpydoc-insertion-style is nil.

numpydoc-template-long

Template text that will be used as the long description if numpydoc-insertion-style is nil.

numpydoc-template-arg-desc

Template text that will be used for each function argument description if numpydoc-insertion-style is nil.

numpydoc-template-type-desc

Template text that will be used for each function argument type description if numpydoc-insertion-style is nil.

numpydoc-ignored-params

All function parameters with names listed here will be ignored when generating a docstring.

numpydoc-auto-fill-paragraphs

If t text that is inserted in a prompt will be automatically paragraph-filled.

Examples

M-x numpydoc-generate with the default configuration, numpydoc-insertion-style set to 'prompt (notice how long text is automatically paragraph-filled):

Using yasnippet (numpydoc-insertion-style set to 'yas):

With numpydoc-insertion-style set to nil; before:

def plot_histogram(
    x: np.ndarray,
    bins: int = 10,
    range: tuple[float, float] | None = None,
    weights: np.ndarray | None = None,
    flow: bool = False,
    ax: plt.Axes | None = None,
) -> tuple[plt.Figure, plt.Axes]:
    if weights is not None:
        if weights.shape != np.shape:
            raise ValueError("x and weights must have same shape.")
    pass

After M-x numpydoc-generate:

def plot_histogram(
    x: np.ndarray,
    bins: int = 10,
    range: tuple[float, float] | None = None,
    weights: np.ndarray | None = None,
    flow: bool = False,
    ax: plt.Axes | None = None,
) -> tuple[plt.Figure, plt.Axes]:
    """FIXME: Short description.

    FIXME: Long description.

    Parameters
    ----------
    x : np.ndarray
        FIXME: Add docs.
    bins : int
        FIXME: Add docs.
    range : tuple[float, float], optional
        FIXME: Add docs.
    weights : np.ndarray, optional
        FIXME: Add docs.
    flow : bool
        FIXME: Add docs.
    ax : plt.Axes | None
        FIXME: Add docs.

    Returns
    -------
    tuple[plt.Figure, plt.Axes]
        FIXME: Add docs.

    Raises
    ------
    ValueError
        FIXME: Add docs.

    Examples
    --------
    FIXME: Add docs.

    """
    if weights is not None:
        if weights.shape != np.shape:
            raise ValueError("x and weights must have same shape.")
    pass

Similar packages

• sphinx-doc.el: Inserts sphinx-compatible docstrings (does not offer customizations or automatically formatted insertions from the minibuffer or yasnippet).
• docstr: Docstring insertion support for any programming language, including NumPy style Python (it has a programmable interface but requires a bit more setup to get the utility provided numpydoc.el).

numpydoc.el's Issues

GL01 Compliance

Thanks for writing and maintaining this package, I find it super useful when writing code.

One thing I've noticed is that the following was automatically generated...

def summarise_shape(shape: npt.NDArray) -> list:
    """Summarise the region properties of a 2D numpy array using Scikit-Image.

    Parameters
    ----------
    shape : npt.NDArray
        2D binary array of a shape.

    Returns
    -------
    list
        List of Region Properties each item describing one labelled region.
    """
    return measure.regionprops(shape)

I use the numpydoc pre-commit hook to make sure I get everything correct and on parsing the above it raised GL01.

Putting the first line on its own new line assuaged the linter.

def summarise_shape(shape: npt.NDArray) -> list:
    """
    Summarise the region properties of a 2D numpy array using Scikit-Image.

    Parameters
    ----------
    shape : npt.NDArray
        2D binary array of a shape.

    Returns
    -------
    list
        List of Region Properties each item describing one labelled region.
    """
    return measure.regionprops(shape)

I'll see if I can work out how to add this and make a Pull Request over the coming weeks but am writing this up now for reference.

Option for not-adding the types of the parameters in the "Parameters" section

Thx a lot for this nice package that has proper support for type annotations and numpydoc-style docstrings!

Since the sphinx html output contains the type annotations in the signature of the functions/methods (can be controlled by setting the autodoc_typehints sphinx-config var), it is not needed that that the types are displayed again in the "Parameters" section. Therefore, I think it makes sense to make to make it configurable, whether the types are shown or not.

Conflict with tree-sitter package

Hey, there!

I've been using this package for a while now and it is very handy and useful for me.
Recently I installed tree-sitter in Emacs and began seeing a strange behavior with my docstring generation.

When tree-sitter is enabled and I run numpydoc-generate, it creates the docs but yasnippet completion is disabled. At the same time, my syntax highlighting goes crazy and I have to kill the buffer and open it again to get back to normal. If I undo the doc generation and try again, I get yasnippet completion but the syntax highlighting still goes crazy.

Running numpydoc-generate once:

Running numpydoc-generate twice:

Do you have any idea if this is a problem with numpydoc package? I don't write docstring very often so it doesn't get in the way that much. But I would really like to be able to use both packages without conflict.

Two minimal configurations are attached. One without tree-sitter (numpydoc works fine) and one with tree-sitter (numpydoc showing weird behavior).

Minimal configuration that works fine:

Minimal configuration with described behavior:

Below are the minimal configurations depicted.

Minimal configuration that works fine:

(require 'package)
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("org" . "https://orgmode.org/elpa/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))
(package-initialize)
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))
(require 'use-package)
(setq use-package-always-ensure t)

(use-package python
  :init
  (setq python-indent-guess-indent-offset nil)
  :config
  (setq python-shell-interpreter "python"))

(use-package numpydoc
  :after yasnippet
  :init
  (setq numpydoc-insertion-style nil)
  (setq numpydoc-insertion-style 'yas)
  (setq numpydoc-insert-examples-block nil)
  :bind (:map python-mode-map
	      ("C-c C-n" . numpydoc-generate))
  :after python)

(use-package yasnippet
  :config (yas-global-mode 1))
(use-package yasnippet-snippets)
(setq yas-snippet-dirs '("~/.emacs.d/snippets"))

Minimal configuration with described behavior:

(require 'package)
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("org" . "https://orgmode.org/elpa/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))
(package-initialize)
(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))
(require 'use-package)
(setq use-package-always-ensure t)

(use-package python
  :init
  (setq python-indent-guess-indent-offset nil)
  :config
  (setq python-shell-interpreter "python"))

(use-package numpydoc
  :after yasnippet
  :init
  (setq numpydoc-insertion-style nil)
  (setq numpydoc-insertion-style 'yas)
  (setq numpydoc-insert-examples-block nil)
  :bind (:map python-mode-map
	      ("C-c C-n" . numpydoc-generate))
  :after python)

(use-package tree-sitter)
(use-package tree-sitter-langs)
(add-hook 'python-mode-hook 'tree-sitter-mode)
(add-hook 'python-mode-hook #'(lambda () (tree-sitter-hl-mode)))

(use-package yasnippet
  :config (yas-global-mode 1))
(use-package yasnippet-snippets)
(setq yas-snippet-dirs '("~/.emacs.d/snippets"))

I really like this package and can't thank you enough for taking the time to write it!

bug dealing with incorrect def

if i have this in my python buffer,

def incorrect:
    pass

position the cursor after the colon, and type C-c C-n, the error message

(wrong-type-argument number-or-marker-p nil)

appears.

turns out this is because of the lack of the parens before the colon. obviously, one shouldn't write syntactically invalid code. but, a nicer error message might be nice.

here's the stack trace:

  1+(nil)
  (substring rawsig (1+ (string-match-p (regexp-quote "(") rawsig)))
  (numpydoc--split-args (substring rawsig (1+ (string-match-p (regexp-quote "(") rawsig))))
  (-map #'s-trim (numpydoc--split-args (substring rawsig (1+ (string-match-p (regexp-quote "(") rawsig)))))
  (let* ((fnsig buffer-substr) (trimmed (s-collapse-whitespace fnsig)) (parts (s-split "->" trimmed)) (rawret (if (nth 1 parts) (s-trim (nth 1 parts)) nil)) (rtype (if rawret (progn (substring rawret 0 (1- (length rawret)))))) (rawsig (cond (rtype (substring (s-trim (car parts)) 0 -1)) (t (substring (s-trim (car parts)) 0 -2)))) (rawargs (-map #'s-trim (numpydoc--split-args (substring rawsig (1+ (string-match-p ... rawsig)))))) (args (-remove #'(lambda (x) (-contains-p numpydoc-ignored-params (let ... ...))) (-map #'numpydoc--arg-str-to-struct rawargs))) (exceptions (numpydoc--find-exceptions))) (let ((args args) (rtype rtype) (raises exceptions)) (progn (record 'numpydoc--def args rtype raises))))
  (save-excursion (let* ((fnsig buffer-substr) (trimmed (s-collapse-whitespace fnsig)) (parts (s-split "->" trimmed)) (rawret (if (nth 1 parts) (s-trim (nth 1 parts)) nil)) (rtype (if rawret (progn (substring rawret 0 (1- ...))))) (rawsig (cond (rtype (substring (s-trim ...) 0 -1)) (t (substring (s-trim ...) 0 -2)))) (rawargs (-map #'s-trim (numpydoc--split-args (substring rawsig (1+ ...))))) (args (-remove #'(lambda (x) (-contains-p numpydoc-ignored-params ...)) (-map #'numpydoc--arg-str-to-struct rawargs))) (exceptions (numpydoc--find-exceptions))) (let ((args args) (rtype rtype) (raises exceptions)) (progn (record 'numpydoc--def args rtype raises)))))
  numpydoc--parse-def("def incorrect:")
  numpydoc-generate()
  funcall-interactively(numpydoc-generate)

Font-lock support for keywords in numpydoc docstrings

What is your stance about supporting font-locking of numpydoc keywords docstrings (similar to the code in https://github.com/glyph/python-docstring-mode/blob/master/python-docstring.el)? Do you think that it makes sense to add it to your numpydoc.el pkg or should it be part of a separate pkg?

Already written docstring

Thanks for this fantastic package. I will definitely use this.

However some features are missing in my opinion.

Some times I have a "return" in my function but only write it as
def f(a, b, c) and not as def f(a, b, c) -> float so it would be nice to still be able to specify the return parameter in the doc even without specifying its type in the function definition.

Another usecase is that sometimes part of the docs are already written. It would be nice to have a default mode where the already written text is kept as a default so that I can use the package to modify only the doc of one argument.

Lastly, I notice the "prompt" option nicely format the doc. It would surely be possible to re-format badly written doc (such as when everything is written on a single line instead of nicely given as a paragraph.

Do you thing some of these things are worth to implement ?

Incorrect parsing of default values containing commas

Minimum working example:

def f(arg="str,str"):
    pass

Calling numpydoc-generate with point in the function results in the following output:

def f(arg="str,str"):
    """FIXME: Short description.

    FIXME: Long description.

    Parameters
    ----------
    arg : FIXME: Add type.
        FIXME: Add docs.
    str" : FIXME: Add type.
        FIXME: Add docs.

    Examples
    --------
    FIXME: Add docs.

    """
    pass

Notice the str" in the Parameters section. Similar output occurs with multi-item tuples and sets, but not lists. Dicts fail to generate any documentation with the message s-trim-right: Wrong type argument: stringp, nil.

newlines in templates elided

Hi, I'm trying your package on Emacs 28.2. It would be a huge help if I can make it work, and I'm willing to do a lot of configuration. But to start with, I'm hitting this problem that I don't know how to work around: when I execute numpydoc-generate, the template outline is inserted correctly, but then when the actual arguments are plugged in the preceding newlines seem to be deleted, or maybe converted to spaces. So it ends up looking like this:

"""FIXME: Short description.

FIXME: Long description.

Parameters ---------- uid FIXME: Add docs.

Returns ------- tuple[str, str | None] FIXME: Add docs.

"""

It happens regardless which numpydoc-insertion-style is in effect - if the prompt style is in effect, it looks fine until I answer the prompt, then it gets mangled. Clearly this negates much of the benefit of the package :(

Thanks for reading,

--
Ian

Support classes?

This looks very nice. I immediately ran into trouble documenting a class, which numpydoc style indicates should contain all normal sections, documenting __init__ parameters, attributes, and (possibly) methods. This would be a great addition. Possibly just __init__ parameters to start and placing a to-be-filled attributes section?