clayrisser / sphinx-markdown-builder

DISCONTINUED: sphinx builder that outputs markdown files. The active version of this project is now being maintained at liran-funaro/sphinx-markdown-builder.

sphinx-markdown-builder

---

DISCONTINUED: This project is no longer maintained. The active version of this project is now being maintained at liran-funaro/sphinx-markdown-builder.

---

sphinx builder that outputs markdown files

Please ★ this repo if you found it useful ★ ★ ★

If you want frontmatter support please use sphinx-jekyll-builder

Built by Silicon Hills LLC

Silicon Hills offers premium Node and React develpoment and support services. Get in touch at nuevesolutions.com.

Recommended Projects

• sphinx-markdown-parser - write markdown inside of docutils & sphinx projects

Features

• Generates markdown

Installation

pip3 install sphinx-markdown-builder

Dependencies

• Python 3

Usage

Build markdown files with Makefile

make markdown

Build markdown files with sphinx-build command

cd docs
sphinx-build -M markdown ./ build

Support

Submit an issue

Screenshots

Contribute a screenshot

Contributing

Review the guidelines for contributing

License

MIT License

Jam Risser © 2018

Changelog

Review the changelog

Credits

• Jam Risser - Author
• Matthew Brett - doctree2md

Support on Liberapay

A ridiculous amount of coffee ☕ ☕ ☕ was consumed in the process of building this project.

Add some fuel if you'd like to keep me going!

sphinx-markdown-builder's Issues

failed on sphinx 1.5.1

better to support some older versions, on 1.5.1 it fails with

sphinx-build -b markdown doc/ doc/_build/ -E
Running Sphinx v1.5.1
/xxx/.virtualenvs/log/lib/python3.6/site-packages/matplotlib/font_manager.py:232: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
  'Matplotlib is building the font cache using fc-list. '

Extension error:
Could not import extension sphinx_markdown_builder (exception: cannot import name '__')

Directive 'figure' is not supported

As a figure works as an image, could it be possible to support the .. figure:: directive?

pydash not installed with version 0.4.0

pydash package not installed when using pip for version sphinx-markdown-builder==0.4.0 from PyPI

Missing plenty of nodes (notes, admonitions, labels, code-block, hlist, centered, glossary...)

I have written a small rst snippet to see what sphinx-markdown-builder has in its guts. I am currently not quite impressed, but I feel this project deserves some interest and some contribution.

My intimate goal is to use this extension (or another) with Vuepress to generate a real SPA static site from a rST documentation. I am planning to write a custom builder/writer to support Markdown+HTML output. So in my case, I can use Vue components inside my markdown documents.

Here some observations on the time said:

• note is not correctly translated. If translated into Vuepress, admonitions could use Custom Containers.
• warning a two lines warning should be translated as a description list that doesn't exist in Markdown :(
• numref internal references don't seem to work here. I would be possible to have anchor HTML [warning](#warning) and place an anchor in the document.
• code-block language. The language used should be added after the opening block
• math, The math environment is not supported :(
• table They are not working :(
• Various tables are not properly translated. I guess the best option is to translate them in HTML and inject the HTML directly into the Markdown.
• glossary : not supported

.. All Directives documentation master file, created by
   sphinx-quickstart on Sun Aug 16 19:21:50 2020.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to All Directives's documentation!
==========================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

This is some **bold** text and some *italic* one. Here a `link <http://google.fr>`_ And an internal ref :numref:`warning`.
.. note::

    This function is not suitable for sending spam e-mails.

.. _warning:

.. warning::

    The *spam* parameter

.. centered:: Some centred text

.. hlist::
    :columns: 3

    * A list of
    * short items
    * that should be
    * displayed
    * horizontally

.. code-block:: c

    void main(void) {
        int i = 2;
    }

.. glossary::

   term 1 : A
   term 2 : B
      Definition of both terms.

.. math::

   (a + b)^2 = a^2 + 2ab + b^2
   (a - b)^2 = a^2 - 2ab + b^2

.. productionlist::
   try_stmt: try1_stmt | try2_stmt
   try1_stmt: "try" ":" `suite`
            : ("except" [`expression` ["," `target`]] ":" `suite`)+
            : ["else" ":" `suite`]
            : ["finally" ":" `suite`]
   try2_stmt: "try" ":" `suite`
            : "finally" ":" `suite`

+------------------------+------------+----------+----------+
| Header row, column 1   | Header 2   | Header 3 | Header 4 |
| (header rows optional) |            |          |          |
+========================+============+==========+==========+
| body row 1, column 1   | column 2   | column 3 | column 4 |
+------------------------+------------+----------+----------+
| body row 2             | Cells may span columns.          |
+------------------------+------------+---------------------+
| body row 3             | Cells may  | - Table cells       |
+------------------------+ span rows. | - contain           |
| body row 4             |            | - body elements.    |
+------------------------+------------+---------------------+

+--------------+----------+-----------+-----------+
| row 1, col 1 | column 2 | column 3  | column 4  |
+--------------+----------+-----------+-----------+
| row 2        | Use the command ``ls more``.     |
+--------------+----------+-----------+-----------+
| row 3        |          |           |           |
+--------------+----------+-----------+-----------+

.. table:: Some caption

    =====  =====  =======
    A      B      A and B
    =====  =====  =======
    False  False  False
    True   False  False
    False  True   False
    True   True   True
    =====  =====  =======

.. figure:: https://nice-assets.s3-accelerate.amazonaws.com/smart_templates/8709f94969aca8c1dee0eb33eda98ec3/assets/preview_afcafb91b9af067cb34ed46d7f074c1c.jpg

    Some caption

- List
- With

  - An indented number of items
  - Here

    .. code-block:: text

       With some code here

A description
    This is a description

Another description
    With some other things


Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Linked file extension

Hi currently in generated index.md file contents linked like this
[h2 Heading](dep/base.md#h2-heading)

How can I configure this to generating links without .md file extension like this
[h2 Heading](dep/base#h2-heading)

Use version ranges instead of pinning exact versions

I'm trying to use your library on Python 3 and with Sphinx 2.0.1, and I have noticed that your package pins every package it depends on, this makes it very difficult to use new Sphinx versions.

update requirements file

Is it possible to remove the strict version matching for the required packages in favor of the greater then equals sign (>=) ?

The package fails on newer versions of sphinx (i.e. 2.2.0) which also installs newer version of docutils (0.15.x).

master file ..\docs\contents.rst not found

I have sphinx-build 1.8.3.

From what I've been following, I only need an index.rst file initially. When I try to run make markdown, it's looking for a file .\docs\contents.rst , which by default I don't have.

I assume this might be configured to work with an older version of sphinx that included a contents.rst by default?

I created a contents.rst file and put some dummy content in, and then it ran just fine.

toctree is not preserved with links

toctree outputs a list, but to stay in the same style as html build it should really include links to the appropriate md file.

Add option to change markdown delimiters.

Whilst it's common for math to use the $ delimiter, on Code Review it is specified by \$.

Can the ability to change the delimiters be made available in conf.py?

This would require changing doctree2md.py#L484, but I don't know how to get data from conf.py.

Add table support

I have already implemented table support, but I'm am not sure how well it works across different setups. Feedback would be much appreciated.

I am keeping track of where I am in the table to hopefully make it easy to add support for nested tables in the future. Keeping track of where you are is a simple pop/push operation. Push (append) the current node to the node type array when the node is entered, and pop it off the node type array when the node is departed.

    def visit_table(self, node):
        self.tables.append(node)

    def depart_table(self, node):
        self.tables.pop()

Notice that I'm doing this with other nodes besides the table node. For example, I do this with thead, tbody and row.

    def visit_row(self, node):
        if not len(self.theads) and not len(self.tbodys):
            raise nodes.SkipNode
        self.rows.append(node)

    def depart_row(self, node):
        self.add('|\n')
        if not len(self.theads):
            self.row_entries = []
        self.rows.pop()

Keeping track of where you are in the table (basically reference counting) simplifies the code. Instead of using integer counters, I am using arrays of nodes. This allows referencing the nodes from other visit methods. This can come in handy quite often. For example, I use the "reference counting node arrays" to generate padding in the table entries making the tables look much prettier.

Basically, I am finding the largest string in a column and using it to calculate the padding. This would be very challenging if I did not have access to nodes outside of the current node.

    @property
    def rows(self):
        rows = []
        if not len(self.tables):
            return rows
        for node in self.tables[len(self.tables) - 1].children:
            if isinstance(node, nodes.row):
                rows.append(node)
            else:
                for node in node.children:
                    if isinstance(node, nodes.row):
                        rows.append(node)
        return rows

    def depart_entry(self, node):
        length = 0
        i = len(self.row_entries) - 1
        for row in self.rows:
            if len(row.children) > i:
                entry_length = len(row.children[i].astext())
                if entry_length > length:
                    length = entry_length
        padding = ''.join(_.map(range(length - len(node.astext())), lambda: ' '))
        self.add(padding + ' ')

https://github.com/codejamninja/sphinx-markdown-builder/blob/master/sphinx_markdown_builder/markdown_writer.py#L320-L329

I am thinking about possibly implementing this reference counting across all of the nodes. It could simplify the design of some of the more complex transpiling.

Please discuss the reference counting more at the issue below.
#8

Desc element not yet supported in Markdown.

I tried to generate docs using this package but it showed 3 warnings: 'The desc element not yet supported in Markdown'. Three files should present a lot of docs but only has titles.

Is there a way to support custom directives?

It'd be very nice to be able to translate custom directives, like a .. video:: one I've created for my doc.

Apply to be Co-Maintainer of sphinx-markdown-builder

I had sent mail to you but get no response, so I file issue here :)

---

Hello clayrisser,

I am a power Sphinx user[1] and I wrote many Sphinxextensions[2].

sphinx-markdown-builder is a awesome project, but it has some BUGs.
I filed a PR[3] to fix one of theme, I hope it can be merged.
But it seems that you are no longer active on this project. Contribute is
always better than fork, can I apply to be the co-maintainer of this project?

If you agree, please also add me[4] as Co-Maintainer on pypi.

[1] https://github.com/SilverRainZ/bullet/blob/master/conf.py
[2] https://github.com/sphinx-notes
[3] #57
[4] https://pypi.org/user/SilverRainZ/

--
Best regards,
Shengyu Zhang

https://silverrainz.me/

No output from automodule:: directive

• Module is loaded and parsed correctly (make html works)
• no markdown is rendered for the 'automodule' section.

Sphinx output:

Running Sphinx v1.8.2
making output directory...
building [mo]: targets for 0 po files that are out of date
building [markdown]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index                                                                                                                                                   
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index                                                                                                                                                    
build succeeded, 1 warning.

The markdown files are in _build/markdown.
/home/[...]/sphinx/index.rst:: WARNING: The desc element not yet supported in Markdown.

index.rst

.. Foo documentation master file, created by
   sphinx-quickstart on Tue Dec 11 15:25:35 2018.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to Foo's documentation!
==============================

.. automodule:: foo.api
   :members:

.. toctree::
   :maxdepth: 2
   :caption: Contents:

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

index.md

<!-- Foo documentation master file, created by
sphinx-quickstart on Tue Dec 11 15:25:35 2018.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. -->
# Welcome to Foo’s documentation!

# Indices and tables

* [Index](https://localhost/)

* [Module Index](https://localhost/)

* [Search Page](https://localhost/)

..let me know if you can use some more info.

installing auto update other libraries

when pip install this library, it auto update my other libraries, that may not be good

Preserve language domain when converting language blocks

Markdown conversion does not preserve restructuredtext code blocks such as:

.. code-block:: python
    import math

.. literalinclude:: test.py
    :language: python

.. highlight:: bash

::
    cat *.rst

And render it in markdown as (without the spaces I've used as escape characters for the backticks):

`` `python
import math
` ``

...

`` `bash
cat *.rst
` ``

Pop from empty list

Getting following exception whilst running make markdown on sphinx-markdown-builder==0.4.0:

~/.local/lib/python3.6/site-packages/sphinx_markdown_builder/markdown_writer.py", line 312, in depart_row
    self.rows.pop()
IndexError: pop from empty list

Currently my hack is to change line 312 to: None if not self.rows else self.rows.pop()

Add `singlemarkdown` mode to help with internal project development

Being able to produce a single README.md that has the entire output would be valuable for teams using products like GitHub that want to make sure there is "go to" documentation for every repository.

Tables are not supported

Tables (grid-like or list-table) in reST become mangled in Markdown.

Add reference counting to all nodes

During my implementation of tables, I found it quite handy to add reference counting support through node arrays.

Instead of reference counting with integers, I am using arrays of nodes.

This solved two major challenges.

1. Knowing where I am in nested nodes of the same type
2. Finding information from nodes outside of the current node

You can see a simple implementation of this with tables.

    tables = []

    def visit_table(self, node):
        self.tables.append(node)

    def depart_table(self, node):
        self.tables.pop()

To detect how deep you are in the tree (at least with nodes of the same type), you can simply grab the length.

    @property
    table_depth():
        return len(self.tables)

To grab the current node of a certain type, simply grab the last node from the array.

    @property
    current_table():
        return self.tables[self.table_depth - 1]

This has been so helpful with tables, I am thinking about adding this to all the nodes. Please give your feedback below.

Feature Request: Add support for graphviz directive

Another great feature would be inclusion of graphs given by the graphviz directive for example:

.. graphviz:: 
    digraph foo { "bar" -> "baz"; }

Feature Request: Add support for todo directive

Another great feature would be inclusion of graphs given by the todo directive for example:

.. todo:: 
    Add todo support ;-)

.. todolist::

Please note I've setup an example within https://github.com/Lightslayer/sphinx-automodule-example under docs/source/todo.rst

Misalignment of elements for nested lists

Misalignment of elements for nested lists are present in the Markdown output. What part of the code handles this?

Input:

1. parent list item

   text

   * child list item

     text

     .. code-block:: none

        command
        command

     text

Current output:

1. parent list item

text	   			<<==== misalignment (text after parent list item)

    * child list item

text				<<==== misalignment (text after child list item)

```none				<<==== misalignment (code block after child list item)
command				<<==== (ditto)
command				<<==== (ditto)
```				<<==== (ditto)

text				<<==== misalignment (text after child list item and code block)

Desired output:

1. parent list item

   text

    * child list item

      text

      ```none
      command
      command
      ```

      text

Deprecate doctree2md.py

I think I would like to eventually deprecate doctree2md.py. Below are a few reasons for this thought.

• doctree2md.py was never fully finished (there are major features missing)
• doctree2md.py was never built with sphinx in mind
• doctree2md.py is a great reference, but I think the developers of sphinx-markdown-builder are getting a good grasp on how sphinx builders work
• I used doctree2md.py to jumpstart the sphinx-markdown-builder, not necessarily to sustain it

Please give your feedback below

Missing Markdown Features

I'm using Matthew Brett's doctree2md to actually do the conversion.

His markdown writer is a great start, but certainly not complete.

In this issue, please list any markdown features that are broken, missing, or incomplete.

If the community would star the features they most want, that will help me prioritize which ones to tackle first.

README typo?

I'm a bit confused about the "Usage" section of the README:

from sphinx_markdown_parser.markdown_builder import MarkdownBuilder

source_suffix = {
    '.rst': 'restructuredtext',
    '.md': 'markdown'
}

def setup(app):
    app.add_source_suffix('.md', 'markdown')
    app.add_source_parser(MarkdownParser)

Should this be from sphinx_markdown_builder.markdown_build import MarkdownBuilder instead of from sphinx_markdown_parser.markdown_builder import MarkdownBuilder. Also I think app.add_source_parser(MarkdownParser) should be app.add_source_parser(MarkdownBuilder), otherwise, where is MarkdownParser imported from/defined at?

Lists have too much blank space

This .rst:

Simple Lists
------------

- Item 1
- Item 2
- Item 3

is output as:

# Simple Lists


* Item 1


* Item 2


* Item 3

When rendered, this puts too much space between the list items.

It renders on GitHub like this:

If the markdown instead were this:

# Simple Lists

* Item 1
* Item 2
* Item 3

then it would render as:

Incorrect output when image directive has :target: set

Images with targets (hyperlinks) are incorrectly converted as hyperlinks and loose the image, e.g.

An image should be included below (without hyperlink):

.. image:: Git-Icon-Black.png

The same image should be included below 
(this time with hyperlink):

.. image:: Git-Icon-Black.png
   :target: https://github.com/codejamninja/sphinx-markdown-builder

example ReST Table
----------------------
...

Becomes the following in the markdown output:

![image](Git-Icon-Black.png)

The same image should be included below
(this time with hyperlink):

[](https://github.com/codejamninja/sphinx-markdown-builder)## example ReST table

...

I've setup an example within https://github.com/Lightslayer/sphinx-automodule-example under the latest docs/source/other.rst.

custom node support

Literalinclude has too many newlines.

When using literalinclude to include a file, each newline is displayed twice. This is as the newlines lines are set to be \r\n. The file is actually using \n so somewhere in Sphinx/docutils it's adding these.

MVCE:

.. literalinclude:: path/to/file

---

The fix is to just change doctree2md.py#425 to:

text = node.astext().replace('\r\n', '\n')

Documetation outdated

Is it possible that the documentation needed to be updated? i.e:
from sphinx_markdown_builder.markdown_builder import MarkdownBuilder

def setup(app):
app.add_source_suffix('.md', 'markdown')
app.add_source_parser(MarkdownBuilder)

Also, the python support badge doesn't match the setup file classifiers.

AttributeError: type object 'MarkdownBuilder' has no attribute 'supported'

Hi, I'm running into a problem when I try to run this package.

I copied the conf.py example for sphinx 1.8 (I'm using 2.2) but needed to edit the example code as is seems a little out of date, it references a sphinx_markdown_parser module.

I'm running from tox with this command: -
tox sphinx-build -M markdown containers/source {env:DOC_OUTPUT_FOLDER:'build'}/containersmd

Error

Exception occurred:
  File "/home/matt.everett/code/mkc/microservices/service-api/src/.tox/doc/lib/python3.7/site-packages/sphinx/registry.py", line 302, in add_source_parser
    if len(parser.supported) == 0:
AttributeError: type object 'MarkdownBuilder' has no attribute 'supported'

Log

# Sphinx version: 2.2.0
# Python version: 3.7.3 (CPython)
# Docutils version: 0.13.1 release
# Jinja2 version: 2.10.1
# Last messages:

# Loaded extensions:
Traceback (most recent call last):
  File "/home/matt.everett/code/mkc/microservices/service-api/src/.tox/doc/lib/python3.7/site-packages/sphinx/cmd/build.py", line 275, in build_main
    args.tags, args.verbosity, args.jobs, args.keep_going)
  File "/home/matt.everett/code/mkc/microservices/service-api/src/.tox/doc/lib/python3.7/site-packages/sphinx/application.py", line 250, in __init__
    self.config.setup(self)
  File "/home/matt.everett/code/mkc/microservices/service-api/src/doc/containers/source/conf.py", line 63, in setup
    app.add_source_parser(MarkdownBuilder)
  File "/home/matt.everett/code/mkc/microservices/service-api/src/.tox/doc/lib/python3.7/site-packages/sphinx/application.py", line 1133, in add_source_parser
    self.registry.add_source_parser(*args, **kwargs)
  File "/home/matt.everett/code/mkc/microservices/service-api/src/.tox/doc/lib/python3.7/site-packages/sphinx/registry.py", line 302, in add_source_parser
    if len(parser.supported) == 0:
AttributeError: type object 'MarkdownBuilder' has no attribute 'supported'

Feature Request: Add support for plot directive

Another great feature would be to allow use of the matplotlib plot directive, for example:

.. plot::

    import matplotlib.plot as plt 
    plt.plot([0,1], [2, -2])
   

Literal node isn't kept in links.

If you setup sphinx.ext.intersphinx and add a link it results in [abc], instead of [`abc`].

Using the mapping:

intersphinx_mapping = {'python': ('https://docs.python.org/3', None)}

And the following reST:

:py:func:`operator.add`

Results in the following Markdown

[operator.add()](https://docs.python.org/3/library/operator.html#operator.add)

It should be:

[`operator.add()`](https://docs.python.org/3/library/operator.html#operator.add)

---

I followed the hello world extension tutorial, and changed the code to include a link to example.com with the literal text of example.com, and it also did the same:

from docutils import nodes
from docutils.parsers.rst import Directive


class HelloWorld(Directive):
    def run(self):
        hyperlink = nodes.reference('', '', internal=False, refuri='https://example.com', reftitle='example.com')
        hyperlink.append(nodes.literal('example.com', 'example.com'))
        paragraph = nodes.paragraph(text='Hello World! ')
        paragraph.append(hyperlink)
        return [paragraph]


def setup(app):
    app.add_directive("helloworld", HelloWorld)

    return {
        'version': '0.1',
        'parallel_read_safe': True,
        'parallel_write_safe': True,
    }

.. helloworld::

Results in:

Hello World! [example.com](https://example.com)

It should be:

Hello World! [`example.com`](https://example.com)

---

It can also be done in raw reST:

|operator.add|_

.. |operator.add| replace:: :code:`operator.add`
.. _operator.add: https://docs.python.org/3/library/operator.html#operator.add

---

Whilst these result in slightly different HTML, they all have the text formatted literally.

Feature Request: Add option to preserve TOC tree order

It would be great if there was an option to allow the generated markdown files to be prefixed with their TOC numbering (which would work on most sites)

Several errors building to markdown Godot-Engine manual

Hi! This project is amazing. 😃 I'm looking forward to build all manuals I need to markdown to read them and consult them within Obsidian, but I'm having some issues 😞

---

I want to build to markdown this project:

• source project files https://github.com/godotengine/godot-docs/tree/3.3

This is an example file from source:

• source rst class_animatedsprite.txt - (renamed to .txt to upload it, github doesn't let upload rst files )

This is how it looks in html:

• html reference https://docs.godotengine.org/en/stable/classes/class_animatedsprite.html

And as a result several errors are shown in the build making really difficult to read this file in markdown:

• markdown result class_animatedsprite.md

---

On the other hands the images were referenced in the text and they points to the right path but the folders weren't copied.

Is there any parameters to solve those issues or are they bugs?

ModuleNotFoundError: No module named 'sphinx_markdown_parser'

as per readme.md i installed sphinx_markdown_parser and sphinx-markdown-builder both, but still facing error

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx/config.py", line 361, in eval_config_file
    execfile_(filename, namespace)
  File "/usr/local/lib/python3.7/site-packages/sphinx/util/pycompat.py", line 86, in execfile_
    exec(code, _globals)
  File "/Users/roambee1/workspace/jsdoc-example/conf.py", line 18, in <module>
    from sphinx_markdown_parser.parser import MarkdownParser
ModuleNotFoundError: No module named 'sphinx_markdown_parser'

Although I'm still able to navigate to MarkdownParser function from

from sphinx_markdown_parser.parser import MarkdownParser

update

Tried with code example but still facing same issue

I'm a newbie in python, help me, please

Filter out yapf from install_requires in setup.py

Right now installing the package will also install yapf. I believe this is not intended as it's only used for development.