seikichi / restructured

Pure JavaScript reStructuredText parser

restructured

A reStructuredText parser for JavaScript.

Online Demo

demo.

Example

> npm install -g restructured
> echo 'Hello, world' | restructured | jq
{
  "type": "document",
  "children": [
    {
      "type": "paragraph",
      "children": [
        {
          "type": "text",
          "value": "Hello, world\n"
        }
      ]
    }
  ]
}

Consider the following script.

import restructured from 'restructured';

const parsed = restructured.parse(`
=====
Title
=====

Subtitle
--------

This is a paragraph.

- This is item 1
- This is item 2
`);

console.log(JSON.stringify(parsed));

The above script outputs the following JSON.

{
  "type": "document",
  "children": [
    {
      "type": "section",
      "depth": 1,
      "children": [
        {
          "type": "title",
          "children": [
            {
              "type": "text",
              "value": "Title"
            }
          ]
        },
        {
          "type": "section",
          "depth": 2,
          "children": [
            {
              "type": "title",
              "children": [
                {
                  "type": "text",
                  "value": "Subtitle"
                }
              ]
            },
            {
              "type": "paragraph",
              "children": [
                {
                  "type": "text",
                  "value": "This is a paragraph.\n"
                }
              ]
            },
            {
              "type": "bullet_list",
              "bullet": "-",
              "children": [
                {
                  "type": "list_item",
                  "children": [
                    {
                      "type": "paragraph",
                      "children": [
                        {
                          "type": "text",
                          "value": "This is item 1\n"
                        }
                      ]
                    }
                  ]
                },
                {
                  "type": "list_item",
                  "children": [
                    {
                      "type": "paragraph",
                      "children": [
                        {
                          "type": "text",
                          "value": "This is item 2\n"
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Progress

• Document Structure
  • Document
  • Sections
  • Transitions
• Body Elements
  • Paragraphs
  • Bullet Lists
  • Enumerated Lists
  • Definition Lists
  • Field Lists
    • Bibliographic Fields
    • RCS Keywords
  • Option Lists
  • Literal Blocks
    • Indented Literal Blocks
    • Quoted Literal Blocks
  • Line Blocks
  • Block Quotes
  • Doctest Blocks
  • Tables
    • Grid Tables
    • Simple Tables
    • List tables
    • CSV tables
  • Explicit Markup Blocks
    • Footnotes
      • Auto-Numbered Footnotes
      • Auto-Symbol Footnotes
      • Mixed Manual and Auto-Numbered Footnotes
    • Citations
    • Hyperlink Targets
      • Anonymous Hyperlinks
    • Directives
    • Substitution Definitions
    • Comments
• Implicit Hyperlink Targets
• Inline Markup
  • Emphasis
  • Strong Emphasis
  • Interpreted Text
  • Inline Literals
  • Hyperlink References
    • Embedded URIs and Aliases
  • Inline Internal Targets
  • Footnote References
  • Citation References
  • Substitution References
  • Standalone Hyperlinks

License

MIT

restructured's Issues

a space after inline markup makes it - and content that follows - disappear

Hi @seikichi - first of all, awesome work and thanks for your effort! I have stumbled upon one problem while playing with the parser and decided to report it:

**test** 
test

results in nothing (tested in the live demo after downloading github master).

I assume it's a minor bug and not critical, but just wanted you to know.

Subtitles are included as nested sections

Technically, a subtitle is supposed to be a direct child of the document. However, restructured includes it a section, resulting in every section being nested an extra level.

Footnotes

I see footnotes aren't implemented. Are there next steps for this feature?

Feature request: support for Doctest Blocks

Thank you for creating this library! Are there any plans to add support for Doctest Blocks in the near future?

TypeError: Elements.InterpretedText is not a constructor

The last version no longer supports references:

$ echo 'Foo :ref:`bar`' | node_modules/.bin/restructured 
/.../node_modules/restructured/lib/Parser.js:5
 */"use strict";function _toConsumableArray(arr){if(Array.isArray(arr)){for(var i=0,arr2=Array(arr.length);i<arr.length;i++){arr2[i]=arr[i];}return arr2;}else{return Array.from(arr);}}function peg$subclass(child,parent){function ctor(){this.constructor=child;}ctor.prototype=parent.prototype;child.prototype=new ctor();}function peg$SyntaxError(message,expected,found,location){this.message=message;this.expected=expected;this.found=found;this.location=location;this.name="SyntaxError";if(typeof Error.captureStackTrace==="function"){Error.captureStackTrace(this,peg$SyntaxError);}}peg$subclass(peg$SyntaxError,Error);peg$SyntaxError.buildMessage=function(expected,found){var DESCRIBE_EXPECTATION_FNS={literal:function literal(expectation){return"\""+literalEscape(expectation.text)+"\"";},"class":function _class(expectation){var escapedParts="",i;for(i=0;i<expectation.parts.length;i++){escapedParts+=expectation.parts[i]instanceof Array?clas

TypeError: Elements.InterpretedText is not a constructor
    at peg$c204 (/.../node_modules/restructured/lib/Parser.js:5:15685)
    at peg$parseInterpretedText (/.../node_modules/restructured/lib/Parser.js:5:113786)
    at peg$parseInlineMarkup (/.../node_modules/restructured/lib/Parser.js:5:97357)
    at peg$parseParagraphBlock (/.../node_modules/restructured/lib/Parser.js:5:146266)
    at peg$parseParagraph (/.../node_modules/restructured/lib/Parser.js:5:144478)
    at peg$parseBodyElementExceptBlockQuote (/.../node_modules/restructured/lib/Parser.js:5:66586)
    at peg$parseBodyElement (/.../node_modules/restructured/lib/Parser.js:5:66817)
    at peg$parseDocument (/.../node_modules/restructured/lib/Parser.js:5:37736)
    at Object.peg$parse [as parse] (/.../node_modules/restructured/lib/Parser.js:8:2638)
    at Object.parse (/.../node_modules/restructured/lib/RST.js:21:33)

can't parse nested RST inside directive

Given the following example RST:

.. method:: db.auth()

   Allows a user to authenticate to the database from within the
   shell.

   The :method:`db.auth()` method can accept either:

   - the username and password.

     .. code-block:: javascript

        db.auth( <username>, <password> )

   - a user document that contains the username and password, and
     optionally, the authentication mechanism and a digest password
     flag.

   .. include:: /includes/apiargs/method-db.auth-param.rst

The current output from restructured is:

mbroadst@gorgor:~/Development/node/restructured (master %)$ ./bin/restructured.js < ./test.rst | jq
{
  "type": "document",
  "children": [
    {
      "type": "directive",
      "directive": "method",
      "children": [
        {
          "type": "text",
          "value": "db.auth()"
        },
        {
          "type": "text",
          "value": "Allows a user to authenticate to the database from within the"
        },
        {
          "type": "text",
          "value": "shell."
        },
        {
          "type": "text",
          "value": "The :method:`db.auth()` method can accept either:"
        },
        {
          "type": "text",
          "value": "- the username and password."
        },
        {
          "type": "text",
          "value": "  .. code-block:: javascript"
        },
        {
          "type": "text",
          "value": "     db.auth( <username>, <password> )"
        },
        {
          "type": "text",
          "value": "- a user document that contains the username and password, and"
        },
        {
          "type": "text",
          "value": "  optionally, the authentication mechanism and a digest password"
        },
        {
          "type": "text",
          "value": "  flag."
        },
        {
          "type": "text",
          "value": ".. include:: /includes/apiargs/method-db.auth-param.rst"
        }
      ]
    }
  ]
}

It makes the incorrect assumption that all data inside the directive is just a text element, and therefore ceases to continue deeply parsing potentially more RST.

If I remove the following line, then the output seems more logical:

mbroadst@gorgor:~/Development/node/restructured (master *%)$ ./bin/restructured.js < ./test.rst | jq
{
  "type": "document",
  "children": [
    {
      "type": "directive",
      "directive": "method",
      "children": [
        {
          "type": "text",
          "value": "db.auth()"
        }
      ]
    },
    {
      "type": "block_quote",
      "children": [
        {
          "type": "paragraph",
          "children": [
            {
              "type": "text",
              "value": "Allows a user to authenticate to the database from within the\n"
            },
            {
              "type": "text",
              "value": "shell.\n"
            }
          ]
        },
        {
          "type": "paragraph",
          "children": [
            {
              "type": "text",
              "value": "The "
            },
            {
              "type": "interpreted_text",
              "role": "method",
              "children": [
                {
                  "type": "text",
                  "value": "db.auth()"
                }
              ]
            },
            {
              "type": "text",
              "value": " method can accept either:\n"
            }
          ]
        },
        {
          "type": "bullet_list",
          "bullet": "-",
          "children": [
            {
              "type": "list_item",
              "children": [
                {
                  "type": "paragraph",
                  "children": [
                    {
                      "type": "text",
                      "value": "the username and password.\n"
                    }
                  ]
                },
                {
                  "type": "directive",
                  "directive": "code-block",
                  "children": [
                    {
                      "type": "text",
                      "value": "javascript"
                    }
                  ]
                },
                {
                  "type": "block_quote",
                  "children": [
                    {
                      "type": "paragraph",
                      "children": [
                        {
                          "type": "text",
                          "value": "db.auth( <username>, <password> )\n"
                        }
                      ]
                    }
                  ]
                }
              ]
            },
            {
              "type": "list_item",
              "children": [
                {
                  "type": "paragraph",
                  "children": [
                    {
                      "type": "text",
                      "value": "a user document that contains the username and password, and\n"
                    },
                    {
                      "type": "text",
                      "value": "optionally, the authentication mechanism and a digest password\n"
                    },
                    {
                      "type": "text",
                      "value": "flag.\n"
                    }
                  ]
                }
              ]
            }
          ]
        },
        {
          "type": "directive",
          "directive": "include",
          "children": [
            {
              "type": "text",
              "value": "/includes/apiargs/method-db.auth-param.rst"
            }
          ]
        }
      ]
    }
  ]
}

However, that's because its just returning null and bailing from processing the DirectiveBlock (I think, sorry I haven't used pegjs before it's been a bit confusing to read through)

Maintenance

Hi @seikichi, thank you for creating this library!

I'm just started working on React writer so I can use reStructuredText with GatsbyJS. I see this library isn't complete and several elements aren't implemented. Especially directives would be useful in my case (#11).

If I start sending PRs, do you have time to review them? Or should I rather fork this repository right away?

Thank you again for your time and effort! So far everything works great and I'm looking forward when I switch from Sphinx to GatsbyJS.

Directives need to distinguish between arguments, options, and content

The directive block is separated from the rest by a blank line, but the parser simply outputs text nodes ignoring blank lines.

e.g. the following rST…

.. figure:: larch.png
   :scale: 50

   The larch.

parses to

{
  "type": "document",
  "children": [
    {
      "type": "directive",
      "directive": "figure",
      "children": [
        {
          "type": "text",
          "value": "larch.png"
        },
        {
          "type": "text",
          "value": ":scale: 50"
        },
        {
          "type": "text",
          "value": "The larch."
        }
      ]
    }
  ]
}

License missing

Hi! I'm thinking of building a ReST + Firebase collaborative editor for fun - something like socrates.io but with ReST instead of Markdown.

It's just a fun little project, it would be open source and I have zero intention of commercialising it. Still, it would be good to know that you're okay with people playing with your code (and the "live demo" example) - so please consider adding a license to this project.

Thanks!

Title and subtitle don't work in README example

The title and subtitle are missing when I run the example in the README. The output I get is

{
  "type": "document",
  "children": [
    {
      "type": "section",
      "children": [
        {
          "type": "section",
          "children": [
            {
              "type": "paragraph",
              "children": [
                {
                  "type": "text",
                  "text": "This is a paragraph.\n"
                }
              ]
            },
            {
              "type": "bullet_list",
              "children": [
                {
                  "type": "list_item",
                  "children": [
                    {
                      "type": "paragraph",
                      "children": [
                        {
                          "type": "text",
                          "text": "This is item 1\n"
                        }
                      ]
                    }
                  ]
                },
                {
                  "type": "list_item",
                  "children": [
                    {
                      "type": "paragraph",
                      "children": [
                        {
                          "type": "text",
                          "text": "This is item 2\n"
                        }
                      ]
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

row:column coordinates

It'd be nice if the generated document could include data about the source, including the starting coordinates and ending coordinates (row:column). This would allow for use cases such as interactive editors.