demizer / go-rst Goto Github PK

There is no way to to align text within the cells.

Consider creating a custom directive that uses GFM markdown table format or the table format from Textile.

Include "docutils_tests_converted=<yes|no>" in all progress.yml items.

• Lex
• Parse

https://github.com/mitchellh/mapstructure

In the checkParseNodes() function in parse_test.go there is a check that compares the number of parsed nodes to expected nodes from the json test data and produces a critical error if the counts do not match.

Using len(eTree) does not include nested nodes in NodeLists.

The lexer functions need to be properly tested, similar to how the parser functions are tested.

peek()
peekBack()
next()
backup()

While implementing inline markup, I found a few problems with next() and backup() that should have been found with proper testing!

Follows #9 and #10

Documentation would only be developer documentation for integrating go-rst with other go projects. The upstream docutils documents restructuredtext adequately.

• Lex
• Parse

Go's package management is lame. Glide makes is slightly better so go-rst should use it.

I just ran into an error where the lexer tests failed but the parser tests still ran and resulted in a panic.

• Lex
• Parse

Move the table to it's own document, but leave the percentage and link to the document containing the table.

• Lex
• Parse

Support exporting to ansi. See https://github.com/Snaipe/python-rst2ansi

• Convert docutils tests into JSON
• Finish parser support
• Write tests

Does the date mean the project was modified on that date?... It would probably seem confusing to someone not familiar with the setup so it should be removed. That's what Git is for!

• Convert the docutils paragraph tests to JSON.
• Finish implementing paragraph parsing.

It should have badge bling and a screenshot. Everyone loves screen shots. See go-elog project for inspiration.

Because it's written in Go and awesome.

Finally, a proper build tool. http://getgb.io/about/

• Lex
• Parse

Requies #10

The blog post should cover:

1. Converting docutils tests to json
   1. Problems with docutils tests
      • psuedo xml not easily portable
   2. testdata
      1. portable rst tests are easier to use.
         • Lex items are easy to setup with separate rst file.
         • UltiSnips for fast *-items.json and *-node.json file creation.
      2. comparing json objects is painful, but not too bad. Highlight recursive compare code.
2. Test infrastructure
   1. testdata directory structure
      • Good and Bad tests
   2. Debug output
   3. go debugger? delve?
   4. calling test data

      • test.go files and adding a new test.

      • running a single test

      • debugging a test problem
        Remove lines 442-451 in parse_test.go to produce a panic.
        Readd the lines and show the following error:

        [demizer@lithium]$ gtr TestParseCommentNoBlankLineBad0000
        === RUN TestParseCommentNoBlankLineBad0000
        
        
        Sat Dec 06 13:31:06 PST 2014 :: [CRITICAL] 3 Parse Nodes
        
        ([]parse.Node) (len=3 cap=4) {
          (*parse.CommentNode)(0xc2080b5980)({
              ID: (parse.ID) 1,
              Type: (parse.NodeType) 8,
              Text: (string) (len=9) "A comment",
              Length: (int) 9,
              StartPosition: (parse.StartPosition) 4,
              Line: (parse.Line) 1
          }),
          (*parse.ParagraphNode)(0xc2080b5bc0)({
              ID: (parse.ID) 2,
              Type: (parse.NodeType) 1,
              Text: (string) (len=13) "no blank line",
              Length: (int) 13,
              Line: (parse.Line) 2,
              StartPosition: (parse.StartPosition) 1
          }),
          (*parse.ParagraphNode)(0xc2080b5e00)({
              ID: (parse.ID) 3,
              Type: (parse.NodeType) 1,
              Text: (string) (len=12) "A paragraph.",
              Length: (int) 12,
              Line: (parse.Line) 4,
              StartPosition: (parse.StartPosition) 1
          })
        }
        
        
        Sat Dec 06 13:31:06 PST 2014 :: [CRITICAL] 4 Expected Nodes
        
        ([]interface {}) (len=4 cap=4) {
          (map[string]interface {}) (len=6) {
              (string) (len=13) "startPosition": (float64) 4,
              (string) (len=4) "line": (float64) 1,
              (string) (len=6) "length": (float64) 9,
              (string) (len=2) "id": (float64) 1,
              (string) (len=4) "type": (string) (len=11) "NodeComment",
              (string) (len=4) "text": (string) (len=9) "A comment"
          },
          (map[string]interface {}) (len=5) {
              (string) (len=4) "type": (string) (len=17) "NodeSystemMessage",
              (string) (len=8) "severity": (string) (len=7) "WARNING",
              (string) (len=4) "line": (float64) 2,
              (string) (len=8) "nodeList": ([]interface {}) (len=1 cap=1) {
                  (map[string]interface {}) (len=4) {
                      (string) (len=2) "id": (float64) 3,
                      (string) (len=4) "type": (string) (len=13) "NodeParagraph",
                      (string) (len=4) "text": (string) (len=63) "Explicit markup ends without a blank line; unexpected unindent.",
                      (string) (len=6) "length": (float64) 63
                  }
              },
              (string) (len=2) "id": (float64) 2
          },
          (map[string]interface {}) (len=6) {
              (string) (len=6) "length": (float64) 13,
              (string) (len=2) "id": (float64) 4,
              (string) (len=4) "type": (string) (len=13) "NodeParagraph",
              (string) (len=4) "text": (string) (len=13) "no blank line",
              (string) (len=13) "startPosition": (float64) 1,
              (string) (len=4) "line": (float64) 2
          },
          (map[string]interface {}) (len=6) {
              (string) (len=4) "text": (string) (len=11) "A paragraph",
              (string) (len=13) "startPosition": (float64) 1,
              (string) (len=4) "line": (float64) 4,
              (string) (len=6) "length": (float64) 11,
              (string) (len=2) "id": (float64) 5,
              (string) (len=4) "type": (string) (len=13) "NodeParagraph"
          }
        }
        
        
        Sat Dec 06 13:31:06 PST 2014 :: [CRITICAL] The number of parsed nodes does not match expected nodes!
        exit status 1
        FAIL  github.com/demizer/go-rst/parse 0.005s
        

The party needs to be broken up int smaller parties.

go-rst needs a proper project website.

The site will be built using Hugo, in markdown initially, but hopefully in the future with go-rst.

The http://restructuredtext.com domain has been acquired for this purpose. The project page should host an online editor and restructuredtext validator using Python, Nim, and Go. See rst.ninjs.org for inspiration.

• Use CloudFlare
• Compile go-rst to Javascript using GopherJS
  • WYSIWYG editor
  • View lex items
  • View parse nodes
• Use hyde for static generation
• Public API
  • Parse to HTML
  • Parse to JSON
• Blog
• Links to docutils spec and various sites.

The project page should also contain a link to the docutils project page and the spec.

I pulled the percentages out of my ass. There must be a way of calculating accurate metrics for the percentage of the spec that has been implemented.

Web frameworks such has martini, revel, and hugo. Explore their API and determine how it can be used with go-rst.

• Lex
• Parse

Many of the lexing function contain this block:

TestLexCommentInDefinitionGood0400
    for {
        l.next()
        if l.isEndOfLine() && l.mark == utf8.RuneError {
            l.emit(itemTerm)
            break
        }
    }

This should be reduced down to a function: emitCurrentLine(t token) or something.

Some parts of the code use named return values, AKA naked return values. This make the code harder to read and is not considered a best practice.

https://plus.google.com/+WilliamKennedy/posts/8hMjHhmyNk2

Relates to #12 and #14

Review the document and make sure it is up-to-date.

Just enough for hyperlink references

• Lex
• Parse

parse_test.go is magical and does comparisons of JSON objects to Go objects, but is barely documented. After not looking at it for four months some of it is kind of fuzzy.

Some parts of http://jsonapi.org/format/ seem like a good fit for the testdata JSON objects.

• Convert docutils tests to JSON.
• Add support for inline markup in the parser.
• Add tests.

Benchmark the parser using the benchmark package.

It seems really good. But it's 8% implementation. There is any plan or roadmap?

• Lex
• Parse

• Convert docutils tests to JSON.
• Finish implementing comment parsing.

Currently, tests are called with hard-coded paths:

    testPath := "test_section/02-title-bad/" +
        "02.02_short_title_short_overline_missing_underline"

Ideally, it should be possible to call a test by the actual file name of the test instead of having using a "testPath" variable.

So for example:

func TestShortTitleWithShortOverlineMissingUnderline0202(t *testing.Test) {
        run_test("short_title_short_overline_missing_underline")
}

The parser currently does not recognize tabstops at all. The spec says all tabstop should be converted into eight spaces.

• Code Blocks
  • Lex
  • Parse
  • For syntax highlighting, use Go based language parser and fallback to pygments.
• Image Blocks
  • Lex
  • Parse
• Admonitions
  • Lex Generic Admonition
  • Parse Generic Admonition
  • Lex Attention
  • Parse Attention
  • Lex Caution
  • Parse Caution
  • Lex Danger
  • Parse Danger
  • Lex Error
  • Parse Error
  • Lex Hint
  • Parse Hint
  • Lex Important
  • Parse Important
  • Lex Note
  • Parse Note
  • Lex Tip
  • Parse Tip
  • Lex Warning
  • Parse Warning

The debug output "START" and "END" for every function is useful while debugging, but is it necessary?
Explore other methods for studying the execution path of the library.

An API for Item insertion into the token ring buffer. This will help clean up systemMessage().

When parsing an itemBlockQuote, the text of the item needs to be added onto the token ring manually so that the next go around in the parser will add the text as a paragraph to the NodeList of the blockquote. systemMessage() does this manually and the code doesn't read well.

The IsSpace() function in parser.go should be replaced with the unicode.IsSpace() standard library function.

Hi there,

Just want to ask is it functioning cause I have needs for rst parser in Go

Thanks,
Joe

Look to docutils for inspiration on the API that should be used to interface with the package.

Once all of the basic section tests are passing, we should implement the full API for transforming a reStructuredText document into a valid HTML webpage, or JSON output, with a working example.

• Pluggable internal API for easily adding new transformers
• Transform to Json
• Transform to HTML