lepture / mistune

A fast yet powerful Python Markdown parser with renderers and plugins.

Mistune v3

A fast yet powerful Python Markdown parser with renderers and plugins.

NOTE: This is the re-designed v3 of mistune.

Looking for old Mistune? Switch branch to:

• v1
• v2

Paid plugins

You can ask me to create a custom mistune plugin or directive for your needs with GitHub sponsor one time tier (Mistune enhance)

Sponsors

Mistune is sponsored by Typlog, a blogging and podcast hosting platform, simple yet powerful. Write in Markdown.

Support Me via GitHub Sponsors.

Install

To install mistune:

$ pip install mistune

Overview

Convert Markdown to HTML with ease:

import mistune

mistune.html(your_markdown_text)

Security Reporting

If you found security bugs, please do not send a public issue or patch. You can send me email at [email protected]. Attachment with patch is welcome. My PGP Key fingerprint is:

72F8 E895 A70C EBDF 4F2A DFE0 7E55 E3E0 118B 2B4C

Or, you can use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

Benchmarks

Here is the benchmark score on my computer. Check the benchmark/bench.py script.

mistune (3.0.0) - atx: 13.901472091674805ms
mistune (slow) - atx: 13.122797012329102ms
mistune (fast) - atx: 13.248443603515625ms
mistune (full) - atx: 15.445232391357422ms
markdown (3.3.7) - atx: 48.41303825378418ms
markdown2 (2.4.3) - atx: 379.30870056152344ms
mistletoe (0.8.2) - atx: 25.46215057373047ms
markdown_it (2.1.0) - atx: 42.37723350524902ms
mistune (3.0.0) - setext: 8.43048095703125ms
mistune (slow) - setext: 8.97979736328125ms
mistune (fast) - setext: 8.122920989990234ms
mistune (full) - setext: 9.525299072265625ms
markdown (3.3.7) - setext: 30.74812889099121ms
markdown2 (2.4.3) - setext: 218.90878677368164ms
mistletoe (0.8.2) - setext: 20.46680450439453ms
markdown_it (2.1.0) - setext: 27.010202407836914ms
mistune (3.0.0) - normal_ul: 60.910940170288086ms
mistune (slow) - normal_ul: 59.69667434692383ms
mistune (fast) - normal_ul: 60.41216850280762ms
mistune (full) - normal_ul: 62.89219856262207ms
markdown (3.3.7) - normal_ul: 83.7857723236084ms
markdown2 (2.4.3) - normal_ul: 175.36139488220215ms
mistletoe (0.8.2) - normal_ul: 74.82385635375977ms
markdown_it (2.1.0) - normal_ul: 103.0113697052002ms
mistune (3.0.0) - insane_ul: 104.1865348815918ms
mistune (slow) - insane_ul: 105.83090782165527ms
mistune (fast) - insane_ul: 103.03664207458496ms
mistune (full) - insane_ul: 105.80086708068848ms
markdown (3.3.7) - insane_ul: 133.82673263549805ms
markdown2 (2.4.3) - insane_ul: 337.23902702331543ms
mistletoe (0.8.2) - insane_ul: 122.10249900817871ms
markdown_it (2.1.0) - insane_ul: 85.92629432678223ms
mistune (3.0.0) - normal_ol: 25.092601776123047ms
mistune (slow) - normal_ol: 25.321483612060547ms
mistune (fast) - normal_ol: 25.11453628540039ms
mistune (full) - normal_ol: 25.945663452148438ms
markdown (3.3.7) - normal_ol: 43.30158233642578ms
markdown2 (2.4.3) - normal_ol: 75.87885856628418ms
mistletoe (0.8.2) - normal_ol: 33.63537788391113ms
markdown_it (2.1.0) - normal_ol: 40.307044982910156ms
mistune (3.0.0) - insane_ol: 46.201229095458984ms
mistune (slow) - insane_ol: 49.14569854736328ms
mistune (fast) - insane_ol: 45.96853256225586ms
mistune (full) - insane_ol: 47.544002532958984ms
markdown (3.3.7) - insane_ol: 50.154924392700195ms
markdown2 (2.4.3) - insane_ol: 210.48712730407715ms
mistletoe (0.8.2) - insane_ol: 84.07974243164062ms
markdown_it (2.1.0) - insane_ol: 83.61554145812988ms
mistune (3.0.0) - blockquote: 15.484809875488281ms
mistune (slow) - blockquote: 16.12544059753418ms
mistune (fast) - blockquote: 15.350818634033203ms
mistune (full) - blockquote: 16.104936599731445ms
markdown (3.3.7) - blockquote: 63.04144859313965ms
markdown2 (2.4.3) - blockquote: 702.4445533752441ms
mistletoe (0.8.2) - blockquote: 28.56159210205078ms
markdown_it (2.1.0) - blockquote: 37.35041618347168ms
mistune (3.0.0) - blockhtml: 7.898569107055664ms
mistune (slow) - blockhtml: 7.080316543579102ms
mistune (fast) - blockhtml: 7.414340972900391ms
mistune (full) - blockhtml: 8.559703826904297ms
markdown (3.3.7) - blockhtml: 46.65660858154297ms
markdown2 (2.4.3) - blockhtml: 122.09773063659668ms
mistletoe (0.8.2) - blockhtml: 12.23611831665039ms
markdown_it (2.1.0) - blockhtml: 26.836156845092773ms
mistune (3.0.0) - fenced: 4.281282424926758ms
mistune (slow) - fenced: 4.092931747436523ms
mistune (fast) - fenced: 4.024267196655273ms
mistune (full) - fenced: 4.453897476196289ms
markdown (3.3.7) - fenced: 33.83779525756836ms
markdown2 (2.4.3) - fenced: 92.49091148376465ms
mistletoe (0.8.2) - fenced: 9.19342041015625ms
markdown_it (2.1.0) - fenced: 12.503623962402344ms
mistune (3.0.0) - paragraph: 95.94106674194336ms
mistune (slow) - paragraph: 561.2788200378418ms
mistune (fast) - paragraph: 93.597412109375ms
mistune (full) - paragraph: 110.09836196899414ms
markdown (3.3.7) - paragraph: 304.1346073150635ms
markdown2 (2.4.3) - paragraph: 267.84825325012207ms
mistletoe (0.8.2) - paragraph: 779.3235778808594ms
markdown_it (2.1.0) - paragraph: 825.5178928375244ms
mistune (3.0.0) - emphasis: 23.591041564941406ms
mistune (slow) - emphasis: 16.934871673583984ms
mistune (fast) - emphasis: 23.232460021972656ms
mistune (full) - emphasis: 25.2840518951416ms
markdown (3.3.7) - emphasis: 76.50399208068848ms
markdown2 (2.4.3) - emphasis: 9.393930435180664ms
mistletoe (0.8.2) - emphasis: 33.68663787841797ms
markdown_it (2.1.0) - emphasis: 60.90521812438965ms
mistune (3.0.0) - auto_links: 3.7980079650878906ms
mistune (slow) - auto_links: 3.3910274505615234ms
mistune (fast) - auto_links: 3.6630630493164062ms
mistune (full) - auto_links: 3.9186477661132812ms
markdown (3.3.7) - auto_links: 23.04673194885254ms
markdown2 (2.4.3) - auto_links: 6.537914276123047ms
mistletoe (0.8.2) - auto_links: 8.360624313354492ms
markdown_it (2.1.0) - auto_links: 19.732236862182617ms
mistune (3.0.0) - std_links: 21.920442581176758ms
mistune (slow) - std_links: 17.487764358520508ms
mistune (fast) - std_links: 19.87743377685547ms
mistune (full) - std_links: 24.514198303222656ms
markdown (3.3.7) - std_links: 39.1237735748291ms
markdown2 (2.4.3) - std_links: 14.519691467285156ms
mistletoe (0.8.2) - std_links: 22.84979820251465ms
markdown_it (2.1.0) - std_links: 32.60660171508789ms
mistune (3.0.0) - ref_links: 47.673940658569336ms
mistune (slow) - ref_links: 39.449214935302734ms
mistune (fast) - ref_links: 44.81911659240723ms
mistune (full) - ref_links: 52.37579345703125ms
markdown (3.3.7) - ref_links: 87.65625953674316ms
markdown2 (2.4.3) - ref_links: 23.118972778320312ms
mistletoe (0.8.2) - ref_links: 59.136390686035156ms
markdown_it (2.1.0) - ref_links: 80.44648170471191ms
mistune (3.0.0) - readme: 56.607723236083984ms
mistune (slow) - readme: 68.8173770904541ms
mistune (fast) - readme: 53.86018753051758ms
mistune (full) - readme: 61.25998497009277ms
markdown (3.3.7) - readme: 211.02523803710938ms
markdown2 (2.4.3) - readme: 533.4112644195557ms
mistletoe (0.8.2) - readme: 110.12959480285645ms
markdown_it (2.1.0) - readme: 247.9879856109619ms

License

Mistune is licensed under BSD. Please see LICENSE for licensing details.

mistune's Issues

false detection of inline comment

any email address starting with something from _inline_tag won't get recognized

e.g.

<[email protected]>
<[email protected]>

Create a Jinja2 extension to be used with Mistune

Something like this one https://github.com/danielchatfield/jinja2_markdown but for Mistune:

{% markdown %}
  Hello World
  ---------------
  - Foo
  - Bar
{% endmarkdown %}

and it could also work as a filter (but not really needed for Markdown as the above case is much better):

{{some_text|markdown}}

If someone has some extra time it would be super cool to have :)

Take care of mistune users

Mistune is not stable yet. If I am going to make some breaking changes, I should notify these projects.

1. IPython (nbcovert) https://github.com/ipython/ipython https://github.com/jupyter/nbconvert @takluyver @minrk
2. crossbar https://github.com/crossbario/crossbar @oberstet
3. https://github.com/Hammerspoon/hammerspoon
4. https://github.com/lektor/lektor @nixjdm

Some parsing troubles

When I do this

mistune.markdown(u'<span class="">[tex\'t](http://example.com)\n</span>')

then i get wrong result

u'<p><span class="">[tex\'t](http://example.com)\n</span></p>\n' 

LaTeX equation should not be parsed when it is enclosed in span tags

import mistune

md = mistune.Markdown(parse_html=False)
print(md.render('''

<div>$$x_y x_{y}$$</div>

<span>$$x_y x_{y}$$</span>

'''))

# Prints:
# <div>$$x_y x_{y}$$</div>
# <p><span>$$x<em>y x</em>{y}$$</span></p>

The LaTeX equation is parsed when it is in the span, not when it is in the div. I think it should not be parsed since it is within HTML tags.

line-breaks break italic/bold

thorstenb/odpdown#12

Simple footnotes

Hi,
I think, adding tiny footnotes are a little bit difficult. I mean, it would be much simpler if you could do something like this:

Here goes a foreign word[^its translation].

Clearly, adding long footnotes in this format doesn't make sense, but it could be helpful for short ones. I've changed InlineLexer so it doesn't check footnote occurrence:

class MyInlineLexer(mistune.InlineLexer):
    def output_footnote(self, m):
        key = mistune._keyify(m.group(1))
        self.footnote_index += 1
        self.footnotes[key] = self.footnote_index
        return self.renderer.footnote_ref(key, self.footnote_index)

But I can't add corresponding footnote_item to footnote list. Actually, I don't want to rewrite major parts of mistune and I'm asking for a compatible solution with current footnote handling.

Parsing without rendering

Is it possible to get AST of the document without rendiring it to HTML?

IPython refer

ipython/ipython#7929

Markdown token tree

ref: #20

Support kramdown extensions (non content elements)

kramdown provides a quite interesting syntax and might be nice to extend the parser to support it.

Hopefully I can contribute some of it if there is interest and time.

Markdown (re)renderer

It's mainly a question, if you like to add the markdown renderer to your code. I created one to share for markdown auto-recoding purposes (for example internationalization).

Setext-style headings with length 1 underlines don't get parsed correctly

I'm not sure if this is intended because otherwise it would be problematic, but why does the lheading regular expression require headers to be underlined with at least 2 characters?

lheading = re.compile(r'^([^\n]+)\n *(=|-){2,} *(?:\n+|$)')

while the syntax reference at Daring Fireball states that:

Any number of underlining =’s or -’s will work.

This causes this legal heading (at least initially):

a
=

to be parsed as a regular paragraph.

Provide a mean to export to commonly consumed parsing trees

• docutils as done by remarkdown.
• emerald-tree to provide moin-2 with a better syntax.

Lake of extension API of preprocessing

default_features -> default_rules

Hi there !

Are you on a strict 'no backward compat' (as the version number is < 1.x) ?

I guess we miht try to reintroduce default_features as a class property that print a warning and maps to default_rules ?

Just saying that as all our CI test are failing since 0,5 release, but not a big deal we can update and.or pin mistune version.

Thanks !

mistune .6 breaks HTML rendering of Jupyter notebooks

We had a user notice an issue with mistune .6. They were able to confirm that downgrading to mistune 0.5.1 solved the problem. Posting here so it can be addressed.
ContinuumIO/anaconda-issues#364

Hook for start of rendering

When I keep state in a custom renderer, to render the first code block after headlines matching a certain pattern differently from other code blocks, how do I make sure to reset when I reuse the renderer?

I mean, there seems to be no hook document that fires when a new source is rendered.

Traceback: 'DocPageRenderer' object has no attribute 'rstrip

I am getting a traceback with 0.3.1 (which works with 0.3.0):

Traceback (most recent call last):
  File "website/crossbario/__init__.py", line 191, in <module>
    pages = DocPages('../crossbar.wiki')
  File "website/crossbario/__init__.py", line 153, in __init__
    self._renderer = mistune.Markdown(renderer = rend, inline = inline)
  File "mistune.py", line 899, in mistune.Markdown.__init__ (mistune.c:16104)
    inline = inline(renderer, **kwargs)
  File "mistune.py", line 502, in mistune.InlineLexer.__call__ (mistune.c:8497)
    return self.output(src)
  File "mistune.py", line 510, in mistune.InlineLexer.output (mistune.c:9059)
    src = src.rstrip('\n')
AttributeError: 'DocPageRenderer' object has no attribute 'rstrip'
make: *** [test] Error 1

Make it easy to extend

Currently, you should write a custom BlockLexer, a InlineLexer, a Renderer, it is annoying. There should be a way to patch the markdown instance.

Benchmark

Read this: http://lepture.com/en/2014/markdown-parsers-in-python

Enable all features, without Cython.

Parsing the Markdown Syntax document 1000 times...
Mistune: 12.9425s
Misaka: 0.537176s
Markdown: 47.7091s
Markdown2: 80.5163s
cMarkdown: 0.680664s
Discount is not available

Parsing the Markdown Syntax document 1000 times...
Mistune: 12.7255s
Misaka: 0.553476s
Markdown: 47.9369s
Markdown2: 79.5075s
cMarkdown: 0.71733s
Discount is not available

With Cython

Parsing the Markdown Syntax document 1000 times...
Mistune: 9.78976s
Misaka: 0.528333s
Markdown: 46.3091s
Markdown2: 76.3403s
cMarkdown: 0.674922s
Discount is not available

Parsing the Markdown Syntax document 1000 times...
Mistune: 9.74075s
Misaka: 0.550502s
Markdown: 46.4342s
Markdown2: 78.2267s
cMarkdown: 0.664128s
Discount is not available

Benchmark with PyPy

Parsing the Markdown Syntax document 1000 times...
Mistune: 8.36888s
Misaka is not available
/Users/lepture/.venvs/pypy/site-packages/markdown/blockprocessors.py:151: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
  (len(parent) and parent[-1] and \
/Users/lepture/.venvs/pypy/site-packages/markdown/blockparser.py:94: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
  if processor.test(parent, blocks[0]):
Markdown: 30.9723s
Markdown2: 88.4563s
cMarkdown is not available
Discount is not available

Parsing the Markdown Syntax document 1000 times...
Mistune: 8.31194s
Misaka is not available
/Users/lepture/.venvs/pypy/site-packages/markdown/blockprocessors.py:151: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
  (len(parent) and parent[-1] and \
/Users/lepture/.venvs/pypy/site-packages/markdown/blockparser.py:94: FutureWarning: The behavior of this method will change in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
  if processor.test(parent, blocks[0]):
Markdown: 31.6023s
Markdown2: 85.5509s
cMarkdown is not available
Discount is not available

https://github.com/lepture/mistune/blob/master/tests/bench.py

mistune fails the standard markdown compliance test

I don't know if this is relevant for you but the test against the standard markdown test suite ends with 155 passed and 286 failed tests. I used the following script to convert the input:

#!/usr/bin/env python

import mistune
import sys

print(mistune.markdown(sys.stdin.read()))

Add font tag to _inline_tag

Please, add font tag to _inline_tag string.

Why is all the documentation using reStructuredText?

That is, why isn't mistune being used to render it's own documentation? That seems as if it would be a good test case.

Custom inline lexer example

Would be great to have an example for doing a custom inline lexer.

The motiviation is to process GitHub Wiki flavored inter-page links like [[Page 2|Page 2]].

That would mean a hook that I can override to resolve and render those links.

Footnotes can only be referenced once

Sometimes a footnote can be referenced from more than one location, like this:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.[^1]
Praesent suscipit metus a quam congue facilisis.[^1]

[^1]: Lorem Ipsum is simply dummy text of the printing and typesetting industry.

But in mistune only the first reference is rendered, while later reference markups are left as-is. AFAIK all other popular implementations (e.g. StackEdit, Python Markdown, Hoedown, etc.) renders both [^1] occurrences as reference marks.

Does mistune support code/syntax highlighting? [question]

^question. If so, how can I use Pygments or some other library? Thanks!

@phoikoi use the renderer feature

Hi @phoikoi

I found that you were modifying mistune source to add some weird things. It can be easy done with a custom renderer. Checkout this part: https://github.com/lepture/mistune#renderer

All you need is subclass of Renderer and overwrite the image function. Please don't modify the source code like this https://github.com/phoikoi/mistune/commit/38f296b400387885493bee296b2962b9f356d846

Inline code does not trim whitespaces surrounding content properly

According to the Markdown specification:

The backtick delimiters surrounding a code span may include spaces — one after the opening, one before the closing. This allows you to place literal backtick characters at the beginning or end of a code span[.]

Mistune however renders the following markup:

`` ` ``

as

<p><code>` </code></p>

with an extra space at the end of the code span.

Span is not recognized as an HTML block

import mistune

class MyRenderer(mistune.Renderer):
    def block_html(self, html):
        print("---- This is a block:", html)
        return html

renderer = MyRenderer()
md = mistune.Markdown(renderer=renderer)
md.render('''
First line.

<div>A div block.</div>

<span>A span block.</span>

''')

I would expect the two div and span blocks to be detected as HTML blocks. Currently, only the div block is recognized as a block. The span block is recognized as a paragraph. I'm not sure if it's a bug or a feature actually!

Autolinks without a / in the URI are not matched

There are many URI schemes that do not have a / in them.

https://en.wikipedia.org/wiki/URI_scheme

moving from misaka - header anchors

I am trying to figure out is mistune can be a replacement for this use case:

html = misaka.html( md,
    misaka.EXT_STRIKETHROUGH | misaka.EXT_AUTOLINK | misaka.EXT_FENCED_CODE | \
    misaka.EXT_TABLES,
    misaka.HTML_SMARTYPANTS | misaka.HTML_TOC )

It looks like HTML_TOC is not supported, and I am not sure about misaka.HTML_SMARTYPANTS

For those moving from misaka, it will be handy to have a table with mapped methods/properties.

Output of _____Z_

Found a case here: trentm/python-markdown2#183

GitHub will render _____Z_ as:

<p>_____Z_</p>

Currently, mistune will render it as:

<p><strong>_</strong>Z_</p>

I thought it should be rendered as:

<p><em>____Z</em></p>

MathJax "blocks" not like expected.

Over on IPython, we've run into a regression from our previous markdown toolchain.

Previously, something like this:

$math$
$$
big math
$$
$other math

Would render up an inline, a block and an inline. At present, the recipe in contrib and in the tests here, as implemented in IPython, creates interesting problems, as something happens to this pattern, and it doesn't render properly:
jupyter/nbviewer#420
jupyter/nbviewer#422

We "fixed" it by making inline ALSO match block math:
https://github.com/ipython/ipython/pull/8066/files

If you have any insight into how we might fix this more elegantly, that would be greatly appreciated!

Problem with integrating Emoji support

Hey!

First of all, thanks for your work!

I am trying to integrate mistune into my flask extension (https://github.com/sh4nks/flask-emoji) but I have some troubles to get it working. The standalone renderer (without mistune) works fine.

The code for the integration: https://github.com/sh4nks/flask-emoji/blob/master/flask_emoji/ext/mistune_renderer.py
I just followed your the example and studied the source a bit (which unfortunately didn't help)
and this is how I initialize it: https://github.com/sh4nks/flask-emoji/blob/master/flask_emoji/__init__.py#L189-L199

This is the output when I use it (it just renders the first appearance of an emoji and ignores the ones):

Single Emoji: ':smile:'
<p><img src="emoji/smile.png" alt="smile" title="smile"></p>

Cats and Dogs: 'It's raining :cat:s and :dog:s'
<p>It's raining :cat:s and :dog:s</p>

Multiple in one line: ':smile: :smiley:'
<p><img src="emoji/smile.png" alt="smile" title="smile"> :smiley:</p>

Other: '![test](test.png) ![test2](test2.png)'
<p><img src="test.png" alt="test"> <img src="test2.png" alt="test2"></p>

Could you help me to get it working? 😉
Thanks!

Question about writing renderer

Hello,

I don't want to go through the trouble of writing my own markdown parser!

But I want a tool that converts markdown to mdoc. And it looks like all the python
markdown parsers just render to HTML.

Can you explain to me how I can extend mistune to render some other format that's
not really similar to HTML?

Or is there some other python markdown parser you would recommend for this problem?

My project is MIT licensed, can I include your code/module in my project?

Greetings

Lexer question

Hello, I'm trying to figure out the lexers of the mistune. I need a simple extension that concatenates subsequent lines of text if there is a '' symbol between them, like this:

    I'm working on documentation generator, so it is \
    important to keep markdown lines short without losing\
    the result formatting.

The trick is that line concatenator should be insensitive to the whitespace: it should ignore the spaces around the delimiter "" in the first line and on the beginning of the second line.

I already have a Python Markdown version of this extension and it looks like this:

class LinesConnector(Extension):

    """ == Lines connector extension == """

    def __init__(self, regex=r"(\S)\s*\\\s*\n\s*(\S)", sub=r"\1 \2", *args, **kwargs):

        regex = re.compile(regex, flags=re.M)

        class Prep(Preprocessor):

            def run(self, lines):
                """ Method ensures that there is exactly one space between 2 joined strings. """
                return regex.sub(sub, "\n".join(lines)).split("\n")

        self.Prep = Prep()

        super(LinesConnector, self).__init__(*args, **kwargs)

    def extendMarkdown(self, md, md_globals):
        md.preprocessors.add('lines-connector', self.Prep, '_end')

It is simply a paragraphs processor, but I can't figure out which method should be used. I tried both inline and block lexers, the text renderer and modification of internal regular expressions. With last two I succeeded in joining lines, but the spaces are still wrong.

How would you proceed with this?

A variant that automatically concatenates subsequent lines of text might be interesting too.

Math expressions

Usage of math expressions in Markdown text is quite common. I've found this PR markedjs/marked#180 in marked which brings this feature. Main issue with current implementation occurs in cases like this:

$ A_{i} + B_{i} $

That inner part of $s must not be rendered as a normal piece of Markdown text. Actually, mentioned PR is not merged yet but I think supporting this expressions only needs exclusion of $s content. For example, output of this simple case:

$ A_{i} $

Is completely compatible with MathJax which renders that as its equivalent expression.

Line break feature

There is a breaks option in marked for making GFM like line breaks. I see the linebreak renderer in your code, but it does render:

this
paragraph

as:

 <p>this
 paragraph</p>

actually, I want this:

 <p>this<br>
 paragraph</p>

mistune 0.7

The next release contains these changes:

https://github.com/lepture/mistune/blob/master/CHANGES.rst#version-07

1. Add two options to fix breaking changes in version 0.6: parse_inline_html and parse_block_html
2. Change option escape default value to False for security reason

@takluyver @minrk @apenwarr @oberstet

compare view: v0.6...3c95375

It will be released in the next week, maybe Jul. 10.

Renderer specific escape function

Is there any way of defining a renderer specific escape function. This is needed for latex rendering, because in latex different character should be escaped (like % and _).

URLs with parenthesis are broken

>>> import mistune
>>> import urllib
>>> import urlparse
>>> 
>>> url = 'https://trello-attachments.s3.amazonaws.com/550878b58559170febf8e69b/500x600/ff07160366332b3962320bdfb1693e3e/download_(2).jpg'
>>> mistune.markdown("here's a [broken URL](%s)" % url)
'<p>here\'s a <a href="https://trello-attachments.s3.amazonaws.com/550878b58559170febf8e69b/500x600/ff07160366332b3962320bdfb1693e3e/download_(2">broken URL</a>.jpg)</p>\n'
>>> urlp = urlparse.urlparse(url)
>>> mistune.markdown("now [it](%s) is not broken anymore" % (urlp.scheme + '://' + urlp.netloc + urllib.quote(urlp.path)))
'<p>now <a href="https://trello-attachments.s3.amazonaws.com/550878b58559170febf8e69b/500x600/ff07160366332b3962320bdfb1693e3e/download_%282%29.jpg">it</a> is not broken anymore</p>\n'

Version 0.6

• don't render text in inline html: #38
• line-breaks break italic/bold #48

Specific HTML attributes

Hi there,

first thanks for your great script! It's much cleaner than comparable scripts, well done! :)

Is there a possibility to add some html attributes like an ID or a class to an element?
I'm looking for something like this:
https://pythonhosted.org/Markdown/extensions/attr_list.html

Is such a features already implemented or is it my turn to be creative? :)

Thanks and best regards,
Felix

Commonmark support

I ran mistune over the standard specification and the results are

153 tests passed, 290 failed, 0 skipped.

(the reference C implementation is about 20x faster than mistune)

Benchmark for Hoep

https://github.com/Anomareh/Hoep

wheel package format

Would it be possible to have a "universal" wheel on pypi ?

Upload wheels to PyPI

Mistune seems to provide an optional C extension for optimization which makes it even more faster. I think it would be great if we can install optimized prebuilt binary distribution by default.

Discussion also is needed to make decision which major platforms to support.

problem in escaping in 0.6

There seems to be a problem in mistune ≥ 0.6 escaping HTML when it shouldn't. For instance, the literal html <h1>text</h1> is passed through unmodified, exactly as it should be. However, the exact same html snippet is escaped if there is any other text outside the tag, instead of being left alone.

Test case:

from mistune import Markdown
md = Markdown(escape=False)
print(md.render("""<h1>not escaped</h1>"""))
print(md.render("""<h1>escaped</h1>
text"""))

Has output:

<h1>not escaped</h1>
<p>&lt;h1&gt;escaped&lt;/h1&gt;
text</p>

Closing tags are escaped

with: mistune.markdown(markdown, escape=False)

markdown:

<a name="top"></a>
#![Maintenance](title-icon.png) Maintenance#

is converted to:

<p><a name="top">&lt;/a&gt;</p>
<h1><img src="title-icon.png" alt="Maintenance"> Maintenance</h1>

closing link tag is escaped.