vmg / sundown

Standards compliant, fast, secure markdown processing library in C

Sundown

Sundown is a Markdown parser based on the original code of the Upskirt library by Natacha Porté.

Features

• Fully standards compliant

  Sundown passes out of the box the official Markdown v1.0.0 and v1.0.3 test suites, and has been extensively tested with additional corner cases to make sure its output is as sane as possible at all times.

• Massive extension support

  Sundown has optional support for several (unofficial) Markdown extensions, such as non-strict emphasis, fenced code blocks, tables, autolinks, strikethrough and more.

• UTF-8 aware

  Sundown is fully UTF-8 aware, both when parsing the source document and when generating the resulting (X)HTML code.

• Tested & Ready to be used on production

  Sundown has been extensively security audited, and includes protection against all possible DOS attacks (stack overflows, out of memory situations, malformed Markdown syntax...) and against client attacks through malicious embedded HTML.

  We've worked very hard to make Sundown never crash or run out of memory under any input. Sundown renders all the Markdown content in GitHub and so far hasn't crashed a single time.

• Customizable renderers

  Sundown is not stuck with XHTML output: the Markdown parser of the library is decoupled from the renderer, so it's trivial to extend the library with custom renderers. A fully functional (X)HTML renderer is included.

• Optimized for speed

  Sundown is written in C, with a special emphasis on performance. When wrapped on a dynamic language such as Python or Ruby, it has shown to be up to 40 times faster than other native alternatives.

• Zero-dependency

  Sundown is a zero-dependency library composed of 3 .c files and their headers. No dependencies, no bullshit. Only standard C99 that builds everywhere.

Credits

Sundown is based on the original Upskirt parser by Natacha Porté, with many additions by Vicent Marti (@vmg) and contributions from the following authors:

Ben Noordhuis, Bruno Michel, Joseph Koshy, Krzysztof Kowalczyk, Samuel Bronson,
Shuhei Tanuma

Bindings

Sundown is available from other programming languages thanks to these bindings developed by our awesome contributors.

• Redcarpet (Ruby)
• RobotSkirt (Node.js)
• Misaka (Python)
• ffi-sundown (Ruby FFI)
• Sundown HS (Haskell)
• Goskirt (Go)
• Upskirt.go (Go)
• MoonShine (.NET)
• PHP-Sundown (PHP)
• Sundown.net (.NET)

Help us

Sundown is all about security. If you find a (potential) security vulnerability in the library, or a way to make it crash through malicious input, please report it to us, either directly via email or by opening an Issue on GitHub, and help make the web safer for everybody.

Unicode character handling

Given that the Markdown spec makes no provision for Unicode character handling, Sundown takes a conservative approach towards deciding which extended characters trigger Markdown features:

• Punctuation characters outside of the U+007F codepoint are not handled as punctuation. They are considered as normal, in-word characters for word-boundary checks.

• Whitespace characters outside of the U+007F codepoint are not considered as whitespace. They are considered as normal, in-word characters for word-boundary checks.

Install

There is nothing to install. Sundown is composed of 3 .c files (markdown.c, buffer.c and array.c), so just throw them in your project. Zero-dependency means zero-dependency. You might want to include render/html.c if you want to use the included XHTML renderer, or write your own renderer. Either way, it's all fun and joy.

If you are hardcore, you can use the included Makefile to build Sundown into a dynamic library, or to build the sample sundown executable, which is just a commandline Markdown to XHTML parser. (If gcc gives you grief about -fPIC, e.g. with MinGW, try make MFLAGS= instead of just make.)

License

Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

sundown's Issues

Underscores in URLs end the link

Check the link in this issue:
ThinkUpLLC/ThinkUp#728

It looks like the new parser ignores underlines in URLs.

bufprintf/vbufprintf corruption issue

Related to: https://github.com/spladug/snudown/issues/1 and #75

In the event of the need to grow the buffer during vbufprintf, a corruption issue exists in the argument list. The state of an argument list after a call to vsnprintf is undefined. In the event of a buffer growth being needed, vbufprintf gets called again. However, this call is made without initing the argument list again. (va_start() must be called again). In the issue linked a test case can be seen to reliably reproduce (on 32bit linux).

andre-d/snudown@a5269f9 is a patch to correct the issue for snudown. The patch should apply for sundown also.

The patch simply wraps the calls to _buf_vsnprintf with va_start and va_end calls. Because va_start must be called within the function with the originating arguments, I have moved vbufprintf into bufprintf and removed vbufprintf.

TOC is not rendered correctly

When I have the following text:

Testing the **table of contents** functionality.


## A Level Two Header

Lorem ipsum dolor sit amet, consectetur adipiscing elit.


### A Leven Three Header

Quisque ultricies purus ut diam scelerisque nec scelerisque arcu fringilla.


#### A Leven Four Header

Praesent consequat erat vel massa lobortis facilisis.


### Another Level Three Header

Proin ut lorem a mi hendrerit ultricies.


## A Second Level Two Header

Vestibulum sollicitudin, arcu in adipiscing suscipit, dolor sem hendrerit mauris,
convallis aliquam velit felis sed mauris.


#### Another Level Four Header

Suspendisse potenti. Proin semper pellentesque porta.


### A Third Level Three Header

Suspendisse potenti. Maecenas et lacus quis justo vulputate ornare.

The following TOC is generated:

<li>
    <ul>
        <li><a href="#toc_0">A Level Two Header</a></li>
        <li>
            <ul>
                <li><a href="#toc_1">A Leven Three Header</a></li>
                <li>
                    <ul>
                        <li><a href="#toc_2">A Leven Four Header</a></li>
                    </ul>
                </li>
                <li><a href="#toc_3">Another Level Three Header</a></li>
            </ul>
        </li>
        <li><a href="#toc_4">A Second Level Two Header</a></li>
        <li>
            <ul>
                <li><a href="#toc_5">Another Level Four Header</a></li>
            </ul>
        </li>
        <li><a href="#toc_6">A Third Level Three Header</a></li>
    </ul>
</li>
</ul></li>
</ul>

When I add level 1 header at the top of the text this is generated:

<ul>
    <li><a href="#toc_0">A Level One Header</a></li>
    <li>
        <ul>
            <li><a href="#toc_1">A Level Two Header</a></li>
            <li>
                <ul>
                    <li><a href="#toc_2">A Leven Three Header</a></li>
                    <li>
                        <ul>
                            <li><a href="#toc_3">A Leven Four Header</a></li>
                        </ul>
                    </li>
                    <li><a href="#toc_4">Another Level Three Header</a></li>
                </ul>
            </li>
            <li><a href="#toc_5">A Second Level Two Header</a></li>
            <li>
                <ul>
                    <li><a href="#toc_6">Another Level Four Header</a></li>
                </ul>
            </li>
            <li><a href="#toc_7">A Third Level Three Header</a></li>
        </ul>
    </li>
</ul></li>
</ul>

There seems to be problems when a level 1 header is missing. And the </li></ul> should not be there.

And the anchors to the headers are like: <a name="toc_0"></a><h2>A Level Two Header</h2>. Isn't better to use something like this: <h2 id="toc_0">A Level Two Header</h2>?

Issue with emphasized links.

_a [foo][1]_ [bar][1]
_a [foo](http://www.google.com/)_ [bar](http://www.google.com/)


[1]: http://www.google.com/

Is rendered as:

_a <a href="http://www.google.com/">foo</a>_ <a href="http://www.google.com">bar</a>

instead of:

<em>a <a href="http://www.google.com/">foo</a></em> <a href="http://www.google.com">bar</a>

Oddly enough, the following work:

 _baz [foo](http://www.google.com/)_ [bar][1]
 _[foo][1] baz_ [bar][1]
 _[foo](http://www.google.com/) baz_ [bar](http://www.google.com/)


[1]: http://www.google.com/

The bug seems to be tied to the emphasized text containing a link preceded by text with both being separated by a space while being followed by another link.

<p> tags are added around <figure> HTML block-level tag.

For example:

<figure>
  <img src="/new.jgp" />
</figure>

is transformed into

<p>
  <figure>
    <img src="/new.jgp" />
  </figure>
</p>

The <p> tags should not be added.
It works for <div> tags but not for <figure> tag.

xmpp: URL

Hi,

I'm using Redcarpet 1.17.2 with :autolink and I have some problems with xmpp: URL. Ideally, xmpp: links could be automatically detected by the autolink feature, but for the moment, I'd like to enter these URL as simple links. It doesn't work because they are transformed to mailto: URL by the autolink:

Redcarpet.new("xmpp:[email protected]", :autolink).to_html
=> "<p>xmpp:<a href=\"mailto:[email protected]\">[email protected]</a></p>\n"
Redcarpet.new("[xmpp:[email protected]](xmpp:[email protected])", :autolink).to_html
=> "<p><a href=\"xmpp:[email protected]\">xmpp:<a href=\"mailto:[email protected]\">[email protected]</a></a></p>\n"

Do you think, you can do something for this case?

2x speedup when autolink extension is used

I didn't come up with this change, but it's been done by another guy who ported upskirt to Go. In Go it gives 2x speedup and since it's literal translation of C code, it should give the same speedup in C and be trivial to port.

The basic idea is that instead of triggering char_autolink() test on characters that are frequent in regular text ('h', 'f', 'm'), it's only triggered on much less frequent ':' and char_autolink() logic is modified accordingly.

Here's how it looks in my Go port: https://github.com/kjk/go-markup/commit/b760fe6f79d95e67cec08ecf1f2c840e86b4fce2
based on the original change: russross/blackfriday@679e168

Autolink issues

Hi,

it seems like that upskit doesn't detect some links like http://www-cs-faculty.stanford.edu/ (minus or accent in the subdomain?). Both Rinku and Redcarpet fails to autolink it, so I think the issue come from upskirt.

Header closing tag level wrong after specific autolink.

The following input

http://aaaaaaaaaaaaaaaa.aaa/aaaaaaa#aaaaa

=

Generates the following output

<p><a href="http://aaaaaaaaaaaaaaaa.aaa/aaaaaaa#aaaaa">http:&#47;&#47;aaaaaaaaaaaaaaaa.aaa&#47;aaaaaaa#aaaaa</a></p>

<h1></h118>

when MKDEXT_AUTOLINK is enabled.

I haven't dug in much here yet, but it is important to note that the number and position of the 'a's in the input is significant, but not what letter they actually are. Some slight variations of input text resulted in </h120>.

The name is infantile and offensive

If you lack the imagination to anticipate that the name will be viewed as infantile and offensive, then I assume you lack the imagination to write code that anticipates problems caused by pathological input.

'char' portability issues

Heya,
Sundown uses 'char' to store its character data in buffer.h. Normally, that would be an unsigned char, but on Solaris it's signed. One example where this causes problems is when passing the data to isspace, which expects unsigned char data but promotes its argument to an int.
I discovered this when using redcarpet on Solaris, where it would mangle certain UTF-8 characters. I've patched this internally and it appears to be fine, but wasn't confident enough about its side effects in other environments to push the patch without further discussion...
Any thoughts?

Issue with nested strong/em.

__foo[___bar___]__
**foo[***bar***]**

Is being rendered as:

<strong>foo[</strong><em>bar__</em>]__

instead of:

<strong>foo[</strong><em>bar</em><strong>]</strong>

Note, mixing asterisks and underscores works as intended though.

__foo[__*bar*__]__
**foo[**_bar_**]**

Segmentation fault

When running the example code (examples/sundown.c) a segmentation fault appears.

I'm not sure what kind of info to provide. Valgrind output, strace output, something else?

document how to test sundown against Markdown test suites

The sundown readme states:

• Fully standards compliant

Sundown` passes out of the box the official Markdown v1.0.0 and v1.0.3
test suites, and has been extensively tested with additional corner cases
to make sure its output is as sane as possible at all times.

It would be useful to document where the official Markdown v1.0.0 and v1.0.3 test suites are hosted and how to test sundown using them.

Enforcing `const` on the renderer API

Believe you missed one in markdown.c:

int (*render_method)(struct buf *ob, struct buf *text, void *opaque);

should be:

int (*render_method)(struct buf *ob, const struct buf *text, void *opaque);

HTML not escaped in some cases

I've been working on a Python extension for Upskirt and I noticed that in some cases HTML is not escaped in some cases.

Tables?

Does ~this~ work?

So? http://example.com/ \"It\" <b>Test</b> works?

| Function name | Description                    |
| ------------- | ------------------------------ |
| `help()`      | Display the help window.       |
| `destroy()`   | **Destroy your computer!**     |

<div>Stripped **or** \"not\"?</div> Test

Test test test

<div>Test</div>

Results in:

<p>Tables?</p>

<p>Does ~this~ work?</p>

<p>So? <a href="http://example.com/">http://example.com/</a> &quot;It&quot; &lt;b&gt;Test&lt;/b&gt; works?</p>

<p>| Function name | Description                    |
| ------------- | ------------------------------ |
| <code>help()</code>      | Display the help window.       |
| <code>destroy()</code>   | <strong>Destroy your computer!</strong>     |</p>

<div>Stripped **or** \"not\"?</div> Test

Test test test

<div>Test</div>

While this:

Tables?

Does ~this~ work?

So? http://example.com/ \"It\" <b>Test</b> works?

| Function name | Description                    |
| ------------- | ------------------------------ |
| `help()`      | Display the help window.       |
| `destroy()`   | **Destroy your computer!**     |

<div>Stripped **or** \"not\"?</div> Test

Test test test

<p>Test</p>

Results in:

<p>Tables?</p>

<p>Does ~this~ work?</p>

<p>So? <a href="http://example.com/">http://example.com/</a> &quot;It&quot; &lt;b&gt;Test&lt;/b&gt; works?</p>

<p>| Function name | Description                    |
| ------------- | ------------------------------ |
| <code>help()</code>      | Display the help window.       |
| <code>destroy()</code>   | <strong>Destroy your computer!</strong>     |</p>

<p>&lt;div&gt;Stripped <strong>or</strong> &quot;not&quot;?&lt;/div&gt; Test</p>

<p>Test test test</p>

<p>Test</p>

The difference in the two source texts is the the last HTML tag. I parsed both texts with the XHTML_SKIP_HTML render flag and I tested it with my own extension, Redcarpet and the example that is included with Upskirt.

:smart option modifies characters in code block.

When using redcarpet with :smart option, it modifies the characters in code block. For example straight quotes ( " and ' ) are converted into curly quotes.

injected h1 html

I hope this is the right place to report markdown rendering issues on here.

The issue can be seen here https://github.com/paulirish/html5-boilerplate/issues/378#issuecomment-894943
Unless I have done something terribly wrong, I'd say this is a bug. It worked before the change!

It is supposed to show a block of html code (in a code block) followed by some text and two hyperlinks.
It does, however, show some weird cropped mix-up of the code, followed by a real proper styled h1-tag with text in it, which was actually part code to appear in said code block (Non-blocking downloads in IE8).

Thoughts?

Add upskirt_version function and UPSKIRT_VERSION constant

It would be very helpful to other bindings, if a upskirt_version function (useful for FFI bindings) and an UPSKIRT_VERSION constant (useful for C extensions) were added.

Error when compiling

I got an error when compiling Sundown with the suplied makefile:

yui% make
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/markdown.o src/markdown.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/array.o src/array.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/buffer.o src/buffer.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/autolink.o src/autolink.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o html/html.o html/html.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o html/html_smartypants.o html/html_smartypants.c
gcc -g -O3 -Wall -Werror  -shared -Wl src/markdown.o src/array.o src/buffer.o src/autolink.o html/html.o html/html_smartypants.o -o libsundown.so.1
/usr/bin/ld: src/markdown.o: relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
src/markdown.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [libsundown.so.1] Error 1

After adding -fPIC to CFLAGS it works again.

Don't know what caused it. I use a 64-bit Linux.

- (single dash) is transformed into en dash

Dashes (“--” and “---”) supposedly are transformed into en- and em-dash entities. However Redcarpet with :smart option transforms - (single dash) into en dash and -- (double dash) into em dash. Therefore there is no way to inset normal dash (-).

Sundown without extensions

The latest sample sundown executable does not build with extensions. Is this intentional?

markdown = sd_markdown_new(0, 16, &callbacks, &options);

markdown analysis instead of parsing

Hey,

This is not bug with Sundown per se, more of a design issue... Wanted to get some thoughts on this.

I'm working on a project that requires not parsing Markdown to HTML, but rather rendering Markdown to the Cocoa text system via character ranges. I'm trying to use Sundown to extract information about the Markdown data.

Since Sundown is designed around the idea of parsing the input text into a different output text, this clashes with my use case. The ideal library would call the renderer to describe the source, and it would be the renderer's (trivial) job to assemble output, when that is the intended use case.

I had a couple questions:

• Would it be possible to adapt Sundown to a more abstracted model in which a renderer could render to more than just text output?
• Is there interest from anybody but me to look into this?
• In the short term, is there any other library which is designed like that?

Cheers,
-Kenneth

PS: If you're interested, my use case is rewriting the core of getmacchiato.com, to use a properly built and solid markdown library instead of the incredibly inefficient regular expression mess I'm using at the moment.

Please add render/html.o to libupskirt

I'm working on ffi-upskirt, which uses FFI to access the exported symbols in libupskirt.so. I noticed that render/html.o is not included in the libupskirt.so library. I would like to access the upshtml_* functions.

Degree of Markup.pl compatibility

This maybe is a question rather than a bug report.

Since the tests are not part of the sources, I've modified upscript.c in my repo (https://github.com/kjk/upskirt/blob/master/examples/upskirt.c) to run the test against 1.0.3 markdown test files from https://github.com/jgm/peg-markdown/tree/master/MarkdownTest_1.0.3/Tests

The test reads the .text file, renders it to html with ups_markdown() and compares to .html reference result (after trimming whitespace at the beginning of the line and collapsing empty lines).

10 tests fail.

Am I using the wrong test files? (those are the only ones I managed to track down).

The tests do look a bit fishy, e.g. in "Amps and angle encoding.text" < in "4 < 5." is supposed to be converted to &lt; while > in "6 > 5." stays as > (and the code, apparently correctly, converts it to &gt;).

So:

• is the comment in readme.markdown that upskirt is compatible with 1.0.1 and 1.0.3 Markdown still true?
• if yes, where can I get the test files it's being tested against?

A question about null in mkd_renderer

If I supply null in the mkd_renderer for a function, will it just mirror the input or omit it completely?

autolink_url fails and autolink_www takes over when text runs into link

This text has a lack of spaceshttp://www.google.com

renders as

<p>This text has a lack of spaceshttp://<a href="http://www.google.com">www.google.com</a></p>

rather than

<p>This text has a lack of spaces<a href="http://www.google.com">http://www.google.com</a></p>

Markdown table extension does not seem to work

I have the following source:

## MKDEXT_NO_INTRA_EMPHASIS

Lorem ipsum _d _olo_ r_ sit amet, __consectetur__ adipiscing _elit_.


## MKDEXT_TABLES

| First Header  | Second Header |
| ------------- | ------------- |
| Content Cell  | Content Cell  |
| Content Cell  | Content Cell  |



## MKDEXT_FENCED_CODE

This is a paragraph introducing:

~~~~~~~~~~~~~~~~~~~~~
a one-line code block
~~~~~~~~~~~~~~~~~~~~~


## MKDEXT_AUTOLINK

Here is an URL: http://github.com/


## MKDEXT_STRIKETHROUGH


Nullam tincidunt dui ~vitae~ nulla ~~tristique~~ ultricies.


## MKDEXT_LAX_HTML_BLOCKS

<div>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</div>


<span>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</span>

<p>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</p>

<strong>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</strong>

And this is the output:

<h2>MKDEXT_NO_INTRA_EMPHASIS</h2>

<p>Lorem ipsum <em>d _olo</em> r_ sit amet, <strong>consectetur</strong> adipiscing <em>elit</em>.</p>

<h2>MKDEXT_TABLES</h2>

<p>| First Header  | Second Header |
| ------------- | ------------- |
| Content Cell  | Content Cell  |
| Content Cell  | Content Cell  |</p>

<h2>MKDEXT_FENCED_CODE</h2>

<p>This is a paragraph introducing:</p>

<pre><code>a one-line code block
</code></pre>

<h2>MKDEXT_AUTOLINK</h2>

<p>Here is an URL: <a href="http://github.com/">http://github.com/</a></p>

<h2>MKDEXT_STRIKETHROUGH</h2>

<p>Nullam tincidunt dui ~vitae~ nulla <del>tristique</del> ultricies.</p>

<h2>MKDEXT_LAX_HTML_BLOCKS</h2>

<div>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</div>

<p><span>Donec quis lacus arcu, nec venenatis dolor. <strong>Suspendisse</strong> nibh eros, <em>pretium</em> in vestibulum non, ultrices sed erat.</span></p>

<p>Donec quis lacus arcu, nec venenatis dolor. **Suspendisse** nibh eros, _pretium_ in vestibulum non, ultrices sed erat.</p>

<p><strong>Donec quis lacus arcu, nec venenatis dolor. <strong>Suspendisse</strong> nibh eros, <em>pretium</em> in vestibulum non, ultrices sed erat.</strong></p>

The table is not converted to HTML. I used the Upskirt example with all markdown extensions enabled. The table syntax is from the PHP Markdown Extra page.

I'm not sure what MKDEXT_NO_INTRA_EMPHASIS and MKDEXT_LAX_HTML_BLOCKS are supposed to do.

inconsistent link rendering in README vs. file

Hello,

I find that markdown links that have newlines in the URL are not rendered properly in the README on a project's root (search for "my personal configuration file"). In contrast, they are rendered correctly when the README file is viewed by itself (search for "my personal configuration file").

Here is the markdown source for the link I'm talking about:

* Edit `~/.wmii/config.yaml` to your liking. For a real-life example,
  see [my personal configuration file](
  http://github.com/sunaku/wmiirc/blob/personal/config.yaml).

Thanks for your consideration.

No longer builds. Missing stack.h?

I just pulled b9670c1 and it doesn't build for me on OS X Lion:

$ make
gcc -c -g -O3 -fPIC -Wall -Werror -Isrc -Ihtml    -c -o src/markdown.o src/markdown.c
src/markdown.c:21:19: error: stack.h: No such file or directory
src/markdown.c:111: error: array type has incomplete element type
src/markdown.c: In function ‘rndr_newbuf’:
src/markdown.c:127: error: dereferencing pointer to incomplete type
src/markdown.c:127: error: dereferencing pointer to incomplete type
src/markdown.c:128: error: dereferencing pointer to incomplete type
src/markdown.c:128: error: dereferencing pointer to incomplete type
src/markdown.c:129: error: dereferencing pointer to incomplete type
src/markdown.c:129: error: dereferencing pointer to incomplete type
cc1: warnings being treated as errors
src/markdown.c:133: warning: implicit declaration of function ‘stack_push’
src/markdown.c: In function ‘add_link_ref’:
src/markdown.c:183: warning: implicit declaration of function ‘calloc’
src/markdown.c:183: warning: incompatible implicit declaration of built-in function ‘calloc’
src/markdown.c: In function ‘free_link_refs’:
src/markdown.c:228: warning: implicit declaration of function ‘free’
src/markdown.c: In function ‘parse_table_header’:
src/markdown.c:1910: warning: incompatible implicit declaration of built-in function ‘calloc’
src/markdown.c: In function ‘sd_markdown_new’:
src/markdown.c:2227: warning: implicit declaration of function ‘malloc’
src/markdown.c:2227: warning: incompatible implicit declaration of built-in function ‘malloc’
src/markdown.c:2233: warning: implicit declaration of function ‘stack_init’
src/markdown.c: In function ‘sd_markdown_free’:
src/markdown.c:2354: warning: implicit declaration of function ‘stack_free’

listitem inline rendering typo

oops, my bad

syntax hilighting destroys my unicode characters!

I was exited about syntax highlighting in readme files so I edited the Readme of one my gems here https://github.com/aziz/farsifu/blob/master/README.md

The code has some Persian text inside it which looks fine when editing, but the syntax highlighted version shows wierd characters!

highlighted_image: http://cl.ly/2i2o062m0v2l1m0C2c2x

correct_form: http://cl.ly/1v2U3b2G0G073z1f3A1b

Misparsed blockquote in list

The blockquote in the list below is not parsed: the ">" is rendered when it should not be. @gruber's Markdown.pl does render it as a blockquote.

Tested at a40cdb9.

1.  Hello
1.  World
> A blockquote in a list item.

1.  There is a sentence.

> A standalone blockquote.

Some further text.

Misparsing when section header has a prefix numeral

I just got a report from a user who has the following Markdown

1. First Section Title
----------------------------------------------------------------

* First list item.
* Second list item.
* Third list item.

2. Second Section Title
-----------------------------------------------------

A sentence in the second section.

This renders properly in TextMate's Markdown preview but with upskirt the numbers in the section titles throw it off. If I remove those numbers it renders how I would expect.

can't compile with GCC 4.6

I just cloned the repository and tried to build it and I got a big error from glibc :

$ make
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/markdown.o src/markdown.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/array.o src/array.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/buffer.o src/buffer.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o src/autolink.o src/autolink.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o html/html.o html/html.c
gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml    -c -o html/html_smartypants.o html/html_smartypants.c
gcc -g -O3 -Wall -Werror  -shared -Wl src/markdown.o src/array.o src/buffer.o src/autolink.o html/html.o html/html_smartypants.o -o libsundown.so.1
*** glibc detected *** /usr/bin/ld: munmap_chunk(): invalid pointer: 0x0976c6a8 ***
======= Backtrace: =========
/lib/i686/cmov/libc.so.6(+0x6aaa1)[0x401b8aa1]
/lib/i686/cmov/libc.so.6(+0x6bd1e)[0x401b9d1e]
/usr/lib/libbfd-2.21.51-system.20110523.so(objalloc_free+0x2c)[0x400fee0c]
======= Memory map: ========
08048000-080f4000 r-xp 00000000 08:02 623114     /usr/bin/ld.bfd
080f4000-080f5000 rw-p 000ac000 08:02 623114     /usr/bin/ld.bfd
080f5000-080f6000 rw-p 00000000 00:00 0 
09759000-099a5000 rw-p 00000000 00:00 0          [heap]
40000000-4001b000 r-xp 00000000 08:02 799364     /lib/ld-2.13.so
4001b000-4001c000 r--p 0001b000 08:02 799364     /lib/ld-2.13.so
4001c000-4001d000 rw-p 0001c000 08:02 799364     /lib/ld-2.13.so
4001d000-4001e000 r-xp 00000000 00:00 0          [vdso]
4001e000-40024000 rw-p 00000000 00:00 0 
40025000-4002b000 rw-p 00000000 00:00 0 
4003d000-40125000 r-xp 00000000 08:02 620199     /usr/lib/libbfd-2.21.51-system.20110523.so
40125000-40131000 rw-p 000e8000 08:02 620199     /usr/lib/libbfd-2.21.51-system.20110523.so
40131000-40135000 rw-p 00000000 00:00 0 
40135000-40148000 r-xp 00000000 08:02 623183     /usr/lib/libz.so.1.2.3.4
40148000-40149000 rw-p 00013000 08:02 623183     /usr/lib/libz.so.1.2.3.4
40149000-4014a000 rw-p 00000000 00:00 0 
4014a000-4014c000 r-xp 00000000 08:02 945046     /lib/i686/cmov/libdl-2.13.so
4014c000-4014d000 r--p 00001000 08:02 945046     /lib/i686/cmov/libdl-2.13.so
4014d000-4014e000 rw-p 00002000 08:02 945046     /lib/i686/cmov/libdl-2.13.so
4014e000-402a1000 r-xp 00000000 08:02 947994     /lib/i686/cmov/libc-2.13.so
402a1000-402a2000 ---p 00153000 08:02 947994     /lib/i686/cmov/libc-2.13.so
402a2000-402a4000 r--p 00153000 08:02 947994     /lib/i686/cmov/libc-2.13.so
402a4000-402a5000 rw-p 00155000 08:02 947994     /lib/i686/cmov/libc-2.13.so
402a5000-402a9000 rw-p 00000000 00:00 0 
402c6000-402e2000 r-xp 00000000 08:02 798166     /lib/libgcc_s.so.1
402e2000-402e3000 rw-p 0001b000 08:02 798166     /lib/libgcc_s.so.1
bfe2a000-bfe4b000 rw-p 00000000 00:00 0          [stack]
collect2: ld terminated with signal 6 [Abandon]
make: *** [libsundown.so.1] Erreur 1

Here is the version of gcc :

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/i486-linux-gnu/4.6.1/lto-wrapper
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 4.6.0-10' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++,go --prefix=/usr --program-suffix=-4.6 --enable-shared --enable-multiarch --with-multiarch-defaults=i386-linux-gnu --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-objc-gc --enable-targets=all --with-arch-32=i586 --with-tune=generic --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.6.1 20110526 (prerelease) (Debian 4.6.0-10) 

I tried with gcc 4.5 and it compiles successfully (make CC=gcc-4.5).

It seems like newer gcc are able to catch some errors.

Invalid memory read in find_emph_char

The simplest repro for this is: "*ca"

find_emph_char() will be called with data pointing to "ca" and c == '*'.

while (i < size) {
    while (i < size && data[i] != c
    && data[i] != '`' && data[i] != '[')
        i += 1;
    if (data[i] == c) return i;

Since there is no '*' in "ca", at this point i == size and data[i] == data[size] i.e. one byte past data. The fix is to add:

if (i >= size) return 0;

It rarely will be fatal in C due to padding inside struct buf. I found it while testing my Go port (https://github.com/kjk/go-markup/commit/122da5f0ca90120bc2840eeeffdd34413286c718) where I use slices for data and Go is unforgiving about out-of-bounds slice access.

Footnote Extension

Would it be possible to add the footnotes extension from PHP-Markdown to Sundown? The details of the syntax and usage are defined here. I'm happy to add this to the redcarpet Ruby library and submit a pull request there but I have little to no C knowledge or experience to add it here myself.

Thanks

Nested parenthesis in href breaks inline hyperlinks

A tricky edge case is parenthesis inside inline urls. Sundown and github flavored markdown both seem to stop on the first occurrence of )

• [inline hyperlink with a paren](http://www.example.com/test#nav(1))
  inline hyperlink with a paren
• [reference hyperlink with paren][1]
  reference hyperlink with paren

Perhaps this can be fixed by counting nested parenthesis?

No Tags or ChangeLog

Hello again, so I need to know when upskirt releases it's next version, so that I can release ffi-upskirt (which depends on a few functions in edge). There should be git tags or a ChangeLog.

Not parsing emphasis when followed by some characters

I've thrown a bunch of test cases in https://gist.github.com/1166716 to show how it fails. From the markdown spec on <em> it states you can use the emphasis delimiters in the middle of a word, and I'm failing to see how 0, ™, °, » trailing it are not covered by that part of the spec. If I wanted it to literally output _Just_™ then I'd write \_Just\_™ to make sure it wasn't interpreted as emphasis.

It's parsed correctly (in my eyes) by markdown.pl - the markdown implementation by John Gruber (and used by TextMate's markdown bundle to preview documents.)

Compiled file, src/html_blocks.h, is in repository

It should probably be removed and ignored.

examples/upskirt.c fails to build with gcc 4.5.2

gcc -c -g -O3 -Wall -Werror -Isrc -Ihtml -fPIC -c -o examples/upskirt.o examples/upskirt.c
cc1: warnings being treated as errors
examples/upskirt.c: In function ‘main’:
examples/upskirt.c:71:8: error: ignoring return value of ‘fwrite’, declared with attribute warn_unused_result

simple to work around, but should likely do some real checking

Map Table Extension

One thing that Markdown lacks is an easy way to create definition lists. I would really like to see an extension for rdoc's style of definition lists. It uses double-colons to signify them. Here's is an example:

Coffee:: black hot drink
Milk:: white cold drink

Which would render to:

[NOTE this has changed. See below.]

<dl>
  <dt>Coffee</dt>
    <dd>black hot drink</dd>
  <dt>Milk</dt>
    <dd>white cold drink</dd>
</dl>

TOC does not ignore headers in fenced code blocks

When generating a table of contents all the headers in fenced code blocks are listed too.

See "Header one" and "Header two" in the TOC: http://misaka.61924.nl/. And you can see both headers in the 4th code block under Usage. This is the source file: https://github.com/FSX/misaka/blob/master/docs/documentation.md

strings.h should be string.h in render/xhtml.c:21

It should be string.h but strings.h in render/xhtml.c:21

cc1: warnings being treated as errors
render/xhtml.c: In function ‘ups_toc_renderer’:
render/xhtml.c:711: error: implicit declaration of function ‘memcpy’
render/xhtml.c:711: error: incompatible implicit declaration of built-in function ‘memcpy’

Definition Lists

One of the main things I use Markdown for is writing documentation. And one thing I do over and over again is document parameters to functions and methods. Definition lists are perfect for this. I end up using MultiMarkdown to get them. Alas, I've been unable to find any support for definition lists in any of GitHub's Markdown flavors.

Would really appreciate definition list support to your upskirt fork. The PHP Markdown Extra definition list syntax is okay, though it's not great for reading as plain text. (I wrote a proposal for a slight variation here. But really, any definition list support would be great.

Thanks!

Feature Request: Wikilinks

Hello,

I realize this can be done with a preprocess override, but I was wondering if you might consider a including a wiki link extension that supports folders in C. Something like [[Example/Link|This is a wikilink]]. It would point to /wiki/link with link text of "This is a wikilink".

Albert

Suggestion about links in titles and TOC

Hi,

when a title contains a link, the table of contents show the markdown syntax of this link: foo [bar](http://www.baz.com). I suggest that only the name of the link should be used for the TOC: foo bar.

Linebreak with other characters instead of 2 spaces

Currently you can add two spaces at the end of a line and a linebreak (br) will be inserted.

I think it would be more user friendly to add an extension that replaces the two spaces with two other characters. Two spaces are not really visible, unless you make whitespace visible in your code editor. And when you have a "trim whitespace" function enabled the spaces disappear.

problems compiling the library

executing make in the latest checkout:

gcc -c -g -O3 -Wall -Werror -Isrc -Irender   -c -o src/markdown.o src/markdown.c
gcc -c -g -O3 -Wall -Werror -Isrc -Irender   -c -o src/array.o src/array.c
gcc -c -g -O3 -Wall -Werror -Isrc -Irender   -c -o src/buffer.o src/buffer.c
gcc -g -O3 -Wall -Werror -shared -Wl src/markdown.o src/array.o src/buffer.o -o libupskirt.so.1
/usr/bin/ld: src/markdown.o: relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
src/markdown.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [libupskirt.so.1] Error 1

adding -fPIC to the Makefile produces:

gcc -c -g -O3 -Wall -Werror -Isrc -Irender -fPIC   -c -o src/markdown.o src/markdown.c
gcc -c -g -O3 -Wall -Werror -Isrc -Irender -fPIC   -c -o src/array.o src/array.c
gcc -c -g -O3 -Wall -Werror -Isrc -Irender -fPIC   -c -o src/buffer.o src/buffer.c
gcc -g -O3 -Wall -Werror -shared -Wl src/markdown.o src/array.o src/buffer.o -o libupskirt.so.1
ln -f -s libupskirt.so.1 libupskirt.so
gcc -c -g -O3 -Wall -Werror -Isrc -Irender -fPIC   -c -o examples/upskirt.o examples/upskirt.c
gcc -c -g -O3 -Wall -Werror -Isrc -Irender -fPIC   -c -o render/xhtml.o render/xhtml.c
cc1: warnings being treated as errors
render/xhtml.c: In function 'ups_toc_renderer':
render/xhtml.c:711:2: error: implicit declaration of function 'memcpy'
render/xhtml.c:711:2: error: incompatible implicit declaration of built-in function 'memcpy'
render/xhtml.c: In function 'ups_xhtml_renderer':
render/xhtml.c:754:2: error: incompatible implicit declaration of built-in function 'memcpy'
make: *** [render/xhtml.o] Error 1

adding #include <string.h> to render/xhtml.c, running make again and it builds fine.

running gcc v4.5.2 on arch linux (64bit)