kristapsdz / lowdown Goto Github PK

View Code? Open in Web Editor NEW

286.0 12.0 34.0 2.2 MB

simple markdown translator

Home Page: https://kristaps.bsd.lv/lowdown

License: ISC License

Makefile 2.02% C 82.38% CSS 0.35% JavaScript 0.01% HTML 12.76% Roff 2.48%

markdown markdown-to-html markdown-to-pdf groff hoedown markdown-to-latex markdown-to-gemini

lowdown's People

Contributors

Stargazers

Watchers

lowdown's Issues

Other sections for -s -Tman?

OK, I know this is probably going to sound like heresy ;) and would understand if you were to decline this, but I was wondering how open you would be at allowing other sections for the manpage standalone output.

I'm cognizant of the footnote on the homepage ("You may be tempted to write manpages in Markdown, but please don't: use mdoc(7), instead"), but there are projects that do have their documentation in Markdown out there today and then use tools like pandoc and ronn to convert their Markdown to e.g. section 1 or 3 manpages.

It would be neat if lowdown could be an alternative to them :)

Gemini output isn't completely documented in lowdown.1 and prints metadata at the top of the file.

-Tgemini isn't mentioned in Output modes of lowdown(1).

Furthermore, using markdown files with metadata information, the metadata gets printed to the resulting gemtext, with no option to disable this. I'm ok with it being either on or off by default, but it would be nice to be able to strip out the metadata.

term output doesn't indent lists enough

For example, given the following input:

bla

* foo

  bar

bla

lowdown -Tterm produces the following output:

    bla

  - foo

    bar

    bla

So it's not clear that bar is a paragraph in a list item rather than at top-level. I think it would be better to indent lists a bit more, e.g.

    bla

    - foo

      bar

    bla

Alternatively, maybe top-level paragraphs shouldn't be indented at all.

P.S. thank you for lowdown! We're using it in the Nix project for rendering Markdown docs in the repl and for command line help.

-Tms : quoted strings begin and end with 201C

U+201C is the left curly quote, and it properly begins a "quoted string" in lowdown's output. However, with -Tms, the end of the quote is also U+201C instead of U+201D. I don't see where this gets rendered in either nroff.c or smartypants.c.

A minimal example:

% echo 'He said, "I got no freeking clue," and walked away.' | ./lowdown -Tms
.LP
He said, \[u201C]I got no freeking clue,\[u201C] and walked away.
%

Lowdown version from GitHub, running on MacOS 10.15.6.

Problems with inline HTML being wrapped in tags

I noticed while attempting to convert a Markdown document to HTML that inline HTML is sometimes wrapped in  tags unnecessarily. For instance, take the input:

# Testing

This is a paragraph. 

<details><summary>Click Me</summary>
Pop!
</details>

The output, after running lowdown -s -Thtml --html-no-skiphtml --html-no-escapehtml test.md:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<title>Untitled article</title>
</head>
<body>

<h1 id="Testing">Testing</h1>

<p>This is a paragraph. </p>

<p><details><summary>Click Me</summary>
Pop!
</details></p>
</body>
</html>

The <details> tag is wrapped in a  block. While most browsers can figure this out and display it fine, according to https://html5.validator.nu/ and https://validator.w3.org/ this is not valid HTML because:

Error: No p element in scope but a p end tag seen.

From line 16, column 11; to line 16, column 14

</details></p>↩</bod

The issue just gets worse if I put tags on their own separate lines; each tag gets wrapped in .

distclean leaves *.old files behind

OK, this is really really minor but:

$ ./configure
$ ./configure
$ make
$ make distclean

This leaves behind config.h.old and config.log.old, generated by the second ./configure invocation.

Fix:

diff --git a/Makefile b/Makefile
index da8f656..e86f58d 100644
--- a/Makefile
+++ b/Makefile
@@ -252,7 +252,7 @@ clean:
        rm -f index.latex.aux index.latex.latex index.latex.log index.latex.out
 
 distclean: clean
-       rm -f Makefile.configure config.h config.log
+       rm -f Makefile.configure config.h config.log config.h.old config.log.old
 
 regress: lowdown
        tmp1=`mktemp` ; \

Your personal Mac Homebrew tap

I've created a tap for Mac Homebrew to install lowdown with brew install lowdown. Adding the formula to the official repo was rejected because this project "isn't popular enough".

My tap is over here: https://github.com/To1ne/homebrew-lowdown

But IMHO it makes more sense if you would host the tap under your account. So feel free to fork that repo and take ownership of it (I'll will delete my repo afterward).

Is it possible to build using Msys2 on Windows host?

lowdown is a cool utility, I wish to port it to Msys2 app repo.

Link conversion for gemini should not have "references" label

Hi,

when I convert from markdown to gemini, link conversion uses references. Input markdown:

[description of the link](https://example.com/link/blub)

turns into

description of the link[Reference: link-1]

=> https://example.com/link/blub [Reference: link-1]

This could be simplified to

description of the link

=> https://example.com/link/blub

or if a reference is wanted to

description of the link [1]

=> https://example.com/link/blub [1]

Support for CSS "start" property

Hi!

I love lowdown and I use it every day. There only one thing that I don't like about it: it ignores the number used to begin an ordered list.

Now, I know (think) that the original Markdown also does this, but other implementations, such as GitHub's, respects it, by converting this:

1. First item

Some text.

2. Second item

to this:

<ol start="1"><li>First item</li></ol>
<p>Some text.</p>
<ol start="2"><li>Second item</li></ol>

It's strange that so few Markdown implementations support this very useful CSS property.

Perhaps lowdown could support it via an option? What do you think?

Email address output

In HTML output, email address have mailto: in front of them.
In roff output, they do not.

Hard wrap doesn't work right with -Tms

Hard wrap (—nroff-hardwrap) with -Tms puts the .LP at the end of the previous line. A minimal example:

% echo 'first paragraph that needs to consist of a lot of text to trigger the hardwrap and whatever else there might be.\n\nAnd another paragraph' | ./lowdown -Tms --nroff-hardwrap
.LP
first paragraph that needs to consist of a lot of text to trigger the hardwrap and whatever else there might be..LP
And another paragraph%                                                                                                                                                                         %

Ideally, hardwrap with -Tms would break lines at the end of each sentence, but I'll accept that it could be harder than it sounds.

Current GitHub source, running on MacOS 10.15.6

how can i uninstall or remove lowdown?

How to add images with ms?

I use ms to compile md files to ms. After I convert ms to ps and ps to pdf with command that I found somewhere and edited:

lowdown -d nointem -e super "$file" -Tms | groff -mpdfmark -Kutf8 -mspdf -kept | ps2pdf - output.pdf

But I can't add images, every time it's just nothing in places where image should be.

Links

How can i put a link to another website or put an image in lowdown ? I'm using article1.md to write retro blog.

Thank you.

css metadata field doesn't allow specifying multiple files as stated in man page

lowdown(1) states

lowdown/man/lowdown.1

Lines 406 to 407 in b4483d0

    
           Multiple CSS files (in order) may be separated by more than one space 
        
           (including newlines).

but the actual rendering code doesn't seem to implement such a splitting logic:

lowdown/html.c

Lines 1052 to 1053 in b4483d0

    
           else if (strcasecmp(m->key, "css") == 0) 
        
           	css = m->value;

lowdown/html.c

Lines 1088 to 1089 in b4483d0

    
           if (css != NULL && !rndr_meta_multi 
        
               (ob, css, "<link rel=\"stylesheet\" href=", " />"))

Maybe I'm misreading the documentaton, it probably should be clarified.

Tables longer than one page produce error

Hi,

I noticed that lowdown does not seem to support tables that are longer than one page:

$ cat <<-EOF | lowdown -sTms | pdfroff -itk -mspdf > table.pdf
app|quality
---|---
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
lowdown|rocks
EOF

error: page 2: table will not fit on one page; use .TS H/.TH with a supporting macro package

If I understand correctly this could be easily fixed if lowdown would enclose the table header in .TS H / .TH macros, according to man groff_ms:

.TS [H] and .TE
Denote a table to be processed by the tbl preprocessor.
The optional H argument instructs groff to create a
running header with the information up to the TH macro.
groff prints the header at the beginning of the table; if
the table runs onto another page, groff prints the header
on the next page as well.

https://www.man7.org/linux/man-pages/man7/groff_ms.7.html

Thank you for the great work!

columns in -Tterm aren't space aligned

given the input (using -Tterm)

| name    | git repository                        |
| ------- | ------------------------------------- |
| fzf     | https://github.com/junegunn/fzf       |
| lf      | https://github.com/gokcehan/lf        |
| lowdown | https://github.com/kristapsdz/lowdown |
| st      | https://git.suckless.org/st           |
| uni     | https://github.com/arp242/uni         |

produces output (control sequences are removed)

fzfhttps://github.com/junegunn/fzf
lfhttps://github.com/gokcehan/lf
lowdownhttps://github.com/kristapsdz/lowdown
sthttps://git.suckless.org/st
unihttps://github.com/arp242/uni

i expected the columns to be spaces aligned. like,

fzf     https://github.com/junegunn/fzf
lf      https://github.com/gokcehan/lf
lowdown https://github.com/kristapsdz/lowdown
st      https://git.suckless.org/st
uni     https://github.com/arp242/uni

using : to specify alignment doesn't effect the alignment in output.

exclamation point immediately before a footnote reference is removed

Consider this simple file, test.txt:

This is a test![^01]

[^01]: footnote text.

Note the exclamation point before the footnote reference. Now when I run lowdown test.txt I get this, in which that exclamation point is missing:

<p>This is a test<sup id="fnref1"><a href="#fn1" rel="footnote">1</a></sup></p>

<div class="footnotes">
<hr/>
<ol>

<li id="fn1">
<p>Footnote&#160;<a href="#fnref1" rev="footnote">&#8617;</a></p>
</li>

</ol>
</div>

In all output modes, the exclamation point immediately before a footnote reference is removed.

I noticed this on lowdown 0.8.3 and 0.8.4. on OpenBSD -current.

Gemini semantics

As requested, and this is me spouting off before having tried your app
so some of these might already be implemented…

Also, Gemini is plain text. It's possible to write a legal Gemini document that looks like this:

##### Hello

>> I wrote
> Then you wrote

and that will be interpreted as:

A level three header line that reads "# Hello"
and two quote lines, one of which starts with an >.

That's technically legal.

However, let's aim for a more strict and beautiful Gemini text.

Text lines

Please change

Line one
Line two

Line one Line two

Please change (note the spaces)

Line one  
Line two

Line one
Line two

Please keep

Line one

Line two

the same.

Link lines

For inline links, do not put footnotes, numbers, or other markers in the text itself.

Instead, please change:

Some text with [a link](url) in the middle of it.

Some text with a link in the middle of it.

=> url a link

The correct placement for all accumulated links that have been "extracted" in such a way, is which ever comes first out of:

Just before the next non-"extracted" link (i.e. a link from a list or stand-alone , see below)
Just before the next header
Just before the end of the document

Do not do "extract" links that already are alone on their line (i.e.
alone in a , or, in a very short ), or that are in a list.

For those, please change

- some [link](url) in a list
- [some other link in a list](url)

=> url some link in a list
=> url some other link in a list

for lists. Both <a> and <li><a></li> should be
turned into link lines without blank lines in between.

* also, this is
=> url completely fine
=> url even good, idiomatic
* in gemini

In general, gemini text needs way fewer blank lines than markdown needs. In gemini, each line can be parsed as its own thing without being affected by preceding or following lines.

However, please be consistent on whether or not header lines are preceded by a blank line.

Links before quotes

The following is implied by what I wrote re short  but just to be explicit:

Please change

[My friend](url) wrote:
> some stuff to me

=> url My friend wrote:
> some stuff to me

Pre-lines (triple-backticks)

Please add an alt text such as "Preformatted text" or "Code".

List lines

Remember that Gemini does not have - for lists, only *.

Also Gemini can not nest lists.

Not sure whether it's best to flatten trees or to split sublists out
to be lists of their own.

Headers

Gemini documents only have three levels of header.

The intent is to split up documents that have many levels of heading
into sub-documents of no more than three header-level each. We don't
accidentally want to create a formatting that people will use for bold
text that isn't semantically a header.

Therefore, please change

#### Level four, five or lower header

to a plain text line

Level four, five or lower header

Blockquote

Please change

>> this nested blockquote
> this normal blockquote

> this nested blockquote this normal blockquote

But, regardless, please change

> this first line of a multi-line blockquote
> this second line of a multi-line blockquote

> this first line of a multi-line blockquote this second line of a multi-line blockquote

and

> this first line of a multi-line blockquote but with two spaces after  
> this second line of a multi-line blockquote

> this first line of a multi-line blockquote but with two spaces after
> this second line of a multi-line blockquote

and keep

> this first line of a multi-line blockquote but with an empty line after
>
> this second line of a multi-line blockquote

the same.

Emphasis and similar

Please remove emphasis by turning

A line with an *asterisked word* in it

into

A line with an asterisked word in it

But don't change 4 * 4 = 16 to 4 4 = 16.

Thank you.

I'm subscribed to this thread in case you need clarifications. I'm on a CET timezone though.

Footnotes begin at 2 (not 1)

Hello,

There appears to be an infamous off-by-one error with the initial numbering of footnotes. This causes a mismatch between the superscript numbering within the body text and the <ol> footnote list.

Here is a minimal example:

$ cat example.md
Some text.[^1]
Some more text.[^2]

[^1]: A footnote.
[^2]: Another footnote.

$ lowdown < example.md
<p>Some text.<sup id="fnref2"><a href="#fn2" rel="footnote">2</a></sup>
Some more text.<sup id="fnref3"><a href="#fn3" rel="footnote">3</a></sup></p>

<div class="footnotes">
<hr/>
<ol>

<li id="fn2">
<p>A footnote.&#160;<a href="#fnref2" rev="footnote">&#8617;</a></p>
</li>

<li id="fn3">
<p>Another footnote.&#160;<a href="#fnref3" rev="footnote">&#8617;</a></p>
</li>

</ol>
</div>

Bew Install Lowdown error on Mac

Tried to do a Brew Install Lowdown on a Mac OS Sierra (v. 10.12.5)

This was the error message:

Last 15 lines from /Users/Huckleberry/Library/Logs/Homebrew/lowdown/02.make:
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o compat_progname.o compat_progname.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o compat_reallocarray.o compat_reallocarray.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o compat_strlcat.o compat_strlcat.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o compat_strlcpy.o compat_strlcpy.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o compat_strtonum.o compat_strtonum.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter -c -o main.o main.c
ar rs liblowdown.a autolink.o buffer.o document.o html.o html_escape.o html_smartypants.o library.o log.o nroff.o nroff_escape.o nroff_smartypants.o tree.o xmalloc.o compat_err.o compat_progname.o compat_reallocarray.o compat_strlcat.o compat_strlcpy.o compat_strtonum.o
ar: creating archive liblowdown.a
cc -o lowdown main.o liblowdown.a
Undefined symbols for architecture x86_64:
"_memrchr", referenced from:
_rndr in liblowdown.a(nroff.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [lowdown] Error 1

Leading whitespace not stripped from metadata when using -X flag

lowdown(5) says that metadata "Values have leading white-space stripped, i.e., space following the colon."

White-space is stripped when metadata is inserted into documents, e.g., when using the -s flag to generate an HTML file. However, when using the -x flag, white-space is not stripped. Consider this very simple document test.md:

Title: Test

Text here

lowdown -s test.md produces

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1" />
<meta name="date" content="2017-11-11" scheme="YYYY-MM-DD" />
<title>Test</title>
</head>
<body>

<p>Text here.</p>
</body>
</html>

while lowdown -X title test.md produces

 Test

i.e., the leading white-space after the colon is not stripped.

This is with lowdown 0.3.1 on OpenBSD 6.2-current.

Apologies if this behaviour is expected and intended.

Prefix is not rendered when body is empty

This:

> This is the first level of quoting.
>
> > This is nested blockquote.
>
> Back to the first level.

Results in this:

    | This is the first level of quoting.

    |     | This is nested blockquote.

    | Back to the first level.

But should in this:

    | This is the first level of quoting.
    |
    |     | This is nested blockquote.
    |
    | Back to the first level.

-Tmd: a built in markdown formatter

Could lowdown in theory output well-formatted markdown when given valid, but not-so-well-formatted
markdown as input?

Since lowdown generates an AST this (to my untrained eyes) seems to be a cool feature that could be added by writing a markdown renderer.

I know other tools exist, like markdownfmt and I believe pandoc can do this, but having this ability in lowdown would remove the need for those.

I would be curious about your thoughts regarding this.

Percent encoding of ~

Currently lowdown percent encodes ~ but it doesn't do for . _ -

All 4 of them are unreserved. Shouldn't ~ also not be percent encoded?

Unexpected literal ".PP" in markdown bullets when rendering man page

Lowdown 0.7.3 on Linux

Consider the following test case (file test.md):

# right
* source
[source](http://source)
* binary
[binary](http://binary)

# wrong
* source
[source](http://source)

* binary
[binary](http://binary)

Render man page:

lowdown -Tman -s test.md

Result:

.SH right
.sp 1.0v
.RS
.ti -\w'\(bu  'u
\(bu  source
\f[I]source\f[R]
.RE
.RS
.ti -\w'\(bu  'u
\(bu  binary
\f[I]binary\f[R]
.RE
.SH wrong
.sp 1.0v
.RS
.ti -\w'\(bu  'u
\(bu  .PP
source
\f[I]source\f[R]
.RE
.sp 1.0v
.RS
.ti -\w'\(bu  'u
\(bu  .PP
binary
\f[I]binary\f[R]
.RE

In section "wrong", notice .PP after the bullet \(bu. This seems wrong and results in the literal text ".PP" being printed in the manual page:

lowdown -Tman -s | man -l -

Result:


·  source source
·  binary binary

·  .PP source source

·  .PP binary binary

If I delete the last two line and render a manual page again ".PP" isn't added -- correctly.

# right
* source
[source](http://source)
* binary
[binary](http://binary)

# wrong
* source
[source](http://source)

Result:


·  source source
·  binary binary

·  source source

With lowdown version 0.4.6 I did not see the stray ".PP" after the bullets. I think that version 0.4.6 outputs .LP for paragraphs while the newer versions output .PP. Regardless, there were no unexpected ".PP" or ".LP" in the man page.

With -Thtml a similar issue surfaces, in the "wrong" section,  follows <li>, which seems wrong. When rendering the HTML in a browser, the effect isn't necessarily noticeable unlike in a man page, in which ".PP" becomes literal text.

<h1 id="right">right</h1>

<ul>
<li>source
<a href="http://source">source</a></li>
<li>binary
<a href="http://binary">binary</a></li>
</ul>

<h1 id="wrong">wrong</h1>

<ul>
<li><p>source
<a href="http://source">source</a></p></li>
<li><p>binary
<a href="http://binary">binary</a></p></li>
</ul>

For comparison, I checked how GitHub renders the same case to HTML, and -- to my surprise -- GitHub behaves like lowdown. For instance, see this page in your browser's developer tools, and compare the HTML code for the single bullet under 2: and the two bullets under 3:. The single bullet doesn't include a  tag while each of the two bullets does.

Lowdown and Sblg ignore `PREFIX` environment variable

Hello,

When running configure and make, Lowdown v0.8.3 and Sblg v0.5.11 both appear to ignore the PREFIX environment variable, and install to /usr/local regardless.

Tested in bash v5.1.4 and fish v3.2.0, running:

$ PREFIX=/home/$USER/.local ./configure
$ PREFIX=/home/$USER/.local make -j12 install

Here's the output of configure:

$ /opt/lowdown-0.8.3 $ PREFIX=/home/$USER/.local ./configure
config.log: writing...
configure.local: no (fully automatic configuration)
arc4random: no
b64_ntop: yes (with -lresolv)
capsicum: no
crypt: yes (with -lcrypt)
endian_h: yes
err: no
explicit_bzero: yes
fts: yes
getexecname: no
getprogname: no
INFTIM: no
lib_socket: yes
md5: no
memmem: yes
memrchr: yes
memset_s: no
mkfifoat: yes
mknodat: yes
osbyteorder_h: no
PATH_MAX: yes
pledge: no
program_invocation_short_name: yes
readpassphrase: no
reallocarray: yes
recallocarray: no
sandbox_init: no
seccomp-filter: yes
setresgid: yes
setresuid: yes
sha2: no
SOCK_NONBLOCK: yes
static: yes
strlcat: no
strlcpy: no
strndup: yes
strnlen: yes
strtonum: no
sys_byteorder_h: no
sys_endian_h: no
sys_mkdev_h: no
sys_sysmacros_h: yes
sys_queue: no
sys_tree: no
unveil: no
WAIT_ANY: yes
__progname: yes
config.h: written
Makefile.configure: written

and here's the output of make install: https://gist.github.com/catleeball/9d624c4b5aa45819cbb79a505a51d858

And here's uname -a:

Linux bip 5.10.19-1032.native #1 SMP Wed Mar 3 13:57:49 PST 2021 x86_64 GNU/Linux

Please let me know if you need any other info!

Horizontal line output in nroff/troff

Hello!

I have both a small problem with horizontal rules in Heirloom nroff/troff, and some ideas about it...

If we begin with this minimal markdown:

Fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo,
quis tempor ligula erat quis odio. Nunc porta vulputate tellus. Nunc
rutrum turpis sed pede. Sed bibendum.

***

Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada
massa, quis varius mi purus non odio. Pellentesque condimentum, magna ut
suscipit hendrerit, ipsum augue ornare nulla, non luctus diam neque sit
amet urna.

The following creates a horizontal rule between the paragraphs using GNU groff:

$ < test.md lowdown -Tms | groff -Tutf8 | less   # okay
$ < test.md lowdown -Tms | groff -ms > out.ps    # okay

However the same with Heirloom nroff/troff we get a horizontal line of 'n' characters:

I'm pretty sure this is an easy fix:

diff --git a/nroff.c b/nroff.c
index 92b9627..bb50a30 100644
--- a/nroff.c
+++ b/nroff.c
@@ -1002,10 +1002,10 @@ rndr_hrule(const struct nroff *st, struct bnodeq *obq)
 	 * The LP is to reset the margins.
 	 */
 
-	if (bqueue_block(obq, ".LP") == NULL)
+	if (bqueue_block(obq, ".RT") == NULL)
 		return 0;
 	if (!st->man && 
-	    bqueue_block(obq, "\\l\'\\n(.lu-\\n(\\n[.in]u\'") == NULL)
+	    bqueue_block(obq, "\\l\'\\n(.lu\'") == NULL)
 		return 0;
 	return 1;
 }

Here we're using the .RT macro to reset the paragraph, which I think removes any indentation, then just drawing a line without the need to subtract indentation.

Now to test:

$ < test.md lowdown -Tms | groff -Tutf8 | less         # okay
$ < test.md lowdown -Tms | groff -ms > out.ps          # okay
$ < test.md lowdown -Tms | nroff | less                # okay
$ < test.md lowdown -Tms | troff -ms | dpost > out.ps  # okay

But ideally a horizontal rule would have some space around it:

diff --git a/nroff.c b/nroff.c
index 92b9627..0827737 100644
--- a/nroff.c
+++ b/nroff.c
@@ -1002,10 +1002,10 @@ rndr_hrule(const struct nroff *st, struct bnodeq *obq)
 	 * The LP is to reset the margins.
 	 */
 
-	if (bqueue_block(obq, ".LP") == NULL)
+	if (bqueue_block(obq, ".RT") == NULL)
 		return 0;
 	if (!st->man && 
-	    bqueue_block(obq, "\\l\'\\n(.lu-\\n(\\n[.in]u\'") == NULL)
+	    bqueue_block(obq, ".sp 1\n\\l\'\\n(.lu\'\n.sp 1") == NULL)
 		return 0;
 	return 1;
 }

This adds a blank line before and after the horizontal rule (which I think looks better):

However! How often do we see horizontal rules like this in typeset documents? Usually we have a 3-line vertical space (i.e. four newlines) or a vertical space with "***" like this:

When I'm writing fiction in markdown, I use the *** for a traditional "scene break" so ideally I'd like to be able to typeset these.

But hardcoding this kind of literary style on everyone would not be very considerate...

While the ms macro file mostly has macros to correlate for markdown things like headings, it has nothing for horizontal rules. So instead, what if we allowed the user to define their own HR macro to insert as the "horizontal rule" in roff output?

In this case, lowdown could output a conditional — if HR is defined, insert that, otherwise, insert the horizontal rule as above:

.LP
Fusce suscipit, wisi nec facilisis facilisis, est dui fermentum leo,
quis tempor ligula erat quis odio. Nunc porta vulputate tellus. Nunc
rutrum turpis sed pede. Sed bibendum.
.RT
.ie d HR \{\
.HR
\}
.el \{\
.sp 1
\l'\n(.lu'
.sp 1
.\}
.LP
Nunc aliquet, augue nec adipiscing interdum, lacus tellus malesuada
massa, quis varius mi purus non odio. Pellentesque condimentum, magna ut
suscipit hendrerit, ipsum augue ornare nulla, non luctus diam neque sit
amet urna.

The above will still output the same horizontal rule (with blank lines above & below), but if the user were to define their own HR like so:

.de HR
.sp 1
.ce 1
***
.sp 1
..

...and then pass that as a configuration:

$ < test.md lowdown -Tms | cat config.ms - | groff -ms > out.ps

Then they would get the literary-style scene break (or whatever else they defined).

Thoughts?

p.s. I'm using the LP macro for paragraphs occurring after a horizontal rule, which I think it appropriate, although it appears lowdown uses the PP macro in this case.

Markdown links in man output

Links like

[page](page_path)

are always rendered as page when using lowdown -sTman foo.md | mandoc.

Ideally I'd see the url separated from the title with : , instead of hiding it completely.

Better support for cli diff

I was trying to utilize lowdown-diff for diffing files with git.

I've been configuring things like this:

In my global ~/.gifconfig:

[diff "markdown"]
	tool = lowdown

[difftool "lowdown"]
	cmd = "lowdown-diff -Tman $LOCAL $REMOTE | groff -Tascii -man | less"

I have in my .gitattributes:

*.md diff=markdown

And I wanted git diff just to utilize lowdown-diff to build a diff, but that doesn't seem to work.

So I'll need to use git difftool instead.

With the config above I can git difftool -t lowdown README.md. But that output isn't really as great as the html output is.

Lowdown crashes

On OpenBSD 6.5-current (specifically March 29 snapshot on amd64) a simple file test.txt consisting of

Title: Test file
Author: Lastname, Firstname
Date:

## First header

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

## Second header

Lorem ipsum dolor sit amet, consectetur adipiscing elit.

causes the command lowdown test.txt to result in this:

assertion "! (i + 1 < sz && '\n' == data[i + 1])" failed: file "document.c", line 3428, function "parse_metadata_val"
Abort trap (core dumped)

Heap corruption & crash with -Tterm

With current master (0.7.7):

$ cat /tmp/repro.md
# NAME

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
$ ./lowdown -Tterm /tmp/repro.md
free(): corrupted unsorted chunks
Aborted

git bisect shows fb44bef (Make -Tterm output aware of UTF-8 characters when computing column width.) as the first bad commit.

valgrind is showing a bunch of errors in the same codepaths, and none with the commit above reverted. Happy to provide the valgrind output, if that helps.

Off-by-one error in Base Header Level metadata (with patch)

As I understand MultiMarkdown's metadata, setting "Base Header Level: 1" in metadata is a null function. With the current code base, Lowdown increments by the value of Base Header Level. The following example shows the issue:

% echo 'title: Big Test\nBase Header Level: 1\n\n# Chapter 1\n\nThis is a test.\n\n## Sub-heading\n\nThis is only a test.' | ./lowdown -Tms -s
.TL
Big Test
.NH 2
.XN Chapter 1
.PP
This is a test.
.NH 3
.XN Sub-heading
.PP
This is only a test.

I've attached a patch for nroff, html, and latex that defaults Base Header Level to 1, then accounts for it when setting heading levels.
basehdr.patch.txt

P.S. I've tested this with nroff output, but not with HTML or LaTeX.

Regression in printing tables between 0.7.9 and master

I was unable to git bisect this, because a few commits were simply not printing anything.

From the following markdown:

# Mirrors

| Repository                                     | Location         |
|------------------------------------------------|------------------|
| https://alpha.de.repo.voidlinux.org/         | EU: Finland      |
| <https://mirrors.servercentral.com/voidlinux/> | USA: Chicago     |
| <https://alpha.us.repo.voidlinux.org/>         | USA: Kansas City |
| <https://mirror.clarkson.edu/voidlinux/>       | USA: New York    |

Using ./lowdown -s -Tman mirrors.md; with -Tms on master it's broken in the same way, and on 0.7.9 it's broken in a weirder way still.

With lowdown 0.7.9, I get:

.TH "Untitled article" 7
.SH Mirrors
.TS
tab(|) expand allbox;
lb lb
l l.
T{
Repository
T}|T{
Location
T}
T{
\f[I]https://alpha.de.repo.voidlinux.org/\f[R]

T}|T{
EU: Finland
T}
T{
\f[I]https://mirrors.servercentral.com/voidlinux/\f[R]

T}|T{
USA: Chicago
T}
T{
\f[I]https://alpha.us.repo.voidlinux.org/\f[R]

T}|T{
USA: Kansas City
T}
T{
\f[I]https://mirror.clarkson.edu/voidlinux/\f[R]

T}|T{
USA: New York
T}
.TE

and with lowdown from master, I get:

.\" -*- mode: troff; coding: utf-8 -*-
.TH "Untitled article" "7" ""
.SH Mirrors
.TS
tab(|) expand allbox;
lb lb
l l.
T{
Repository
T}
|
T{
Location
T}
T{
\fIhttps://alpha.de.repo.voidlinux.org/\fR
T}
|
T{
EU: Finland
T}
T{
\fIhttps://mirrors.servercentral.com/voidlinux/\fR
T}
|
T{
USA: Chicago
T}
T{
\fIhttps://alpha.us.repo.voidlinux.org/\fR
T}
|
T{
USA: Kansas City
T}
T{
\fIhttps://mirror.clarkson.edu/voidlinux/\fR
T}
|
T{
USA: New York
T}
.TE

With the first one, I get a normal looking table without any issues, but the latter generates a completely broken table (I'm displaying them with mandoc(1)).

Custom metadata keys

Hi,

I'd like to have an updatedate key, similar to date, to indicate the date when the post was last updated. It'd be nice if lowdown supported custom metadata keys, perhaps possibly with a notion for declaring the "type" of the field's value.

Can this library support installing a shared object?

I have a patch for the Makefile at https://github.com/NixOS/nixpkgs/blob/be7f1c9cf06a231a06cd51775c900e5e579ff95e/pkgs/tools/typesetting/lowdown/shared.patch, but I don't know what to do about the autogenerated configure script.

Download Issue while trying to use lodown package

Hi,

I am trying to download PNAD-Contínua using lodown package with the query below:

pnadc_cat <-
get_catalog( "pnadc" ,
output_dir = file.path( path.expand( "~" ) , "PNADC" ) )
pnadc_cat <- subset( pnadc_cat , year == 2018 & quarter == '03' )
pnadc_cat <- lodown( "pnadc" , pnadc_cat )

Before the error, R trys 10 times to download and it keeps giving the message as follow:

download issue with
'ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_continua/Trimestral/Microdados/Documentacao/Dicionario_e_input_20181114.zip'

The error log is this:

Error in httr_filesize(this_url, attempts, sleepsec) :
httr::HEAD( 'ftp://ftp.ibge.gov.br/Trabalho_e_Rendimento/Pesquisa_Nacional_por_Amostra_de_Domicilios_continua/Trimestral/Microdados/Documentacao/Dicionario_e_input_20181114.zip' )
failed after 10 attempts
year quarter interview
26 2018 03

When I put this address at google Chrome it download correctly. Do you think that there is something else I might be trying?

Odd indentation in definition list with -Tman

Not sure this is actually related to definition lists, this may be a generic indentation problem.

I'm trying to parse the mmdblookup manual page, from the libmaxminddb project.

This excerpt:

This application accepts the following options:

-f, --file

:    The path to the MMDB file. Required.

-i, --ip

:    The IP address to look up. Required.

-v, --verbose

:    Turns on verbose output. Specifically, this causes this
     application to output the database metadata.

--version

:    Print the program's version number and exit.

-h, -?, --help

:    Show usage information.

Seems to be misparsed. The mandoc/groff output is:

   -f, --file
              The path to the MMDB file. Required.

     -i, --ip
              The IP address to look up. Required.

-v, --verbose
              Turns on verbose output. Specifically, this causes this
               application to output the database metadata.

    --version
              Print the program's version number and exit.

-h, -?, --help
              Show usage information.

It seems there are a bunch of e.g. .ti -\w'-f, --file 'u, which I don't really understand. Why would we de-indent based on the length of the line above? Anyway, I'm beyond my depth here, so leaving it to you :)

The -Thtml output looks OK. The -Tterm output… crashes, due to both #47, and I think another underlying bug...

Support metadata definition in command line arguments

Hi! In Void Linux we make heavy usage of markdown for our https://github.com/void-linux/void-docs project. Since last year we have a void-docs package which ships the generated HTML from mdbook, together with the markdown sources and mandoc converted by pandoc.

Regarding specifically the markdown -> mandoc conversion, I'd like to switch from pandoc to lowdown, if at all possible. However, this requires some changes, since we currently pass information such as page title and section in the pandoc command line (https://github.com/void-linux/void-docs/blob/58f2e4d95aa9110301db9dde2cccabd568731011/res/build.sh#L22). I'd also like to avoid including the metadata supported by lowdown in each and every file. Therefore, I'd like to know if there's interest in supporting a --metadata or similar command line flag, which would allow me to define these values in the lowdown command line like --metadata title="something". If you think that's a feature you'd like to have, I can definitely work on it. Just want to be sure a PR to this effect has chances of being accepted.

Furthermore, would a header metadata entry, like the one supported by pandoc, and which creates a title line like .TH "SUMMARY" "7" "" "" "Void Docs" (for header="Void Docs"), be accepted?

On a side note, I'm glad to have found lowdown. The term output allows me to depend only on C instead of a Rust program (we are currently using mdcat to display markdown in the terminal) :)

Backticks unescaped

Backticks are unescaped in MS output.

Tarball includes config.h; leading ./

The tarball (for e.g. both 0.7.5 and 0.7.6) includes config.h, presumably from the author's system :)

Making the lowdown.tar.gz target depend on distclean would make this go away:

-lowdown.tar.gz:
+lowdown.tar.gz: distclean

This would have the side-effect that subsequent invocations of make fail, as Makefile.configure would disappear. The alternative would be to manually rm config.h in .dist.

Separately, the tarball contents have a leading ./ for all files, e.g. ./lowdown-0.7.6/LICENSE.md. This would fix that:

-	( cd .dist/ && tar zcf ../$@ ./ )
+	( cd .dist/ && tar zcf ../$@ lowdown-$(VERSION) )

(Alternatively, you could also just do tar -C .dist zcf $@ lowdown-$(VERSION), which seems to be portable enough?)

<a> tag is missing in some corner cases

Running in shell:

$ printf "pre-text [text](http://url)\n\n# title" | lowdown # FAIL the <a> is missing
<h1 id="title">title</h1>
$ printf "pre-text [text](http://url)" | lowdown # PASS
<p>pre-text <a href="http://url">text</a></p>
$ printf "pre-text [text](url)\n\n# title" | lowdown # PASS relative link
<p>pre-text <a href="url">text</a></p>

<h1 id="title">title</h1>
$ printf "pre-text [text](//url)\n\n# title" | lowdown # PASS without protocol in href
<p>pre-text <a href="//url">text</a></p>

<h1 id="title">title</h1>
$ printf "[text](http://url)\n\n# title" | lowdown # PASS without pre-text
<p><a href="http://url">text</a></p>

<h1 id="title">title</h1>

So it fails only when:

link contains protocol in href
link in the first line
text before the link in the same line
a header goes below the link

Single quotes at the beginning of line are not escaped

OK, this is a good one -- you can't make these things up!

The libmaxminddb man page has this excerpt (line wrappings intact):

This function will work with IPv4 addresses even when the database contains
data for both IPv4 and IPv6 addresses. The IPv4 address will be looked up as
'::xxx.xxx.xxx.xxx' rather than being remapped to the `::ffff:xxx.xxx.xxx.xxx`
block allocated for IPv4-mapped IPv6 addresses.

With lowdown 0.8.0 and -Tman, this results into an almost identical output (only ``` escaped).

But, this means that we now have the token '::xxx.xxx.xxx.xxx' , at the beginning of a line, which groff interprets as a macro, hiding it from the output, and emitting a warning (warning: macro '::xxx.xxx.xxx.xxx'' not defined)

Latest update doesn't build on MacOS 'unknown type name 'uint32_t''

It looks like stdint.h needs to be included in nroff_escape.c and util.c

Sample make output, below, is for nroff_escape.c, once that gets the include, make complains about util.c.

(Incidentally, there are two warnings – on successful completion of make – regarding -Wdeprecated-declarations, output is at the bottom of this issue.)

$ ./configure PREFIX=$HOME/local
config.log: writing...
configure.local: no (fully automatic configuration)
arc4random: yes
b64_ntop: yes (with -lresolv)
capsicum: no
err: yes
explicit_bzero: no
getprogname: yes
INFTIM: no
md5: no
memmem: yes
memrchr: no
memset_s: yes
PATH_MAX: yes
pledge: no
program_invocation_short_name: no
reallocarray: no
recallocarray: no
sandbox_init: yes
seccomp-filter: no
SOCK_NONBLOCK: no
strlcat: yes
strlcpy: yes
strndup: yes
strnlen: yes
strtonum: no
sys_queue: yes
systrace: no
unveil: no
zlib: yes
__progname: yes
config.h: written
Makefile.configure: written

$ make
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o autolink.o autolink.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o buffer.o buffer.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o diff.o diff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o document.o document.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o html.o html.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o html_escape.o html_escape.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o html_smartypants.o html_smartypants.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o library.o library.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o libdiff.o libdiff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o log.o log.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o nroff.o nroff.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o nroff_escape.o nroff_escape.c
In file included from nroff_escape.c:25:
./lowdown.h:171:4: error: unknown type name 'uint32_t'
                        uint32_t num; /* index in ordered */
                        ^
1 error generated.
make: *** [nroff_escape.o] Error 1

main.c:75:7: warning: 'sandbox_init' is deprecated: first deprecated in macOS
      10.8 [-Wdeprecated-declarations]
        rc = sandbox_init
             ^
/usr/include/sandbox.h:46:5: note: 'sandbox_init' has been explicitly marked
      deprecated here
int sandbox_init(const char *profile, uint64_t flags, char **errorbuf);
    ^
main.c:76:4: warning: 'kSBXProfilePureComputation' is deprecated: first
      deprecated in macOS 10.8 [-Wdeprecated-declarations]
                (kSBXProfilePureComputation,
                 ^
/usr/include/sandbox.h:78:19: note: 'kSBXProfilePureComputation' has been
      explicitly marked deprecated here
extern const char kSBXProfilePureComputation[];

--parse-no-codeind inverted?

Hi,

I happen to have a file with some indentation that lowdown treats as code blocks, even with `--parse-no-codeind. Maybe my brain is broken and I'm interpreting this wrong; no-codeind should disable treating indented text as code, but seems to do the opposite?

Thanks for your time and effort,
Matt

Example file:

# indented

    $ # indented code
    $ ls -al

# end

Output:

$ lowdown -Ttree example
LOWDOWN_ROOT
  LOWDOWN_DOC_HEADER
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 8 Bytes: indented
  LOWDOWN_BLOCKCODE
    data: 27 Bytes: $ # indented code\n$ ...
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 3 Bytes: end
  LOWDOWN_DOC_FOOTER

$ lowdown -Ttree --parse-no-codeind example
LOWDOWN_ROOT
  LOWDOWN_DOC_HEADER
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 8 Bytes: indented
  LOWDOWN_BLOCKCODE
    data: 27 Bytes: $ # indented code\n$ ...
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 3 Bytes: end
  LOWDOWN_DOC_FOOTER

$ lowdown -Ttree --parse-codeind example
LOWDOWN_ROOT
  LOWDOWN_DOC_HEADER
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 8 Bytes: indented
  LOWDOWN_PARAGRAPH
    lines: 2, blank-after: 1
    LOWDOWN_NORMAL_TEXT
      data: 21 Bytes:     $ # indented cod...
    LOWDOWN_NORMAL_TEXT
      data: 13 Bytes: \n    $ ls -al
  LOWDOWN_HEADER
    level: 1
    LOWDOWN_NORMAL_TEXT
      data: 3 Bytes: end
  LOWDOWN_DOC_FOOTER

Build fails on MacOS, could not find 'diff.h'

After updating the master branch, I get the following:

[master] ~/src/lowdown $ make
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o autolink.o autolink.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o buffer.o buffer.c
cc -g -W -Wall -Wextra -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wno-unused-parameter   -c -o diff.o diff.c
diff.c:33:10: fatal error: 'diff.h' file not found
#include <diff.h>
         ^~~~~~~~
1 error generated.
make: *** [diff.o] Error 1

diff(1) on MacOS appears to be gnu diff, but perhaps header files aren't installed – I'm having trouble finding them, at any rate. Or did you have something else in mind here?

URL handling in -Tman

Imagine this excerpt:

Please report all issues to
[our GitHub issue tracker](https://github.com/kristapsdz/lowdown/issues).

The -Thtml output is, somewhat obviously :)

<p>Please report all issues to
<a href="https://github.com/kristapsdz/lowdown/issues">our GitHub issue tracker</a>.</p>

But, the -Tman output is:

.PP
Please report all issues to
\f[I]our GitHub issue tracker\f[R].

In other words, the URL is lost from the HTML output. A user that is looking at that man page does not know where they should report that bug report, or what link to click.

pandoc adds the URL in parenthesis after the text (our GitHub issue tracker (https://github.com/kristapsdz/lowdown/issues), for what it's worth :)

New feature (with code): "pretty" paragraphs for -Tms

I've implemented a new option, —nroff-pretty-para, to use .PP except after headings, lists, tables, and code blocks (all of which are followed by .LP). Patch attached.

lowdown_pretty_para.patch.txt

[Feature proposal] Support for GFM's todo extension

Github-flavored markdown has support for todo lists that can be completed:

todo item
completed item

I'd like to add functionality for this. Would it be something you'd want to merge?

	Multiple CSS files (in order) may be separated by more than one space
	(including newlines).

	if (css != NULL && !rndr_meta_multi
	(ob, css, "<link rel=\"stylesheet\" href=", " />"))

kristapsdz / lowdown Goto Github PK

lowdown's People

Contributors

Stargazers

Watchers

Forkers

lowdown's Issues

Text lines

Link lines

Links before quotes

Pre-lines (triple-backticks)

List lines

Headers

Blockquote

Emphasis and similar

Recommend Projects

Recommend Topics

Recommend Org