Would it be possible to have a VS code extension that runs codebraid? I see there's a similar open issue with Spyder integration (#17). The main motivation would be to make it easier and more user-friendly to run codebraid (in my experience, not everyone is comfortable using the command line).

When I try to run the below code as a block in codebraid with options .bash .cb.run I get an error that appears to be related to formatting. However I cannot figure out what the issue is.

array=( one two three )
for i in "${array[@]}"
do
	echo ${i}
done

Error:

codebraid pandoc --from markdown --to markdown test_for.md
Traceback (most recent call last):
  File "/Users/aguang/miniconda3/bin/codebraid", line 10, in <module>
    sys.exit(main())
  File "/Users/aguang/miniconda3/lib/python3.6/site-packages/codebraid/cmdline.py", line 62, in main
    args.func(args)
  File "/Users/aguang/miniconda3/lib/python3.6/site-packages/codebraid/cmdline.py", line 85, in pandoc
    converter.code_braid()
  File "/Users/aguang/miniconda3/lib/python3.6/site-packages/codebraid/converters/base.py", line 1161, in code_braid
    self._extract_code_chunks()
  File "/Users/aguang/miniconda3/lib/python3.6/site-packages/codebraid/converters/pandoc.py", line 1146, in _extract_code_chunks
    self._load_and_process_initial_ast(source_string=source_string, single_source_name=source_name)
  File "/Users/aguang/miniconda3/lib/python3.6/site-packages/codebraid/converters/pandoc.py", line 1070, in _load_and_process_initial_ast
    source_name, line, line_number = next(source_name_line_and_number_iter)
StopIteration

I'm getting errors with pandoc 2.11. Codebraid complains about this version with the error message that pandoc version >= 2.4 is required...

Is there a reason that --citeproc is omitted from PANDOC_OPTIONS? I got it working along with --bibliography by adding

('-C', '--citeproc'): 0

to PANDOC_OPTIONS.

Minimal failing example

Input:

1.  <!-- -->

Output:

1.  `{.codebraid--temp format="html"}     <!-- -->`

Expected output: same as input.

In my case, I convert a markdown file to HTML which then is rendered with QT. The main code is

webView = QWebEngineView()
webView.setHtml(pageSource)

as you may have noticed, the contain pageSource is used here. If an HTML file is used, an extra step has to be performed

pageSource = open('page.html').read()

So what I suggest is

pageSource = codebraid.convert('web.cbmd', from='markdown', to='markdown',  ignorePandoc=True)

and in this case, I think pandoc is not necessary, so I set ignorePandoc=True

Thanks

I'm using Python 3.8 in a virtual environment provided by Poetry running on Windows 10. I find that the Python executable called by Codeweave does not respect the Windows PATH environment variable. Instead of calling the Python executable of the currently activated virtual environment (that Codebraid itself is running in), it calls a Windows system Python. By default, this produces a message referring you to the Microsoft store, and this message appears in the output HTML!

I tracked the issue down to the calls to subprocess.run in _subproc_default. There is a known issue with subprocess:
https://bugs.python.org/issue15451

From this Stack Overflow answer I was able to fix the problem by inserting the following at the beginning of _subproc_default

args = [shutil.which(args[0])] + args[1:]

Thank you for Codebraid!

Just wanted to get your thoughts on how you use codebraid for your work.

My current workflow is to write the markdown in VSCode and then run the codebraid command every time I want a new version of my document but it's a bit cumbersome.

What's your set up when you write with codebraid? Do you have any recommendations on software?

When running python with tabs instead of spaces for scope, codebraid does not compile the document.
This will fail

```{.python .cb.run}
for i in range(5):
	print(i)

While this will run

```{.python .cb.run}
for i in range(5):
    print(i)

I've come up to a problem where if I include a \gather environment in my markdown file, I believe that codebraid interprets it as code and fails when running it. Here's a minimal example that doesn't run on my system (backslashes added to python code cell so github showed the whole snippet):

---
title: Midterm
geometry: margin=1in
output: pdf_document
---

\```{.python .cb.nb jupyter_kernel=python3 show=code:verbatim+stdout:raw}
print("hello")
\```
Table: My Table

\begin{gather}
X_{1i} = a_0 + a_1 X_{2i} + \hat{v}_i \\
Y_i = b_0 + b_1 \hat{v}_i + e_i
\end{gather}

Am I doing something wrong?

According to the current state oif the documentation, .cb.run should give "raw" output. This does not happen with "implicit" output, i. e. the result of an evaluation without explicit printing: in this case, most of the interpreters print the result to their stanrd output.

Nothing is printed when an inline expression in Python, Sage and Julia ; however, both R and bash give their raw output. When exported to markdown, the following code :

Language `run`                       `nb`                        `expr`                       `run`+print
------- --------------------------- ---------------------       -----------------------      ------------------------------------
Python   `1+2`{.python .cb.run}      `1+2`{.python .cb.nb}       `1+2`{.python .cb.expr}      `print(1+2)`{.python .cb.run}
Sage     `1+2`{.sage .cb.run}        `1+2`{.sage .cb.nb}         `1+2`{.sage .cb.expr}        `print(1+2)`{.sage .cb.run}
Julia    `1+2`{.julia .cb.run}       `1+2`{.julia .cb.nb}        `1+2`{.julia .cb.expr}       `print(1+2)`{.julia .cb.run}
R        `1+2`{.R .cb.run}           `1+2`{.R .cb.nb}            `1+2`{.R .cb.expr}           `print(1+2)`{.R .cb.run}
bash     `expr 1 + 2`{.bash .cb.run} `expr 1 + 2`{.bash .cb.nb}` `expr 1 + 2`{.bash .cb.expr} `echo $(expr 1 + 2)`{.bash .cb.run}

results in

  Language     `run`        `nb`       `expr`   `run`+print
  ---------- --------- -------------- -------- -------------
  Python                 `3`{.expr}      3           3
  Sage                   `3`{.expr}      3           3
  Julia                  `3`{.expr}      3           3
  R           \[1\] 3    `3`{.expr}      3        \[1\] 3
  bash           3      `3`{.expr}\`     3           3

And the export to PDF via latex gives "teletype" printing in the nb column and normal (i. e. roman) printing in the other columns.

Similarly, the following code:

## Block syntax

### Python

```{.python .cb.run}
1+2
```
### Sage

```{.sage .cb.run}
1+2
```

### Julia

```{.julia .cb.run}
1+2
```

### R

```{.R .cb.run}
1+2
```

### Bash

```{.bash .cb.run}
expr 1 + 2
```

Gives the following result:

Block syntax
------------

### Python

### Sage

### Julia

### R

\[1\] 3

### Bash

3

In both cases (inline and block), Python's, Sage's and Julia's implicit printings do not get in the output, whereas bash's and R's do.

This inconsistency should be fixed or, to the least, documented.

The same problem exists for the implicit printing of figures created by R or Sage. This should at least be documented...

I have the following markdown:

% example
% Jay Cunningham
% 22 April 2021

# What's happening?

```{.python .cb.nb jupyter_kernel=python3 example=false session=foo}
print("hello, world")

I'm running the following command to create slides:

codebraid pandoc -f markdown example.md -t revealjs -s -i -o example.html

If I install jupyter-client==6.1.12, it works fine. If I upgrade it to the most-current jupyter-client==6.2.0, I (usually?) get the following error:

Traceback (most recent call last):
  File "/Users/j260381/.pyenv/versions/kayday-deep-dive/bin/codebraid", line 8, in <module>
    sys.exit(main())
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/cmdline.py", line 107, in main
    args.func(args)
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/cmdline.py", line 153, in pandoc
    converter.code_braid()
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/converters/base.py", line 1199, in code_braid
    self._process_code_chunks()
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/converters/base.py", line 1213, in _process_code_chunks
    cp.process()
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/codeprocessors/base.py", line 603, in process
    self._run_jupyter(session)
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/kayday-deep-dive/lib/python3.8/site-packages/codebraid/codeprocessors/base.py", line 1384, in _run_jupyter
    if msg['parent_header'].get('msg_id') != cc_jupyter_id:
TypeError: 'coroutine' object is not subscriptable
sys:1: RuntimeWarning: coroutine 'ZMQSocketChannel.get_msg' was never awaited

It sometimes works the first time I run codebraid after installing 6.2.0, but it always fails the second or further times I run it after making a change to the markdown.

When using inline code, the resulting text shows up in a different font and with quotation marks. Is there a way to get around this behavior? For example:

1+2{.python .cb.nb session=python} yields '3', not just 3 as a number in the text.

Looks like codebraid might getting plugin for Spyder IDE! Great work on the package!

spyder-ide/spyder-reports#79

Many people will wonder why codebraid instead of Jupyter. Is it all about Pandoc?

It makes sense to address that in the README.

Oftentimes when running codebraid, I get a StopIteration error and it is difficult to tell where this error comes from. Is there a way to make the StopIteration error more informative, like giving the snippet of text that causes the issue or the line number?

Pandoc version 2.10 is read as 2.1

$ codebraid pandoc --from markdown --to markdown notebook.md
Traceback (most recent call last):
  File "/home/removed/.local/bin/codebraid", line 8, in <module>
    sys.exit(main())
  File "/home/removed/.local/lib/python3.8/site-packages/codebraid/cmdline.py", line 62, in main
    args.func(args)
  File "/home/removed/.local/lib/python3.8/site-packages/codebraid/cmdline.py", line 80, in pandoc
    converter = converters.PandocConverter(paths=args.files,
  File "/home/removed/.local/lib/python3.8/site-packages/codebraid/converters/pandoc.py", line 569, in __init__
    raise RuntimeError('Pandoc at "{0}" is version {1}, but >= 2.4 is required'.format(pandoc_path, float(pandoc_version_match.group())))
RuntimeError: Pandoc at "pandoc" is version 2.1, but >= 2.4 is required

$ pandoc -v
pandoc 2.10.1
Compiled with pandoc-types 1.21, texmath 0.12.0.3, skylighting 0.8.5
Default user data directory: /home/removed/.local/share/pandoc or /home/removed/.pandoc
Copyright (C) 2006-2020 John MacFarlane
Web:  https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

When converting from markdown to PDF, and you escape asterisks, you get a StopIteration error. For example:

A\*

and run codebraid pandoc -o test.pdf test.md

I was trying to understand how to generate plots in a codebraid file with julia codeblocks. Julia has its own jupyter kernel, usually called IJulia.

Would something like this work, or does codebraid not yet support Julia jupyter kernels? Also, usually Julia jupyter kernels are attached to a Julia virtual environment. I think codebraid designed to pick the Julia environment that is in the same directory as the pandoc markdown file--as per the changelog and the addition of the --project=@. flag. So would this set then allow me to call the local IJulia jupyter kernel from the local Julia environment--as opposed to the global environment, etc.

I tried the following code:

```{.julia .cb-nb jupyter_kernel=julia session=random example=true}
using Plots
x = 1:10; y = rand(10); # These are the plotting data
plot(x, y)

While there are no errors, the file does not show the actual plot. I also attached the original file and the pdf.
julia_output.pdf
julia-braid-test.md

There's a comparison between codebraid and knitr, jupyter notebook, Pweave. Can add comparison between codebraid and jupyter-book as well?

I get an error when I try to convert from markdown to pdf with latex commands defined in header-includes in font matter.

File test.md

---
header-includes: |
    \newcommand{\Ker}{\mathrm{Ker}}
---

# Title

$\Ker$

Command run

codebraid pandoc --from markdown --to pdf test.md -o test.pdf

Error

! LaTeX Error: Command \Ker already defined.

I don't get any error if I try to convert from markdown to markdown with codebraid or when I convert from markdown to pdf directly with pandoc :

pandoc --from markdown --to pdf test.md -o test.pdf

pandoc.exe is in one directory of my PATH environmental variant, so I can run pandoc.exe under any directory in the dos console
however, codebraid can not find pandoc.exe. So where should I put pandoc.exe? Or should codebraid consider PATH?

thanks

I recently experienced an issue with working with multiple source files that would then be combined into one larger document, e.g. multiple files representing book chapters. If the files are set up to run individually with the notebook executor (i.e. .cb.nb) then execution will fail silently when trying to execute and combine the files into a single document.

Minimal reproducing example

Say you have two source files ch1.md and ch2.md that you want to execute+compile into book.pdf:

Contents of ch1.md:

# Ch. 1 - Uniform distribution

A histogram of uniformly-distributed random numbers.

```{.python .cb.nb jupyter_kernel=python3}
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng()
plt.hist(rng.uniform(size=1000))
```

Contents of ch2.md

# Ch 2. - Normal Distribution

A histogram of normally-distributed random numbers.

```{.python .cb.nb jupyter_kernel=python3}
import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng()
plt.hist(rng.standard_normal(size=1000))
```

Executing/converting the files individually works as expected:

$ codebraid pandoc --from markdown --to pdf ch1.md --standalone -o book.pdf

However, if you try to compile both documents into a single book, neither document is executed, though no warning or error are given on the command line:

$ codebraid pandoc --from markdown --to pdf ch1.md ch2.md --standalone -o book.pdf

In the latter case, if you look at the output book.pdf you will find an error printed:

SOURCE ERROR in "ch2.md" near line 6:
Some options are only valid for the first code chunk in a session: "jupyter_kernel"

IMO it would be helpful to the user if this error were raised at the command line rather than (or in addition to being) embedded in the output document. In my actual use-case with much larger chapters, it was a very long time before I noticed this in the output book.

The error in book.pdf seems to suggest that the problem lies with the "special" metadata jupyter_kernel, which is only supposed to be supplied in the first code cell. This suggests that an author would have to modify source file metadata if they wanted to switch between building individual chapters and the entire book. I hadn't noticed this mentioned in the docs before - if it's not there, then it would be an improvement if this behavior were documented.

Perhaps this can be avoided if .cb.run is used instead of .cb.nb? Is there a preferred way of using codebraid to have flexible outputs w/ multiple source files?

With the addition of Jupyter kernels and showing plots directly from executed code, would there be some way to integrate codebraid with pandoc-fignos so that there could be figure captions in the resulting text?

My dream is to use codebraid to write README.md files for all my
projects.

To achieve this I have come up with a Makefile to convert all .cbmd
files in a repository. This makes it much easier to get up and running
with a more automated workflow. I feel it is a must to convince all my
programmer friends that using Codebraid for this purpose is awesome. I
would love to see literate programming reach a wider audience.

I have been exploring the possibilites in this repo if you are curious.

Would you be interested in a PR with an example for this workflow?
Perhaps as a subfolder in the examples directory.

Makefile:

CODEBRAID_DOCS := $(shell find . -name '*.cbmd')
MARKDOWN_DOCS = $(CODEBRAID_DOCS:.cbmd=.md)

.PHONY: cbmd watch

# Must cd into the target file directory so Pandoc and Codebraid are
# able to use relative paths as one would expect
%.md : %.cbmd
	cd $(@D); codebraid pandoc --from markdown --to gfm --filter=mermaid-filter --output $(notdir $@) --overwrite $(notdir $^)

cbmd : $(MARKDOWN_DOCS)

watch :
	find . -name '*.cbmd' | entr make cbmd

I have just tried converting a document using the relatively new -d option, which resulted in codebraid throwing errors... Why? Aren't pandoc arguments just passed through?

Minimal example:

[@test]

Fails with:

$ codebraid pandoc /tmp/test.md 
Traceback (most recent call last):
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/bin/.codebraid-wrapped", line 9, in <module>
    sys.exit(main())
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/lib/python3.7/site-packages/codebraid/cmdline.py", line 62, in main
    args.func(args)
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/lib/python3.7/site-packages/codebraid/cmdline.py", line 85, in pandoc
    converter.code_braid()
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/lib/python3.7/site-packages/codebraid/converters/base.py", line 1161, in code_braid
    self._extract_code_chunks()
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/lib/python3.7/site-packages/codebraid/converters/pandoc.py", line 1146, in _extract_code_chunks
    self._load_and_process_initial_ast(source_string=source_string, single_source_name=source_name)
  File "/nix/store/n1nd5j5m583llfbbmqm379kmg98335zg-codebraid-0.4.0/lib/python3.7/site-packages/codebraid/converters/pandoc.py", line 1007, in _load_and_process_initial_ast
    node_type = node['t']
KeyError: 't'

Hi,
This looks like a really nice project, and I really wanted to give it a try but unfortunately it crashes badly. First, invoking simply codebraid (without any args) results in:

Traceback (most recent call last):
  File "/home/<user>/venv/pyactr/bin/codebraid", line 11, in <module>
    sys.exit(main())
  File "/home/<user>/venv/pyactr/lib/python3.6/site-packages/codebraid/cmdline.py", line 57, in main
    args.func(args)
AttributeError: 'Namespace' object has no attribute 'func'

And using codebraid pandoc rep.md -o rep.pdfreturns:

Traceback (most recent call last):
  File "/home/<user>/venv/pyactr/bin/codebraid", line 11, in <module>
    sys.exit(main())
  File "/home/<user>/venv/pyactr/lib/python3.6/site-packages/codebraid/cmdline.py", line 57, in main
    args.func(args)
  File "/home/<user>/venv/pyactr/lib/python3.6/site-packages/codebraid/cmdline.py", line 78, in pandoc
    other_pandoc_args=other_pandoc_args)
  File "/home/<user>/venv/pyactr/lib/python3.6/site-packages/codebraid/converters/pandoc.py", line 1143, in convert
    raise TypeError
TypeError

Do you know what could be wrong?

Currently using

python{.python .cb.run show=code+stdout}
var = 'Hello from Python!'
print(var)

(ie syntax specific markdown) in a markdown file and executing it using codebraid with the '--to markdown' flag causes an error. Just using {.python .cb.run show=code+stdout} results in

{.python .numberLines startFrom="1"}
var = 'Hello from Python!'
print(var)

Would it be possible to implement a feature whereby language-specific syntax highlighting is preserved during execution?

Could you please consider adding a feature of returning non-zero exit codes if one or more exceptions were issued? It would help e.g in using codebraid in Makefiles.

Currently we are to grep the output documents for magic words from stack traces which is a bit fragile.

Hey,
Thanks for this project, seems to nearly fill exactly what I was looking for.

I am getting stuck on one thing - I'm writing shell code cells, but I'm wanting to use a custom executable/shell for these. For this example let's say I want to use /usr/bin/fish, but full disclosure, what I really want is to get codebraid to run its stuff in a container, by passing the executable podman run -it docker.io/ubuntu:21.10.

When passing executable= to the shell session block as per docs, I get an "executable not found" error:

```{.bash .cb-nb session=dock executable="podman run --name mine -v .:/demo -it ubuntu:21.10"}
set -e
cd /demo/demo
pwd
```

``` {.error .sysConfigError}
SYS CONFIG ERROR in "post.md" near line 88:
Could not find executable "podman run --name mine -v .:/demo -it ubuntu:21.10"
```

When passing executable= to the shell session block as a list because I'm guessing it might get send to subprocess.run, that block doesn't run and future blocks use bash directly.

```{.bash .cb-nb session=dock executable=["podman", "run", "--name", "mine", "-v" ".:/demo", "-it", "ubuntu:21.10"]}
set -e
cd /demo/demo
pwd
ls -l # does not run
```

```{.bash .cb-nb session=dock}
curl -sL https://nixos.org/nix/install | bash -s -- --no-daemon
pwd
ls -l # executes outside container
```

Is there some permutation like the above that's meant to work with shell blocks?

If not, is there a way I can set up a new language by writing the config file in my project, without needing to fork codebraid locally?

Simple answer to both might be "no", no worries if so :)

Thanks!

I'm creating some tables in codebraid and it takes a while, but after a minute, I get the error:

Jupyter kernel "python3" timed out during execution

I checked the code and I think it might have something to do with:

Would it be possible to let the jupyter kernel timeout be an option set when running codebraid? I can open a pull request, but I'm not sure whether you would want it exposed in Converter or somewhere else?

The following code

```{.python .cb.run}
print('''
|   Name    |  SST-2  |  MRPC  |
|-----------|---------|--------|
| accuracy  |  0.868  | 0.757  |
| buildtime | 781.831 | 50.946 |
''')
```

produces correct output documents of html format (<table>...</table>) but for markdown documents the result is

  Name        SST-2     MRPC
  ----------- --------- --------
  accuracy    0.868     0.757
  buildtime   781.831   50.946

i.e. 1) Strange double-space at the start of every line and 2) no | characters mark table borders. As a result, Github doesn't parse such text as tables. Please, consider checking!

I run codebraid with teh following command:

codebraid pandoc \
  -f markdown -t markdown --no-cache --overwrite --standalone \
  --self-contained -o out.md inp.md.in

Julia support may need --project=@. or possibly a way to add custom --project. This only exists for Julia >=v0.7.

This raises the question of whether there should be a system for adding user-supplied flags to the run_command template, or allowing the user to override run_command completely. It also suggests that version-conditional run_command might be useful in some cases (even if it is ultimately avoided for Julia by dropping support for <v0.7).

Related PythonTeX issue: gpoore/pythontex#158

This was discovered in looking into #25.

Input:

```{.python .cb.run}
print('''
|   Name    |  SST-2  |  MRPC  |
|-----------|---------|--------|
| accuracy  |  0.868  | 0.757  |
| buildtime | 781.831 | 50.946 |
''')
```

Convert -t markdown with Pandoc 2.9.2.1:

```{=markdown}
|   Name    |  SST-2  |  MRPC  |
|-----------|---------|--------|
| accuracy  |  0.868  | 0.757  |
| buildtime | 781.831 | 50.946 |
```

The ```{=markdown} shouldn't be there, so there is probably an incompatibility with something changed in a recent version of Pandoc.

This isn't really a codebraid issue, but do you know if any static website generator (pelican? jekyll? Hugo?) easily supports codebraid?

If anyone knows of integration it would be nice to add to the readme, and I'll switch to any system which let's me easily use it :)

Hi. I hoped that I can display Altair plots in Jupyter mode just like we plot Matplotlib. Unfortunately, I can't get a working solution up to this moment.

For example, the code

 ```{.python .cb.nb jupyter_kernel=python3}
import altair as alt
alt.renderers.enable('notebook')
import pandas as pd
from pandas import DataFrame
import matplotlib.pyplot as plt
%matplotlib inline

df=DataFrame({'step':[0,1,2,3], 'value':[0,1,2,3]})
x=alt.Chart(df).mark_line().encode(x='step', y='value')
x.display() # Not sure, putting just `x` here produces the same result
```

outputs just

<vega.vegalite.VegaLite at 0x7fe448947fd0>

but not an image.

Could you please comment on what may be wrong and do we have a chance to make this work?

Note, I'm using current HEAD version pip install git+https://github.com/gpoore/codebraid@011464539bfb09b8611c8aef0d543532cea958bf

In knitr, one could have something like this

```{r include = FALSE}
var = 123
```

Note that $\sqrt{`r var`} =`r sqrt(var)`$.

The last line will get rendered to html as
Note that \(\sqrt{123} = 11.0905365\).

Does CodeBraid allow executing the python code within $...$ to accomplish the same thing? My initial experiment suggests that it doesn't. Any help is greatly appreciated.

I'm trying to use codebraid with a standalone MathJax installation. Currently I can run Pandoc without any errors:

pandoc --mathjax=$MJPATH -t revealjs -s -i index.md -o docs/index.html

where MJPATH is the path to MathJax.js in a MathJax@2 installation. But if I try to run essentially the same line with the same markdown file through codebraid:

codebraid pandoc --mathjax=$MJPATH -t revealjs -s -i index.md -o docs/index.html

I get this error:

Traceback (most recent call last):
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/package-dev/lib/python3.8/site-packages/codebraid/converters/pandoc.py", line 773, in _run_pandoc
    proc = subprocess.run(cmd_list,
  File "/Users/j260381/.pyenv/versions/3.8.7/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pandoc', '--from', 'json', '--to', 'revealjs', '--standalone', '--preserve-tabs', '--output', 'docs/index.html', '--variable', 'theme:simple', '--incremental', '--css', 'style.css', '--mathjax', '/Users/j260381/Projects/Training/pkg-dev-week1/node_modules/mathjax/MathJax.js']' returned non-zero exit status 64.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/j260381/.pyenv/versions/package-dev/bin/codebraid", line 8, in <module>
    sys.exit(main())
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/package-dev/lib/python3.8/site-packages/codebraid/cmdline.py", line 107, in main
    args.func(args)
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/package-dev/lib/python3.8/site-packages/codebraid/cmdline.py", line 159, in pandoc
    converter.convert(to_format=args.to_format, standalone=args.standalone,
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/package-dev/lib/python3.8/site-packages/codebraid/converters/pandoc.py", line 1244, in convert
    converted_bytes, stderr_bytes = self._run_pandoc(input=json.dumps(self._final_ast),
  File "/Users/j260381/.pyenv/versions/3.8.7/envs/package-dev/lib/python3.8/site-packages/codebraid/converters/pandoc.py", line 788, in _run_pandoc
    raise PandocError(message)
codebraid.converters.pandoc.PandocError: Failed to run Pandoc:
JSON parse error: Error in $: Failed reading: not a valid json value at '*'

It doesn't seem to make a difference what's actually in the markdown file. Using --mathjax without a path works just fine, though, so that makes it a somewhat academic exercise (as grabbing it from the CDN works well enough for my use case).

I'm using:

Python 3.8.7
macOS 10.15.7
codebraid 0.5.0
MathJax 2.7.9
Pandoc 2.13

Hi. I'm using codebraid to run machine-learning experiments which sometimes take long time to complete and tend to show lots of rich debug output like gaudges and progress bars (wget tool prints similar thing).

Currently, I put long-running code in separate blocks and hide rich output by adding {hide=stdout+stderr}. Ideally, it would be nice to include only the final state of progress bars, but I think it is not that important. More useful would be to bypass certain output to the client shell as-is to be able to monitor the progress of experiment.

Could you please consider adding an option which would redirect stout/stderr of a block into the user shell when enabled?

Hi. I quickly reviewed the documentation but found no clues about per-chunk caching. I suppose it is not supported, is it?

@gpoore

I went through the code, please find below some suggestions. My goal is not to criticize at all the time and work that you have dedicated to the project. Codebraid is filling a very useful niche and am grateful for your efforts. I think a "novice" friendlier code base might attract more people to contribute. Please let me know what you think.

1. Some functions are very long e.g. the function _run. It would helpful to have more modules and smaller functions, cf. Unix philosophy
2. Related to above, it seems that a lot effort is spent to read and process the cell configuration. It looks like an DSL, maybe one can have that in a separate module
3. It would be helpful to use type hinting
4. pandoc json/AST might not be ideal to process. Maybe one can use another parser/format (marked, cmark have nicer ASTs) to pre-process and execute the code
5. It would be helpful to remove pandoc from the code base: codebraid would only convert from input-markdown to executed-markdown
6. Maybe consider to support just jupyter kernels to leverage their capabilities. Don't know the implications in terms of performance

Hi,

First off, thank you for this awesome project.

Your python example in GitHub shows that matplotlib plots needed to be inserted through markdown after being created in python, but your Scipy 2019 article has the figure being created without that step. Does that have something to do with calling the Jupyter kernel option? Can you have that output display as png so that it can be used with pandoc-fignos?

Thank you

It would beneficial to integrate with jupytext and hence jupyter notebooks. Jupyter can indeed become an interactive editor for codebraid.

To be able do so a seamless conversion from codebraid to ipynb is required. I believe a pandoc filter could achieve that.

@gpoore Would you be interested in such an approach?

When writing in markdown, codebraid throws an error if you write in raw latex:

## My document

This is text

\newpage

This is text on a new page

Do you have any interest in supporting markdown literary programming?

Markdown Literary programming that don't break the syntax of any programming language

Markdown files with html coded characters result in a StopIteration error from codebraid.

Minimal reproducing example

$ echo "I need $20." > foo.md
$ cat foo.md
I need $20.
$ pandoc --from markdown --to html foo.md -o foo.html  # Works as expected
$ cat foo.html
<p>I need $20.</p>
$ codebraid pandoc --from markdown --to html foo.md -o foo.html
Traceback (most recent call last):
  File "/home/ross/.virtualenvs/elegant-scipy/bin/codebraid", line 8, in <module>
    sys.exit(main())
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/cmdline.py", line 107, in main
    args.func(args)
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/cmdline.py", line 153, in pandoc
    converter.code_braid()
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/base.py", line 1198, in code_braid
    self._extract_code_chunks()
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/pandoc.py", line 1124, in _extract_code_chunks
    self._load_and_process_initial_ast(source_string=source_string, single_source_name=source_name)
  File "/home/ross/.virtualenvs/elegant-scipy/lib/python3.9/site-packages/codebraid/converters/pandoc.py", line 994, in _load_and_process_initial_ast
    source_name, line, line_number = next(source_name_line_and_number_iter)
StopIteration

It seems this is tangentially related to other issues re: handling escape characters e.g. #36. The discussions there and in #38 nicely explain how the StopIteration exception itself comes about.

Version info

Python: 3.9.2
codebraid: 0.5.0
pandoc: 2.13

It would be useful if codebraid could read directly from stdin.
This would allow for piping the output of a pandoc command into codebraid e.g.:

pandoc --filter <filter> --to markdown | codebraid pandoc ...

A common way to implement this in argparse is to use - to specify reading from stdin e.g.:

pandoc --filter <filter> --to markdown | codebraid pandoc - -o result.md

My experience with Pythontex being mostly positive, I'd like to do similar things with Markdown, which is more and more prevalent.

Would you consider adding support for Sagemath ?

I currently have separated doc linted with markdown-lint and scripts linted with shellcheck. I would like to merge the 2 to avoid doc maintenance.

Is there a way to run shellcheck with codebraid to lint the embedded code and not break the execution afterward ?

Thanks.