sisl / tufte_algorithms_book

A template for textbooks in the same style as Algorithms for Optimization

Tufte Algorithms Book Template

This book template provides a starting point upon which authors may freely build to generate their own textbook entirely in LaTeX. We used this setup for Algorithms for Optimization, and have continued to refine it for a new textbook on decision making under uncertainty. The template allows for the direct compilation of a print-ready PDF, including support for figures, examples, and exercises.

We do all of our development in Ubuntu.

Install Julia.

Install LaTeX via texlive. We recommend this repo.

Clone the repository to a location of your choosing:

git clone https://github.com/sisl/tufte_algorithms_book.git

Initialize and update the submodules:

git submodule init
git submodule update

Compile the style:

cd style
sudo python setup.py install
cd ..

Compile the lexer:

cd lexer
sudo python setup.py install
cd ..

Install the required Julia packages (listed in the Project.toml). You can install them manually, or simply instantiate the project:

julia --project -e 'using Pkg; Pkg.instantiate()'

Install pdf2svg, which is used by PGFPlots (we assume Ubuntu - other operating systems may install pdf2svg differently):

sudo apt-get install pdf2svg

Install pgfplots.

We require pythontex 0.18. You will probably have to update your version on texlive on miktex. Alternatively, you can download the latest version of pythontex from https://github.com/gpoore/pythontex.

(Note that on arch-based systems, one should use tllocalmgr instead.)

(If you see a pygments-related import error you might want to check out #30 and try downgrading to Pygments 2.6.1)

Test

Running make test pulls all the code and then runs all tests in juliatest blocks. See runtests.jl for details.

Compilation

• make compile compiles the whole book
• make compile CHAPTER='introduction chapter2\/introduction' will comment out every include or input statement, except for chapter/introduction and chapter/chapter2/introduction where chapter can also be appendix. Requires vim to be installed.
• make clean removes all generated files except book.pdf

If you host your project under Gitlab, .gitlab-ci.yml is a CI/CD template to start with.

tufte_algorithms_book's Issues

workflow for including code

What is a good method for displaying code in a document and then using it in a juliaconsole block?
The following seems to work, but I suspect there is a simpler way.

\begin{example}
\begin{juliablock}
some code
\end{juliablock}
\caption{example code}
\end{example}

Then I copy the code in the juliablock by hand into a file called example_code.jl.
Then I do this:

\begin{juliaconsole}
include("example_code.jl"); nothing
... use code...
\end{juliaconsole}

Issue with Pygments's unirange

When compiling the book, I am getting this error:

ImportError: cannot import name 'unirange' from 'pygments.util' (/usr/local/lib/python3.10/dist-packages/Pygments-2.15.1-py3.10.egg/pygments/util.py)
make: [makefile:14: compile] Error 1 (ignored)

Project.toml/Manifest.toml

It would be nice to have toml files instead of the deprecated REQUIRE file such that people can just instantiate the environment. I can prepare a PR if that would be helpful.

Document Replacement Feature

Document the replacement feature for the algorithm environment.

install instruction for julia packages (jlpkg missing)

my julia didn't come with jlpkg, so I could not issue the
jlpkg --project=@. add $(cat REQUIRE | grep -v julia)
command. Yes, I know, I can also install jlpkg...

For those not immediately famliar with with julia's internal (REPL) method, do:

using Pkg
Pkg.add("Colors")

(and all other packages listed in the REQUIRE file.

I'm not a julia expert yet, but I noticed this installs them in my private ~/,julia tree, not benefiting others that use the new system installed version of julia.

Cannot use functions in REPL (juliaconsole environment) defined in juliaverbatim environment.

I have an algorithm defined as follows:

\begin{algorithm}
\begin{juliaverbatim}
function my_large_sum()
    s = 0
    for i=1:100000000000
        s += i
    end
    return s
end
\end{juliaverbatim}
\end{algorithm}

This algorithm is rendered correctly and I can use it in tests, but not in the console environment:

\begin{juliaconsole}
import BenchmarkTools: @btime;
@btime my_large_sum()
\end{juliaconsole}

Doing this produces the following:

How can I access these functions in the console space?

Error running Weave

When running your example, this is the output I get. I added PythonTex v 0.18 dev as required

This is PythonTeX 0.18dev

----  Messages for juliacon:default:default  ----
* PythonTeX error
    Running code for Julia console failed
WARNING: Method definition latex(IO, Markdown.LaTeX) in module Markdown at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Markdown\src\IPython\IPython.jl:28 overwritten in module WeaveMarkdown at C:\Users\simon\.julia\packages\Weave\BarbQ\src\WeaveMarkdown\markdown.jl:10.
  ** incremental compilation may be fatally broken for this module **

ERROR: KeyError: key "tex" not found
Stacktrace:
 [1] getindex at .\dict.jl:467 [inlined]
 [2] get_format at C:\Users\simon\.julia\packages\Weave\BarbQ\src\Weave.jl:48 [inlined]
 [3] run_doc(::Weave.WeaveDoc; doctype::String, out_path::Symbol, args::Dict{Any,Any}, mod::Nothing, fig_path::Nothing, fig_ext::Nothing, cache_path::String, cache::Symbol) at C:\Users\simon\.julia\packages\Weave\BarbQ\src\run.jl:20
 [4] weave(::String; doctype::String, informat::Nothing, out_path::Symbol, args::Dict{Any,Any}, mod::Nothing, fig_path::Nothing, fig_ext::Nothing, cache_path::String, cache::Symbol, template::Nothing, css::Nothing, highlight_theme::Nothing, pandoc_options::Array{String,1}, latex_cmd::Array{String,1}, keep_unicode::Bool) at C:\Users\simon\.julia\packages\Weave\BarbQ\src\Weave.jl:177
 [5] #weave#115 at C:\Users\simon\.julia\packages\Weave\BarbQ\src\Weave.jl:233 [inlined]
 [6] weave(::String, ::String) at C:\Users\simon\.julia\packages\Weave\BarbQ\src\Weave.jl:233
 [7] top-level scope at none:1


--------------------------------------------------
PythonTeX:  book - 1 error(s), 0 warning(s)

make: [makefile:14: compile] Error 1 (ignored)

Compile generates `ImportError`

Following the instruction, I have the following compiling error after make compile. I am not sure which issue is this. Is it related to some python packages? But this error does not seems to affect the output of the pdf. I

Output written on book.pdf (24 pages, 126772 bytes).
Transcript written on book.log.
This is PythonTeX 0.17
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/texlive/2020/texmf-dist/scripts/pythontex/pythontex3.py", line 2394, in do_pygments
    lexer[codetype] = get_lexer_by_name(pygments_settings[codetype]['lexer'], **p)
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pygments/lexers/__init__.py", line 116, in get_lexer_by_name
    for cls in find_plugin_lexers():
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pygments/plugin.py", line 55, in find_plugin_lexers
    yield entrypoint.load()
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2450, in load
    return self.resolve()
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pkg_resources/__init__.py", line 2456, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "<frozen zipimport>", line 259, in load_module
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pygments_julia-0.0.0-py3.8.egg/pygments_julia/__init__.py", line 19, in <module>
    from pygments.util import shebang_matches, unirange
ImportError: cannot import name 'unirange' from 'pygments.util' (/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pygments/util.py)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/TeX/texbin/pythontex", line 62, in <module>
    pythontex.main()
  File "/usr/local/texlive/2020/texmf-dist/scripts/pythontex/pythontex3.py", line 2811, in main
    do_multiprocessing(data, temp_data, old_data, engine_dict)
  File "/usr/local/texlive/2020/texmf-dist/scripts/pythontex/pythontex3.py", line 1383, in do_multiprocessing
    result = task.get()
  File "/Users/chentianchi/opt/anaconda3/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
ImportError: cannot import name 'unirange' from 'pygments.util' (/Users/chentianchi/opt/anaconda3/lib/python3.8/site-packages/pygments/util.py)
make: [compile] Error 1 (ignored)

Failing CI at deploy

Recently the CI fails with the following message:

Stage: deploy
Name: Deploy
Trace: 	exec /busybox/sh 
else
	echo shell not found
	exit 1
fi

Any idea what could have gone wrong?

??pythontex?? error in code algorithm block

algorithms don't seem to get parsed properly, e.g. from the example

I think I have pythontex installed correctly on 0.17, and the package tests pass

Weave inserts sandbox module

Weave creates a kind of gensym module or namespace that pollutes the rendered latex

\begin{juliaconsole}
struct A end
A()
\end{juliaconsole}

Renders as this:

julia> struct A end
julia> A()
Main.##WeaveSandBox#245.A()

Did you encounter this in your book? Is there a way around it?

pipeline failed

Looks like pythontex v0.17 (it was v0.17-dev) breaks the pipeline, I'll need some time to investigate it, just open an issue here in case others meet the same problem.

Work in new table of contents format

@tawheeler greatly improved the appearance of the TOC as part of the new Algorithms for Decision Making book. It would be good to get this merged in here.

build issue for Plots.Image()

After fully rebuilding the book with a new checkout from this repo, and manually adding ImageIO which it requests, I still get an error that I can't get past:

----  Messages for jl:default:default  ----
* PythonTeX notice
    1 message(s) could not be classified
    Interpreted as errors, based on the return code(s)
  ERROR: LoadError: FileIO.File{FileIO.DataFormat{:UNKNOWN}}("/Users/abradley/Dropbox/Julia/linear_systems_book_v4/tmp_10000000000001.png") couldn't be recognized by FileIO.
  
  Stacktrace:
   [1] #save#30(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(FileIO.save), ::FileIO.Formatted, ::Any) at /Users/abradley/.julia/packages/FileIO/Glbhe/src/loadsave.jl:196
   [2] save at /Users/abradley/.julia/packages/FileIO/Glbhe/src/loadsave.jl:196 [inlined]

It only happens for a particular figure in the book, which is a 2d image. Other figures that are just regular line plots work fine. So the offending command is

push!(g, Axis(Plots.Image(abs2.(f),(-1e-3,1e-3),(0.,3e-3),colormap=ColorMaps.Named("Blues"),colorbar=false),
		     width="5cm",height="10cm"))

I am using

julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 4

Any help greatly appreciated!

semicolon does not suppress output in juliaconsole

This:

\begin{juliaconsole}
rand(100);
\end{juliaconsole}

Prints 100 numbers in my latex document.

I noticed in the slides in your JuliaCon talk that a semicolon does suppress output. Any ideas what is going on here? I did find that this

\begin{juliaconsole}
rand(100); nothing
\end{juliaconsole}

Suppresses output. But, this is of course undesirable in a document.

Could not find style module 'algforopt'

On a fresh clone in MacOS Ventura, with julia 1.8.0:

make test gives: ALL TESTS PASSED.

But make compile gives

This is PythonTeX 0.18
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/texlive/20220321_3/libexec/lib/python3.10/site-packages/pygments/styles/__init__.py", line 82, in get_style_by_name
    mod = __import__('pygments.styles.' + mod, None, None, [cls])
ModuleNotFoundError: No module named 'pygments.styles.algforopt'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/Cellar/texlive/20220321_3/share/texmf-dist/scripts/pythontex/pythontex3.py", line 2403, in do_pygments
    formatter[codetype] = LatexFormatter(**p)
  File "/usr/local/Cellar/texlive/20220321_3/libexec/lib/python3.10/site-packages/pygments/formatters/latex.py", line 257, in __init__
    Formatter.__init__(self, **options)
  File "/usr/local/Cellar/texlive/20220321_3/libexec/lib/python3.10/site-packages/pygments/formatter.py", line 66, in __init__
    self.style = _lookup_style(options.get('style', 'default'))
  File "/usr/local/Cellar/texlive/20220321_3/libexec/lib/python3.10/site-packages/pygments/formatter.py", line 21, in _lookup_style
    return get_style_by_name(style)
  File "/usr/local/Cellar/texlive/20220321_3/libexec/lib/python3.10/site-packages/pygments/styles/__init__.py", line 84, in get_style_by_name
    raise ClassNotFound("Could not find style module %r" % mod +
pygments.util.ClassNotFound: Could not find style module 'algforopt'.
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/Cellar/texlive/20220321_3/share/texmf-dist/scripts/pythontex/pythontex3.py", line 2882, in <module>
    main(python=sys.version_info.major)
  File "/usr/local/Cellar/texlive/20220321_3/share/texmf-dist/scripts/pythontex/pythontex3.py", line 2828, in main
    do_multiprocessing(data, temp_data, old_data, engine_dict)
  File "/usr/local/Cellar/texlive/20220321_3/share/texmf-dist/scripts/pythontex/pythontex3.py", line 1390, in do_multiprocessing
    result = task.get()
  File "/usr/local/Cellar/[email protected]/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/pool.py", line 774, in get
    raise self._value
pygments.util.ClassNotFound: Could not find style module 'algforopt'.
make: [compile] Error 1 (ignored)

differences between `jlcode`, `jlblock` and `juliaconsole`?

Hi,

Thank you very much for providing such a powerful toolchain, I like the style of your book, and I'm planning to use this to write some tutorial materials for undergraduate students.

According to the README, I've successfully set the dependencies up. When I look into the template details, I find four environments defined:

• jlcode
• jlblock
• juliaconsole
• juliatest

I'm not sure if this is correct, but here's my understanding:

* codes in jlcode are used only once and don't affect the following codes, and jlblock do.
* jlblock and jlconsole only diff in the pygments colors.
* juliatest acts the same as juliaconsole except they're separated and fed into two pipelines.

Also, I don't quite get the difference between jlblock and juliaconsole.

Could you please explain the usage of these four environments?

Best

Johnny Chen

Compiling only chapters

you mentioned in the juliacon-talk that it is possible to compile chapters separately. How is this done?

Share between console and block?

Is it possible to define eg. a function in a jlblock and call that function in juliaconsole? There is a workaround proposed in gpoore/pythontex#55 can this be applied to the julia version?

working on Julia 1.2?

not sure if this is expected, but on Julia 1.2

gink:tufte_algorithms_book abradley$ julia install_pkgs.jl REQUIRE
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
  Updating `~/.julia/environments/v1.2/Project.toml`
  [92933f4c] + ProgressMeter v1.0.0
  Updating `~/.julia/environments/v1.2/Manifest.toml`
  [92933f4c] ↑ ProgressMeter v0.9.0 ⇒ v1.0.0
 Resolving package versions...
 Installed Suppressor ─ v0.1.1
  Updating `~/.julia/environments/v1.2/Project.toml`
  [fd094767] + Suppressor v0.1.1
  Updating `~/.julia/environments/v1.2/Manifest.toml`
  [fd094767] + Suppressor v0.1.1

[ Info: add and build julia 1.1
Progress:   9%|███▊                                     |  ETA: 0:00:06
[ Info: add and build Colors
ERROR: LoadError: MethodError: no method matching iterate(::Nothing)
Closest candidates are:
  iterate(!Matched::Core.SimpleVector) at essentials.jl:604
  iterate(!Matched::Core.SimpleVector, !Matched::Any) at essentials.jl:604
  iterate(!Matched::ExponentialBackOff) at error.jl:214
  ...
Stacktrace:
 [1] first(::Nothing) at ./abstractarray.jl:342
 [2] |>(::Nothing, ::typeof(first)) at ./operators.jl:854
 [3] macro expansion at /Users/abradley/Dropbox/tufte_algorithms_book/install_pkgs.jl:41 [inlined]
 [4] macro expansion at /Users/abradley/.julia/packages/ProgressMeter/NIpPa/src/ProgressMeter.jl:606 [inlined]
 [5] top-level scope at /Users/abradley/Dropbox/tufte_algorithms_book/install_pkgs.jl:30
 [6] include at ./boot.jl:328 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1094
 [8] include(::Module, ::String) at ./Base.jl:31
 [9] exec_options(::Base.JLOptions) at ./client.jl:295
 [10] _start() at ./client.jl:464
in expression starting at /Users/abradley/Dropbox/tufte_algorithms_book/install_pkgs.jl:29

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 4

Priviliges

is there any way to run the python scripts ( or get the things done they do ) without having to use sudo?
I don't have admin rights on my work machine.

make compile error

I had never used PythonTex before. make compile results in

 PythonTeX error
    The version of the PythonTeX scripts does not match the last code
    saved by the document--run LaTeX to create an updated version.

searching the web did not help. How would one fix this?

Additional Dependencies

I found that you also need to add Colors.