mastering-spark-sql-book's Introduction

The Internals of Spark SQL

The project contains the sources of The Internals of Spark SQL online book.

Tools

The project is based on or uses the following tools:

Apache Spark with Spark SQL
MkDocs which strives for being a fast, simple and downright gorgeous static site generator that's geared towards building project documentation
Material for MkDocs theme
Markdown
Visual Studio Code as a Markdown editor
Docker to run the Material for MkDocs (with plugins and extensions)
Read the Docs for online deployment

Previewing Book

Simply use the official Docker image to get up and running with the above tools.

Start mkdocs serve in the project root (the folder with mkdocs.yml) as follows:

$ docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material

Consult the MkDocs documentation to get started and learn how to build the project.

$ docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material --help
Usage: mkdocs [OPTIONS] COMMAND [ARGS]...

  MkDocs - Project documentation with Markdown.

Options:
  -V, --version  Show the version and exit.
  -q, --quiet    Silence warnings
  -v, --verbose  Enable verbose output
  -h, --help     Show this message and exit.

Commands:
  build      Build the MkDocs documentation
  gh-deploy  Deploy your documentation to GitHub Pages
  new        Create a new MkDocs project
  serve      Run the builtin development server

Use mkdocs build --clean to remove any stale files.

Consider --dirtyreload for a faster reloads.

$ docker run --rm -it \
    -p 8000:8000 \
    -v ${PWD}:/docs \
    squidfunk/mkdocs-material \
    serve --dirtyreload --verbose --dev-addr 0.0.0.0:8000