quarknerd / blog

The Scott Logic blog

Scott Logic Blogs

See below for technical details of the blog creation stack and, e.g., instructions on how to install and run the blog site locally.

Note that if you're looking to author a blog post, then you don't need to read any further! Instead, please see our company extranet for instructions on how to create a new page and view it before publication on the blog.

Technical Stack

The blog is a static website, designed to be hosted on GitHub pages.

The underlying content is generated through a series of Ruby gems and libraries, starting with a dedicated github-pages gem.

Within that stack, Jekyll is used as the static content generation engine, consuming template files written in either HTML or Markdown (syntax extended by Kramdown).

Common content or structure can be further injected or managed using the Liquid templating language.

Cloning the repository

Sparse checkout requires Git 2.25.0

Ensure that that your SSH configuration will also let you connect to private GitHub repositories.

If you wish to develop changes to the blog locally, you may find that there's a lot of content, and prefer just to download the bits you need.

# see comment above about configuring SSH, and modify the clone URL accordingly to use the correct SSH identity
# you may also consider forking the blog repository, and cloning your personal fork instead
git clone --depth 1 --filter=blob:none --no-checkout [email protected]:ScottLogic/blog.git
cd blog
git sparse-checkout init --cone
# if you want to write blog posts, modify this variable with the author name you
# wish to write posts under (typically derived from your SL email address)
AUTHOR='abirch'
git sparse-checkout set _includes _layouts _data category scripts scss assets "$AUTHOR"
git checkout gh-pages

This gets the repository down to ~8MB and ~150 files (whereas checking out all authors' posts would take hundreds of megabytes).

Run local copy of blog (for blog devs only)

NOTE: Instructions are work in progress.

If you plan to use Docker, then you can skip ahead now!

The blog consists of static HTML pages with content generated using:

• github-pages for deployment hooks
• Jekyll for static site generation generator
• Kramdown for an extended markdown syntax
• Liquid for templating functionality
• Nokogiri for efficient XML and HTML handling, relying on:
  • Native XML Parsers
• Bundler to manage gems and dependencies
• Ruby.

In theory, once you've installed Ruby and Bundler, given that the project contains a valid Gemfile, then using Bundler should bring in most of the dependencies automatically. However, due to Nokogiri's reliance on Native XML parsers you may require additional steps. Thorough instructions for setting up your development environment are detailed below.

Native environment

Prerequisites

First, install Ruby and (if on Linux) a few build dependencies for Nokogiri.

On Linux:

sudo apt-get install ruby2.3 ruby2.3-dev build-essential dh-autoreconf libxslt-dev libxml2-dev zlib1g-dev

On Windows, if you use Chocolatey, simply run choco install ruby in a PowerShell instance with elevated priveleges. If you don't use Chocolatey, you can use RubyInstaller or see the Ruby website for alternative ways to install Ruby. You don't need to install any other dependencies on Windows at this stage.

Secondly, update Gem and install the Jekyll, Bundler and Nokogiri gems.

On Linux:

sudo gem update
sudo gem install jekyll bundler nokogiri

On Windows, in a PowerShell instance with elevated priveleges:

gem update
gem install jekyll bundler nokogiri

Thirdly, configure Bundler to store project dependencies in vendor/bundle, and, when in the root directory of your clone of the blog, install the project dependencies.

bundle config path vendor/bundle
cd PATH/TO/BLOG
bundle install

Finally, run jekyll -v to check whether Jekyll is working. If so, you're good to run the blog!

Running in the native environment

Once you've got all the prerequisites for your operating system, you can run the blog. Navigate to the root directory of your clone of the blog and execute Jekyll using Bundler.

bundle exec jekyll serve

The blog will then be available on localhost.

If you need to re-compile the scripts or SCSS, you can use the NPM scripts.

npm install
npm run scripts
npm run style

Docker

Use a bash-compatible shell.

Install gem dependencies

First, output gem dependencies to a directory container_gem_cache on our host machine:

./shell/docker-gem-install.sh

Run dev watch

Now we can serve the blog:

BLOG_USERNAME=abirch ./shell/docker-dev-watch.sh

Visit the blog on localhost.

CI/CD

We use GitHub Actions for CI/CD. The workflow definitions are in YAML files in .github/workflows.

Compress Images Once a Month

Uses the calibreapp/image-actions Action to automatically compress images. The compression algorithm is near-lossless. It compresses all images in the repo once per month, and creates a Pull Request to merge in the resulting changes.

Check accessibility of changed content

Runs pa11y-ci with the aXe test runner to detect some common accessibility problems. It serves the blog locally and runs the tests on the rendered webpages. It only checks pages and blog posts which have changed, but doesn’t take any interest in changes to layouts or includes, so changes to those should be tested across the whole site separately. This workflow runs on Pull Requests, pushes to gh-pages and on manual dispatches.

Generate Read More related

Generates Read More links on blog pages across the blog, using the OpenAI API to determine which blog posts are on similar themes. This workflow runs only on manual dispatches on the gh-pages branch and creates a Pull Request to merge in the resulting changes.

Remove Unused Images

For each image in the repo, searches all the blog posts, pages, YAML data files and JavaScript scripts for any occurrences of its filename. If the filename occurs nowhere, deletes the image. Then makes a Pull Request to merge in its changes. This workflow runs only on a manual dispatch on the gh-pages branch.