This single-day free webinar provides an introduction to the command line interface (CLI) and a few tools for digital preservation with a focus on audiovisual assets. It's hosted by the Smithsonian Library and Archives (SLA) Audiovisual Preservation Initiative (AVMPI) and Audiovisual Archivists Interest Lunch (AVAIL).
The presentation slides can be accessed at this link
The test files can be accessed at this link
With the ever-increasing scale and complexity of digital archival collections, archivists need to adapt their tools and workflows; while every institutional context is different, there are often opportunities to employ open source and command line tools to meet these challenges. Among the many benefits of utilizing the command line, the two most immediate tend to be: increased reliability of processes and more interesting work for archivists.
This 3-hour workshop will cover the basics of the command line interface (CLI) with a focus on its use in audiovisual archival workflows and digital preservation. The command line software discussed will help archivists navigate their terminals, find/ move/ rename digital objects, understand checksums and CRCs and introduce principles of scripting and automation for handling file data at scale.
This is an introductory course and users with no programming or command line experience are welcome; any archivist who routinely moves files, verifies metadata across systems, or works with audiovisual materials will learn techniques to improve their efficiency and gain familiarity with systems and workflows which take advantage of CLI capabilities. Users don't need to have administrative privileges or the ability to install software on their local machines in order to participate. For users who can install software on their machines, there will be office hours prior to the workshop to go over any questions that arise as part of the setup (and instructions are provided at this link).
what the command line is, what we can do with it, why it matters
basic structures, navigation, inputs & outputs
moving files, verifying integrity of files and metadata
workflow setup, loops
While not an exhaustive list, the webinar will touch on some fundamental software to managing digital files, with an emphasis on audiovisual formats.
These tools include:
Programming is Forgetting: Toward a New Hacker Ethic by Allison Parrish
Heroes in a Bash Shell by Command Line Heroes
The Bash Parser - what happens after you press enter on a command in the terminal
Script Ahoy - a bash helper for archivists by Dianne Dietrich and Jarret Drake
Bash for Archivists - an intro course by Reto Kromer
Man Pages - the Wikipedia page for "man pages" aka manual pages, describing the history and use of manual pages in the command line
You will need a text editor - text editors are different than Microsoft Word. You'll be editing .txt and .sh files. On Windows, the default editor is Notepad, which will work fine. There's also Notepad++. On Mac, the default text editor is TextEdit, also fine.
I strongly recommend looking into the Visual Studio Code (VSCode) text editor, which is available on all platforms. It's a bit much for what we're doing today but it's the most popular text editor out there and can be customized to fit your needs.
I will mostly be using a terminal-based text editor (i.e. a text editor that opens within the terminal) named Vim, which is probably already installed on your system. Vim is legendarily difficult to use, though, and I don't really recommend it for beginners. Still, if you want to dive into the deepest deep end, it's there.
Macs ship with an application named “Terminal” which is their default command line interface. It will work great for this webinar
To install the tools/ software for this webinar, we will be using something called a “package manager.” Package managers help people manage their software without the use of “installers” like you might be used to, they’re very common on Linux and in programmer spaces, generally.
For Macs, the packagae manager is named Homebrew and, once you have installed it, proceed to the tools install section
If you have administrative access to your machine, you can install Homebrew with the steps on their website (it’s a Terminal command).
If you don't have admin access on your Mac, you can still install Homebrew and the other tools for this workshop by using the below steps.
- Open Terminal
- type
Cmd + Spaceto search your mac - type
terminaland press enter
- In Terminal, navigate to your home folder
- type
cd ~/and press enter
- install Homebrew in your home folder
- type the below and press enter
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"and press enter
- Once that command completes, go to Terminal -> Preferences -> Shell
- In the "Startup" Section, check the box to "Run Command"
- type this command into the box
export PATH=$PATH:${HOME}/brew/bin - Check the box to "Run inside shell"
With either install method, you can check if your install was successful by opening Terminal and typing brew help - if you get help output, you're good; if you get an error, email me.
This workshop is based on the BASH programming language/ shell, which, until recently, was not available by default on Windows as it was on Mac. Windows used a proprietary command-line interface referred to as cmd.exe, originally released in 1993. It’s like bash in its operation but there are many, many syntactical and technical differences. Because it's Windows, and they can't do anything just one way, there's another application called PowerShell which is a bit more Bash-like. While I have experience scripting on Windows, those skills are not very sharp at this time and I just don’t think I can support you at the level I’d need to in this workshop.
That being said, you are welcome to follow along in CMD/ PowerShell and make the modifications necessary, if you’d like.
Otherwise, we’re going to install Bash on your Windows machine, through the Windows subsystem for Linux (WSL). WSL is like having Linux installed, except without the headache of partitioning discs or creating bootable disc images - it’s Linux running as a Microsoft Windows application. It’s an official Windows software, so you can trust it at the same level you trust anything from them (😉)
With WSL installed, we will then use the Ubuntu Linux package manager, named apt, to install software.
Once you have installed WSL, proceed to the tools install section
Use Windows Search (hit the Windows key on your keyboard), type wsl and hit enter. You should see a box pop up, this is cmd.exe starting WSL (sorry for the alphabet soup).
In a few seconds, you should see a prompt that looks like this brnco@CHM3083:/mnt/c/WINDOWS/system32$ but with your login info. If you see that, great, you have WSL up and running! Otherwise, see below for official Microsoft documentation.
To install WSL, follow the instructions on the Microsoft website
Here is a great video describing the steps, as well
This section assumes that you have completed the above steps to install Homebrew/ WSL
This software is installed entirely via the command line. Each command is written on its own line and will appear formatted like this. Type the command in exactly as you see it here and press enter after each command.
You may need to link your package manager to the repository which hosts MediaArea's software releases. To do so, type the following command into your temrinal:
wget https://mediaarea.net/repo/deb/repo-mediaarea_1.0-24_all.deb && dpkg -i repo-mediaarea_1.0-24_all.deb && apt-get update
that command may need to be run as sudo: sudo su - hit Enter, run the wget command above, and type exit to exit super user mode (you'll learn more about this during the webinar!)
MediaInfo displays structured technical metadata from a variety of audiovisual media filetypes, everything from sample rates to listing subtitle tracks
brew install mediainfo
sudo apt install mediainfo
BWF MetaEdit displays and embeds metadata into a WAVE file's headers, per the Broadcast WAVE specification
brew install bwfmetaedit
sudo apt install bwfmetaedit
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent