Coder Social home page Coder Social logo

camerondurham / cs350-docker Goto Github PK

View Code? Open in Web Editor NEW
28.0 3.0 6.0 354 KB

Docker container and run scripts for MIT's xv6 kernel. Useful for USC CSCI 350.

Shell 40.71% Dockerfile 26.57% Makefile 8.71% PowerShell 24.01%
docker docker-image operating-systems usc xv6 unix docker-desktop development-environment dockerfile mit-xv6

cs350-docker's Introduction

CSCI 350 Operating Systems Image


Intro

This repo contains a simple dev environment to do operating system development (at least for xv6). After pulling the Docker image from Docker Hub, you should be able to use the run.sh or run.ps1 script to start, stop, and work in a virtualized environment all from your command line.

The interface is based off of Noah Kim's and my csci104/docker.

For more information, please see the wiki. For any questions, bugs or problems with these instructions, please create an issue. To help me in maintaining this repo, issues help track problems people have so any knowledge is documented, shared, and easier to track. Thanks!

Supported Platforms

This Docker image has been tested and verified to be functioning correctly on the following platforms:

  • Catalina, Big Sur (Intel)
  • Big Sur, Monterey (Apple Silicon)
  • Windows 10 (Docker with WSL2 backend)
  • Debian Sid, Arch Linux

Recommended Editors

Getting Started (using ch)

Skip to the Getting Started (run scripts) section if you do not want to install ch.

If you have the container helper CLI (ch) installed already, this is the fastest way to start. See detailed instructions here to install ch. (full disclosure, this is a shameless plug since I wrote the CLI but I do at least dogfood my own tools!).

Benefits of installing this way is that you don't have to be in a specific directory to start the csci350 environment. Also, you won't have to clone this repo to get started since your main dependency is the Docker image with qemu and gdb installed.

Simplest solution to install ch (at least on macOS) is if you have homebrew. See the getting started notes to run the install script for Windows or Unix-based system.

brew tap camerondurham/tap
brew install camerondurham/tap/ch

1. Find the Filepath to Your Work Directory

Find the absolute path to your csci350 directory where you keep your homework.

See the Filepaths in terminal wiki from csci104/docker if you need some help with this.

On macOS/Linux:

Navigate to your directory in the terminal and run pwd, the output should be something like /Users/username/path/to/csci350

In Windows Powershell

Navigate to the directory in Powershell and run Get-Location, you will want the output like C:\Users\Username\path\to\csci350

2. Create the ch Environment

Use ch create to create and save the environment settings, replacing PATH_TO_YOUR_WORKDIR with the path from step 1.

You can copy-paste this below in macOS/Linux:

ch create csci350 \
    --image camerondurham/cs350-docker:v1 \
    --volume PATH_TO_YOUR_WORKDIR:/xv6_docker \
    --security-opt seccomp:unconfined \
    --port 7776:22 \
    --port 7777:7777 \
    --port 25000:25000 \
    --cap-add SYS_PTRACE \
    --shell /bin/bash \
    --privileged \
    --platform linux/amd64

Below is the single-line version for Windows. Please remember to replace PATH_TO_YOUR_WORKDIR wirh the path from part 1.

ch create csci350 --image camerondurham/cs350-docker:v1 --volume PATH_TO_YOUR_WORKDIR:/xv6_docker --security-opt seccomp:unconfined --port 7776:22 --port 7777:7777 --port 25000:25000 --cap-add SYS_PTRACE --shell /bin/bash --privileged --platform linux/amd64

NOTE: The command above requires ch version at least v0.3.7. Refer to install instructions if you see errors about non-existent --platform flag.

3. Start the Environment

ch shell csci350 --force-start

See the ch README Command section or ch --help for other commands.

Getting Started (run scripts)

Skip this section if you followed Getting Started (ch) instructions since ch will take care of starting/stopping/getting shells into your environment.

The instructions below will walk you through running the setup script and the run script you'll use to access the csci350 environment.

1. Set Up

  1. Install Docker desktop from the website

    • Windows Users Only: Docker provides better performance using WSL 2 than legacy Hyper-V. Set up WSL 2 using Microsoft's guide here. After you have WSL 2 running, refer to the screenshots at the end of this readme for settings you have to enable in Docker Desktop app.
    • Linux Users Only: Install Docker engine from here,
  2. Clone this repository.

  3. Specify your desired mount location (i.e. your xv6 project folder)

Windows users only

Make sure you run this in an Admin PowerShell to let you run scripts:

# must execute this in admin powershell and select [A] to run scripts
Set-ExecutionPolicy RemoteSigned

2. Running

  1. Modify the work variable at the top of the run script in the project folder. For example:

    macOS/Linux:

    # in run.sh
    work=~/projects/cs350/xv6-public-master/

    Windows Powershell:

    # in run.ps1
    $work="C:\Users\Username\cs350\xv6-public-master"

    Windows Terminal using WSL2:

    work=/mnt/c/Users/Username/cs350/xv6-public-master
  2. Run the run.sh/run.ps1 script. If this is your first time starting, this will pull the Docker image. This image will be cached until there's a new image available or you manually remove it.

    macOS/Linux/Windows Terminal (WSL2):

    ./run.sh start

    Windows Powershell:

    .\run.ps1 start

This script is only a wrapper for some simple Docker commands.

3. Working in the Environment

To start up a Linux shell inside the Docker image, you'll want to start a terminal session inside the Docker image:

macOS/Linux/Windows Terminal (WSL2):

./run.sh shell

Windows Powershell:

.\run.ps1 shell

If you're only using this as a build toolchain, you can run the following command to create xv6.img and fs.img for use in QEMU later. This assumes that the Makefile exists in the project directory.

macOS/Linux/Windows Terminal (WSL2):

./run.sh build

Windows Powershell:

.\run.ps1 build

4. Exiting the Environment

To quit qemu and return to Linux shell:

ctrl + a
x

Don't hit all three keys at the same time. Do ctrl + a then x

You may get better performance by only using the build toolchain in the container, and running QEMU natively after installing it through Homebrew or your package manager of choice. This is because newer versions of GCC no longer compile the bootloader correctly. You can also get QEMU to run graphically this way!

5. Stopping

After you're done working in the environment, you might want to stop the container. Don't worry if you forget to do this, since Docker Desktop will automatically and safely stop running containers when you shutdown your computer.

macOS/Linux/Windows Terminal (WSL2):

./run.sh stop

Windows Powershell:

.\run.ps1 stop

Demo

Here's what it looks like to interact with a setup environment:

asciicast

System Requirements

Below are the system requirements for Docker Desktop:

Windows host:

  • Windows 10 64-bit: (Build 15063 or later)
    • Pro, Enterprise, or Education: using Hyper-V and Containers Windows features
    • Any Windows 10 version: using WSL2 container backend (recommended)

If you are using Windows 10 Home, you can obtain a "free" license for Windows 10 Education here.

Mac host:

  • Mac hardware must be a 2010 or newer model
  • macOS must be version 10.13 or newer
  • 4 GB RAM minimum

Linux host:

  • Use the Docker-provided install instructions if it exists for your distro

Troubleshooting

xv6 will not start shell, hangs at qemu command

If you're having issues starting xv6, such as the system is hanging at qemu commands, try the following

  1. Outside the Docker shell, pull updated repo directory from xv6-public: git clone [email protected]:mit-pdos/xv6-public.git
  2. Ensure all .pl files are executable and re-build:
# run these commands in the docker shell
chmod +x *.pl
make clean
make qemu-nox

make qemu fails

Since this Docker image does not run a virtual display or window server, you cannot run make qemu. Instead, use make qemu-nox. Adding an X server would have minimal benefit, since you can simply use your current terminal window to debug your xv6 programs.

Makefile:104: recipe for target 'bootblock' failed

Run these commands

make clean
chmod 755 *.pl cuth dot-bochsrc printpcs runoff runoff1 show1 spinp
make qemu-nox

xv6 stuck on qemu-system-i386 macOS Apple Silicon

If your shell is stuck on qemu-system-i386 -nographic -drive ... after running make qemu-nox or returns an error rosetta error: Unimplemented syscall number 282, you may need to disable Rosetta emulation in Docker.

Uncheck the "Use Rosetta for x86/amd64 emulation on Apple Silicon" option in Docker Settings > General. (thank you to @WardahJabeen for this suggestion in #13!).

xv6 stuck on qemu-system-i386 Windows

If your shell is stuck at qemu-system-i386 -nographic -drive file=fs.img,index=1,media=disk,format=raw -drive file=xv6.img,index=0,media=disk,format=raw -smp 2 -m 512 after running make qemu-nox, it probably means you don't have WSL/WSL2 enabled properly.

You can check if you have WSL by checking here.

If you don't have WSL, you can follow the steps here.

Now that you have WSL and a linux distro installed, you can check if you have WSL2 enabled by runnning PowerShell in administration and run this command

wsl --list --verbose

You'll then see the name of your WSL, State, and Version (what you see may vary based on the distro you installed, this write up is using Ubuntu)

PS C:\WINDOWS\system32> wsl --list --verbose
  NAME                   STATE           VERSION
* Ubuntu                 Running         2
  docker-desktop         Running         2
  docker-desktop-data    Running         2

Here, you see a list of WSL running processes along with its state and WSL version.

If it shows 1 under version for any of the names, you can change it to 2 by entering this command in PowerShell in administration

wsl --set-version <process name> 2

You can change the default distro by running

wsl --setdefault <process name>

After it finishes changing, open your Docker Desktop and enable Use the WSL 2 based engine. Then navigate to Settings > Resources > WSL Integration, make sure you check Enable integration with my default WSL distro, and enable integrations with any of the additional distros you have. You'll have to restart your docker by doing this operation.

Docker Desktop Setting Screenshots

Docker Desktop General Settings Docker Desktop Resource Settings

cs350-docker's People

Contributors

a-harhar avatar camerondurham avatar h313 avatar markkyaw avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

cs350-docker's Issues

Make distinction between run script and `ch` instructions

Instructions for using run.{sh,ps1} shell scripts vs ch are unclear.

Describe the solution you'd like
Make clear distinction between run script and ch instructions. Ideally instructions should be in different headers even if it means a longer readme. IMHO having 2 sets of instructions under each step is confusing.

Example: unclear for new users whether they need to edit run.{sh,ps1} script if using ch.

ch create for csci350 errors out, but run_scripts workflow works for some people

Describe the bug

When running the ch command from cs350-docker to create the csci350 container, it produces the following error:

v1: Pulling from camerondurham/cs350-docker Cannot create new environment. Error from Dockerfile build: no matching manifest for linux/arm64/v8 in the manifest list entries

To Reproduce

Simply using the ch create command from cs350-docker causes this issue to appear. The issue states no manifest for linux/arm64/v8 -- I'm not entirely sure about this but I think it is attempting to use my host platform as the image's platform?

This issue does not occur when using the run_scripts workflow, which works completely fine!

Screenshots

Running with ./run.sh start:
image

Running with ch create:
image

Desktop (please complete the following information):

  • OS: MacOS v13.0.1
  • Latest Docker Version (just updated)

Additional context
While run_scripts workflow is working for me, some other students are complaining that it doesn't work for them either. They are running into the same issue as before:

./run.sh start

Running Docker image

[+] Running 0/1

โ ‡ gdbserver Pulling 2.9s

no matching manifest for linux/arm64/v8 in the manifest list entries

Publish to Docker Hub

  • Publish cs350-docker image on Docker Hub for public use
  • Add sample Dockerfile/docker-compose solution

Publish image to AWS ECR and Sign Image Build

Describe the solution you'd like
Consider pushing image to public ECR and monitoring/limiting pulls to avoid excess charges. See AWS ECR Free Tier.

This image should ideally be signed whether going to AWS or Docker Hub. Should look into this project: https://github.com/sigstore/cosign

Additional context
Not sure exactly why I want to do this besides learning how to write a gh workflow to do this without risking excess AWS charges. Should be really simple. Also, ECR should have less restrictive pull rate limiting (citation needed).

xv6 stuck on qemu-system-i386 Mac M1 and M2 Chip

It is the same error as "xv6 stuck on qemu-system-i386 Windows" error under Troubleshooting section causing make qemu-nox to fail but for Mac M1 and M2 chip.
Seems to be with the latest update of Mac OS. Also, it didn't occur on Mac M1 chip in previous semesters.
Error: qemu-system-i386 -nographic -drive file=fs.img,index=1,media=disk,format=raw -drive file=xv6.img,index=0,media=disk,format=raw -smp 2 -m 512 rosetta error: Unimplemented syscall number 282

Use specific version over Docker latest tag

Docker's :latest tag is unreliable and confusing. The tag is not actually implemented to mean the "most recent" image since it's just an ordinary tag. See this article which goes into much more detail about why :latest is an anti-pattern.

Should use tagged versioning via git like the camerondurham/ch repo's CI or use tags representing the base ubuntu version.

Fix Failed Workflow

Describe the bug

Failed workflow building and publishing image: https://github.com/camerondurham/cs350-docker/runs/3476950995?check_suite_focus=true

To Reproduce
Steps to reproduce the behavior:

  1. Build Docker image

Docker build fails with following error, need to run in Ubuntu environment to reproduce:

Error: buildx failed with: error: failed to solve: process "/bin/bash -o pipefail -c apt-get update && apt-get install -y --no-install-recommends     apt-utils=1.6.13     build-essential=12.4ubuntu1     cmake=3.10.2-1ubuntu2.18.04.1     g++=4:7.4.0-1ubuntu2.3     gcc=4:7.4.0-1ubuntu2.3     gcc-multilib=4:7.4.0-1ubuntu2.3     gdb=8.1.1-0ubuntu1     gdbserver=8.1.1-0ubuntu1     openssh-server=1:7.6p1-4ubuntu0.3     qemu=1:2.11+dfsg-1ubuntu7.36     qemu-system-common=1:2.11+dfsg-1ubuntu7.36     rsync=3.1.2-2.1ubuntu1.1     && apt-get autoclean -y     && apt-get autoremove -y     && rm -rf /var/lib/apt/lists/*     && mkdir /var/run/sshd     && echo 'root:root' | chpasswd     && sed -i 's/PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config     && sed 's@session\\s*required\\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd" did not complete successfully: exit code: 100

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.