The senior-project from jadams41

This is the repository containing my work on Senior Project at Cal Poly.

Project Information

Author: J. Ethan Adams ([email protected], [email protected])

Project Advisor: Dr. John Bellardo (http://users.csc.calpoly.edu/~bellardo/index.php)

Project Background

This project is a fork of my csc454 project where I developed much of the basic functionality of a simple x86_64 Operating System. Work for senior project will pick up where I left off in this class (to see a description of my previous work, see https://github.com/jadams41/Senior-Project/blob/master/archived/CPE454_README.md)

Project Scope and Overview

The goal of this to develop a custom, functional x86_64 unix-like Operating System.

Project Objectives

OS includes robust implementation of all functionality outlined in the project for csc454.
Fully implemented and functional networking stack (capable of communicating across layer-3 networks via TCP/IP).
OS can be built for physical hardware targets. All implemented OS features will be supported and verifiably functional for each supported hardware target.

NOTE: more objectives will likely be added once Objective 2 has been accomplished.

Current Functionality

Kernel Boot Procedure (assembly):
1. Create stack and store in esp register
2. Test for Multiboot-compliant bootloader and store Multiboot Information Structure pointer
3. Check for CPUID and Long Mode availability
4. Create initial page tables and enable paging
5. Enter Long Mode
6. Call kmain with stored multiboot information
Initailize VGA buffer and basic VGA output capability
Initialize PS/2 driver to handle keyboard input
Enable Interrupts
1. Initialize Programmable Interrupt Controller and enable cascading between master and slave PICs. Mask all interrupts.
2. Initialize idt and load with lidt.
3. Initialize ist(most interrupts use the general interrupt stack while GPF, DF, PF, and Syscalls are given unique interrupt stacks).
4. Unmask and install ISRs for PS/2, Serial, DF, ATA, GPF, PF, and Syscall interrupts.
Configure Serial Communication Driver (connected to host machine for debugging purposes). All following kernel output will be printed to VGA buffer and sent to host.
Process Multiboot tags and extract relevant information (memory map and elf sybmol tags).
Create new 4 level page table and initialize set up virtual memory (and dynamic memory allocation).
1. Kernel Heap is initalized.
2. Allocated virtual memory is paged "on demand" (no physical page used for an allocated virtual page until memory is accessed and page fault is raised).
Probe for ATA devices and initialize if found.
1. Probe for filesystems on each found device and represent each as "Virtual Filesystem".
Probe for PCI devices and initialize if found.
1. Currently only focused on initializing rtl8139 NIC
Initialize found Network Interface Controller.
Process and modify relevant information stored in PCI configuration space.

Demos of Current Functionality:

Receiving Network traffic

Receiving and Parsing ARP packets

TODO: Add demo of this functionality

Received ARP Packet Handling Functionality

TODO: Add demo of this functionality

Project Planning

Milestones

Milestone 0: Set up Senior Project and Document Project Scope

Take care of administrative tasks for setting up Senior Project
- Create project proposal documenting scope and objectives.
- Procure project advisor.
- Submit Pre-Enrollment Proposal to officially enroll in Senior Project I.
Set up Project
- Create GitHub Repository for Documenting project information
- Populate project's README with initial information regarding project background, scope & objectives, and tentative timeline of critical work.

Milestone 1: Necessary Maintenance and Preparations for New Development

Revise previous work.
- Review all of previous work - make sure functionality and limitations are understood and documented.
- Fix identified erroneous/limited implementation.
Create a more robust and stable development environment.
- Review and document functionality of current build system (make).
- Migrate build of project (operating system) to use bazel. Ignoring this for now
- Document how to build and run project with new build system.
- Research, implement, and document project testing strategy.
- Explore possibility of migrating project from C to C++.
Complete all cpe454 functionality milestones
- Review and document initial project's accomplishment of milestones.
- Fix all partially implemented functionality milestones.
- Implement remaining non-attempted milestones.

Milestone 2: Implementation of Network Stack

Develop Peripheral Component Interconnect (PCI) driver (Needed to discover, configure, and interact with Network Interface Card)
- Research and document relevant PCI functionality and control.
  - Implement basic PCI driver (capable of reading from and writing to PCI configuration space).
  - Implement pci_probe (discovery and handling of all devices connected to PCI bus):
    - Detect all devices connected to the PCI bus.
    - Develop super structure to represent PCI device (PCIDevice) with child structures to handle different types of PCI devices. Access to PCI control should be handled through function pointers set in super PCIDevice structure.
    - Integrate functionality: discover all connected devices, store relevant information in PCIDevices, return list of discovered PCIDevices.
Implement Network Interface Card (NIC) driver
- Research and document different NIC models.
  - Choose initial NIC to develop driver for and document rationale.
  - Implement primitive driver functionality:
    - NIC initalization (utilizing pci control in PCIDevice struct returned from pci_probe)
    - Basic implementation of asynchronous NIC control (transmitting and receiving packets).
    - Enable NIC's interrupts and install ISRs which use asynchronous NIC control functionality.
  - Test basic NIC functionality (transmit and receive ethernet frames):
    - Research and document strategies for networking between qemu VM's and host machine.
    - TEST: Guest receives ethernet frame from host VM:
      - Verify that interrupt fires after frame is sent from host -> VM
      - Verify that frame's header information and data can be properly extracted in VM.
    - TEST: Guest VM sends ethernet frame to host:
      - Verify that ethernet frame sent from guest is received on the host VM.
      - Verify that frame's header information and data are consistent (when received on host).
Build Network Stack
- Link Layer:
  - Ethernet
    - Discover and store interface's MAC Address
    - Receive and parse Ethernet frames (test with arping verify with wireshark running on bridge interface)
    - Create and send Ethernet frames (test by sending broadcast frame, verify packet validity with wireshark)
  - ARP: Address Resolution Protocol
    - Create ARP infrastructure
      - Structs for representing/extracting info/creating ARP packets
      - Create primitive ARP table for keeping track of received ARP traffic
    - Test ARP Infrastructure
      - Receive and correctly parse ARP traffic (test with arping -I <bridge_iface> -b <ip_on_bridge_subnet> and verify with wireshark running on bridge interface connected to os)
    - Send test ARP traffic (both ARP_REQUEST and ARP_REPLY)
      - Verify that sent packets valid (by running wireshark on bridge interface connected to the os).
      - ARP_REQUEST the bridge interface's IP address and make sure ARP_REPLY is received from bridge interface.
    - Flush out desired kernel ARP functionality:
      - Update arp_table when received ARP traffic contains new/updated ipv4->mac_addr mapping.
      - Handle all received ARP traffic automatically:
      - Upon receiving ARP_REQUEST for os's (mock) ipv4, create and send correct ARP_REPLY with our information. Verify by:
        
        arping for os's mock ip
        
        ensuring that the os receives/handles the ARP_REQUEST and creates+sends ARP_REPLY back.
        
        Seeing the reply is received by arping
        
        Checking host's arp table (using arp) and checking for entry: os_mock_ip -> os_mac_addr
- Internet Layer
  - IPv4: Internet Protocol Version 4
    - Create Internet Protocol infrastructure:
      - Create more robust network device infrastructure and data structures (like Linux's net_device) for configuring/representing/storing IP information.
      - Structs+Functions for representing/parsing/creating IP packets.
    - Send and receive/parse mock IP traffic (maybe use ICMP or UDP "traffic" for this).
    - Figure out what kernel data structures are needed for IP layer
  - ICMP: Internet Control Message Protocol
    - Create ICMP infrastructure:
      - OS recognizes and handles ICMP packets (probably only need control messages associated with ping).
      - OS can send relevant ICMP requests (probably only need to support control messages associated with ping).
    - Flush out desired kernel ICMP functionality:
      - OS automatically handles all incoming ICMP traffic
        
        OS extracts and uses relevant information from received ICMP traffic upon arrival.
        
        OS automatically replies appropriately to all received ICMP requests (that are supported).
      - "Userspace" accessible ping functionality (can probably be done in a kernel thread):
        
        As similar as possible to running ping on linux command line
        
        Outputs same way as linux ping utility (to make testing/validating easier).
- Transport Layer
  - UDP: Universal Datagram Protocol
  - TCP: Transmission Control Protocol
- Application Layer
  - DHCP: Dynamic Host Configuration Protocol
    - NOTE: rush to get here ASAP (build only what is needed in order to get DHCP DISCOVER->REQUEST functionality working in order to support dynamic IP addressineg of the OS).
  - HTTP: HyperText Transmission Protocol
    - NOTE: Final objective for Milestone 3

Potential Future Milestones

Implement better graphics support.
Improve process control capabilities.
Implement disk-encryption support.

jadams41 / senior-project Goto Github PK

senior-project's Introduction