zystemos / pluto Goto Github PK

Grub will pass a magic number and multiboot header address in eax and ebx respectively. Kmain should take these as parameters since they hold useful information (memory map etc.).

We'll need to support user mode so that some processes can be run in a less privileged state.

Depends on #35

A heap memory manager will need to be implemented so that dynamic sizes of memory can be allocated and freed. One that I have implemented before uses a list of 'holes', each one having a header stating if it is occupied and what its size is. The allocator then goes through this list of holes and finds one with a size closest to the desired size. If it's too big then it splits the surplus off into a separate hole.

Of course there are many ways this can be done.

It would be useful to implement a system for testing the kernel at runtime. This would be used to test components of the kernel, such as paging, multitasking and the filesystem.

A possible approach to running a certain testcase would be to boot the kernel in testing mode (perhaps with a build specific for that testcase), have it initialise the component being tested and printing to serial with some predetermined format based on success or failure. A program could then be attached to the kernel's serial port ensuring that the correct output is received.

The results could then be compiled into a testing report.

Possible dependency on #37 if serial output is how we monitor test results.

Have the ability to test input values from called to arch specific functions like outb. There should be an init function to set up the testing for arch_mock which the user will provide a list of values to be expected. Then each call to the mocked arch function will test the parameters in order. The user will need to know how many calls will be made and the number of parameters that will be expected to be passed. And more or less should fail the test. Any parameters that don't match should fail the test.

The architecture-specific code needs to be separated from the architecture-agnostic code. Arch-specific code will be placed within its own directory under the arch directory. E.g.:

arch/
    x86/
        init.zig
    x86_64
        init.zig

etc.

At build time the arch file for the arch being built will then be built to an object and linked with the kmain object, forming the main executable.

This will involve interfacing with the cmos chip, so would be nice to also add a interface to abstract he cmos functionality. Then add the current date and time to the screen under the logo in the center (this is how I have it in DeanOS).

Like for the PIC where the IRQ have named values, do the same for the ISR. This will also need to update all the magic number in all files, e.g. in paging.

ziglang/zig#2627 is preventing us from implementing Directory and DirectoryEntry as packed structs due to byteoffset and size bugs.

The help text for the zig build debug option is confusing, since it says that it causes the kernel to be built with debug info but this happens irrespective of the option's value. As it stands, our options are

• The option should be removed (meaning zig build run will have to be branched out to zig build run-debug).
• It should be made true by default.
• It should be made false by default.

For option 2 and 3, a value of false will mean we have to strip the debug info from the build with --strip or the equivalent build.zig function.

I didn't get 4KB paging working with my original effort (I seem to be cursed).

The kernel should be moved to the higher half to better facilitate user processes in the future. This can be done by putting the initial boot code at 1MiB, setting up paging and jumping to code addressed at 3 GB + 1 MiB with a rudimentary page directory.

This will make the code clean and consistent. Once a style is decided 2 things need to be done:

• A checker should be added to CI with a failure returned if it isn't up to standard.

• A linter script should be added to the repository so any developer can check a commit before pushing with a mistake.

We'll need a very simple memory allocator that will allocate linearly between given a start and end address, and that doesn't need to care about freeing. This allocator will be used to allocate memory needed before a more sophisticated heap is set up.

Once ziglang/zig#313 is done

We're currently using Zig's normal executable building procedure with zig build-exe but this doesn't use our build.zig file. I propose that we start using zig build [steps] instead and revamp build.zig. An idea for a build/run workflow is:

1. Build the kernel elf image with zig build
2. Run the kernel with qemu with zig build run, zig build run -Ddebug=true to make it wait for a debugger connection
3. Launch a debugger instance with zig build debug

This could also coincide with the effort to make adding -curses to the qemu invocation unnecessary.

Once #59 is done, we can add some extensive runtime tests for x86 that we wouldn't want to run in a normal build.

• Syscalls
• Paging
• IRQs and ISRs
• PIC
• PIT

This work could be split as the person who implemented the features above is probably best suited to testing it.

Depends on #59

We'll need to setup segmentation as it's pretty much required by other features of the architecture. It can follow a flat model where the user and kernel segments overlap entirely, leaving permissions and access management to paging.

The readme could do with some improvement. It will need some more extensive building and running instructions (all the dependencies, possible options to pass to zig build, exact zig version required etc.) and a list of the project goals. Contribution guidelines would also be good, including the style guide decided upon in #61

It would be a good idea to set up continuous integration for the kernel, which would make sure that the codebase remains in a stable state. It could be used to ensure that each pull request can be built in a number of configurations (architecture etc.) and that all tests pass.

Criteria for a CI service for us to use (in order of priority) are:

1. Free: We're probably not in a situation to be able to pay for CI
2. Flexible configuration: Kernel development is a rather niche area and so not many CI services will support it out of the box

Relates to #12

Add the programmable interrupt timer. This then can be used with a scheduler.

Have the ability to read in the keys pressed and handle the control characters, e.g. shift... Also provide an interface to abstract all the functionality of a keyboard.

The syscall handler type could be refactored to add a list of 5 arguments. This would mean each individual handler wouldn't have to extract the arguments from the context itself, and would protect against syscall ABI changes.

From fn (ctx: *arch.InterruptContext) u32 to fn (ctx: *arch.InterruptContext, arg1: u32, arg2: u32, arg3: u32, arg4: u32, arg5: u32) u32

The handler would then be called with

handler(ctx, syscallArg(ctx, 0), syscallArg(ctx, 1), syscallArg(ctx, 2), syscallArg(ctx, 3), syscallArg(ctx, 4));

Create functions that interface with the PIC controllers so abstract the functionalist of the PIC

Here is one example:
https://andrewkelley.me/post/zig-stack-traces-kernel-panic-bare-bones-os.html

When looking at the x86 boot code in pluto.elf with objdump you can see a lot of NOPs that seem to have no use whatsoever.

$ i686-elf-objdump -d bin/iso/boot/pluto.elf 
00102000 <_start>:
  102000:       b9 00 10 10 00          mov    $0x101000,%ecx
  102005:       0f 22 d9                mov    %ecx,%cr3
  102008:       0f 20 e1                mov    %cr4,%ecx
  10200b:       83 c9 10                or     $0x10,%ecx
  10200e:       0f 22 e1                mov    %ecx,%cr4
  102011:       0f 20 c1                mov    %cr0,%ecx
  102014:       81 c9 00 00 00 80       or     $0x80000000,%ecx
  10201a:       0f 22 c1                mov    %ecx,%cr0
  10201d:       e9 de df ef bf          jmp    c0000000 <KERNEL_VADDR_START>
  102022:       90                      nop
  102023:       90                      nop
  102024:       90                      nop
  102025:       90                      nop
  102026:       90                      nop
  102027:       90                      nop
  102028:       90                      nop
  102029:       90                      nop
  10202a:       90                      nop
  10202b:       90                      nop
  10202c:       90                      nop
  10202d:       90                      nop
  10202e:       90                      nop
  10202f:       90                      nop
  102030:       eb fe                   jmp    102030 <_start+0x30>

An investigation will be needed to determine if this is a build procedure issue or compiler issue.

zig#2616 is preventing us from using a build of zig master so the newly added alignment functions will need to be removed once we can use zig master.

Compiling the architecture-specific code to an object andl inking it with the main object is becoming cumbersome and presenting problems when implementing the x86 GDT and IDT. This issue tracks the work on redesigning the compilation and source structure so that only the minimal boot code is compiled to an object and all other architecture symbols (init, outb etc.) are imported in the normal zig way.

The symbol file path given to the gdb command is incorrect. We need to pass the path to the elf file in the same string as the "symbol-file" argument.

Manage the physical memory.

• Some areas of physical memory are unusable and will need to be managed so not to use this memory.
• Manage the memory mapped I/O. May this should be a separate manager, but will have ties with the physical memory manager as shouldn't allocate memory here.
• Biggest of all, manage where everything is, as data and code have to be somewhere in memory.

Also, we can decide with the 32 bit OS, handle 4KB chunks where for 64 bit OS handle 4MB chunks, as who has less than 4GB of RAM these days, am I right ;).

Here is an example written in C:
https://github.com/DrDeano/DeanOS/blob/master/src/kernel/kernel/pmm.c

A good tutorial:
http://www.brokenthorn.com/Resources/OSDev17.html

We need a basic bootable kernel for x86. This can be imported from my previous kernel but with the relevant conversions to Zig (Andre Kelley's video on this will be helpful when it comes to the language's intricacies).

This work will include making a master Makefile that will also make the chosen architecture's bootloader.

A discussion point is whether or not the kernel should be in the higher half (with addresses starting at 3GiB + 1MiB) from the very start. I think it should as it makes mapping the kernel into user processes easier, and means you don't have to relocate an executable's code up to the higher half later on.

The kernel will need syscalls so that programs can interact with the kernel. We could choose the same interrupt number as linux, which would be 0x80. The number used to select which syscall is desired could be placed in eax, and the kernel will perform the appropriate action for that number. Parameters should be passed according to the architecture's ABI, so for x86 we'd follow the cdecl ABI.

Depends on #51

The IDT is needed to handle exceptions and interrupts. Each IDT entry should be associated with a default handler which should check if a higher-level handler has been registered and if so call it, else continue on. This higher-level handler system should be separate from the IDT system.

An example flow:

Interrupt -> Handler in IDT (Stores state on stack, calls default handler) -> Default handler (Checks list/array of pointers to handler functions) -> Specialised handler (Does its stuff) -> Default handler -> Handler in IDT (restores state from stack etc.)

Depends on #14 and #5

It would be nice to at some point use the super-duper bootloader written by @DrDeano . Perhaps with the stretch goal of migrating some of it to Zig if we can (probably a separate issue).

The latest version of zig brings some bug fixes and improvements that we'll want when building pluto. This will require some syntactic changes.

Paging needs to be implemented to map the higher half kernel to physical memory. This will also require a simple watermark memory manager that takes the end of the loaded kernel code and allocates memory on a linear basis after that (this may require a separate ticket).

4KB paging will be looked at later on.

Depends on #6, #22 and #19

We'll need a rudimentary cooperative multitasking system that presents an interface to a program, allowing it to have its state stored, relinquish control to another task. When control is returned to the task in the future, its state should be restored.

Control could be relinquished by sending an interrupt since the interrupt handling system already does a lot of the work of saving the CPU state, and this would lay the foundation for a more advanced multitasking and scheduling system, where all task transitions occur via syscalls and timer interrupts.

Preemptive scheduling with a timer interrupt should probably be done in a separate issue.

Depends on #36 if interrupts are the only way for a program to relinquish control.

Logging to serial. Should support multiple levels (debug, warning, info and error). An example log message is [DEBUG] This is the message.

The logging interface should provide a function for each level which all call a generic function, and the generic function itself.

Options can be passed from build.zig to any zig source file. This can be used for configuration and creating builds for a specific purpose, such as running extensive runtime tests.

The option is added by calling addBuildOption(...) on a LibObjExeStep (e.g. the result from addObject(...), then used by importing zig-cache/{object name}_build_options.zig.

This is pretty much done with @SamTebbs33 paging. But once the physical memory manager is complete, can integrate this into the paging, so can allocate pages into physical memory.

• #67
• #8

Once this is done, this can be integrated into the heap memory manager.

The test step in build.zig currently tests all build modes. Let's change it to only test the one supplied by the user but test all in CI.

It would be nice if we could replace the x86 interrupt assembly (irq_asm.s and isr_asm.s) with zig code. This should be possible using naked functions and inline assembly, so we precisely control what happens. The interrupt handler functions could be generated with a pattern similar to this, but something simpler would be nice (perhaps just write them out manually?).

There is a lot of comments in the code of how it works

Instead of the existing system of importing the arch.zig file, have the architecture arch.zig file be added as a package in the build script. This will remove the kernel level arch.zig file. This will also help with the testing.

Is blocked by:
ziglang/zig#855

We can't do this as we have the kernel as a package and as the kernel will import the arch package and arch will import the kernel package, will need to wait for this.

Once we have the foundations of multitasking done (#35), we'll need to add some sort of scheduler that will preempt and resume processes on a timeslice basis. The implementation and scheduling policy is up to the implementer.

Depends on #35

When building the kernel, the entire build process is gone through even if no source files have changed so some investigation is needed. If this turns out to be an issue with zig and it is supposed to only build changed files then a bug report should opened. If we've done something wrong then the required changes should be made.

Only have the the IRQ handles be registered once has this could be a security risk as malicious program could re-register a IRQ handle

The arch interface should present a function that registers an interrupt handler so that arch-agnostic features (like system calls) can be implemented in an arch-agnostic way.

Serial output can be used for logging purposes as we can attach it to stdio from qemu and print debug messages that we may not want to print to the tty. Serial input is out of scope for this issue.