zystemos / pluto Goto Github PK
View Code? Open in Web Editor NEWAn x86 kernel written in Zig
An x86 kernel written in Zig
Grub will pass a magic number and multiboot header address in eax
and ebx
respectively. Kmain should take these as parameters since they hold useful information (memory map etc.).
We'll need to support user mode so that some processes can be run in a less privileged state.
Depends on #35
A heap memory manager will need to be implemented so that dynamic sizes of memory can be allocated and freed. One that I have implemented before uses a list of 'holes', each one having a header stating if it is occupied and what its size is. The allocator then goes through this list of holes and finds one with a size closest to the desired size. If it's too big then it splits the surplus off into a separate hole.
Of course there are many ways this can be done.
It would be useful to implement a system for testing the kernel at runtime. This would be used to test components of the kernel, such as paging, multitasking and the filesystem.
A possible approach to running a certain testcase would be to boot the kernel in testing mode (perhaps with a build specific for that testcase), have it initialise the component being tested and printing to serial with some predetermined format based on success or failure. A program could then be attached to the kernel's serial port ensuring that the correct output is received.
The results could then be compiled into a testing report.
Possible dependency on #37 if serial output is how we monitor test results.
Have the ability to test input values from called to arch specific functions like outb. There should be an init function to set up the testing for arch_mock which the user will provide a list of values to be expected. Then each call to the mocked arch function will test the parameters in order. The user will need to know how many calls will be made and the number of parameters that will be expected to be passed. And more or less should fail the test. Any parameters that don't match should fail the test.
The architecture-specific code needs to be separated from the architecture-agnostic code. Arch-specific code will be placed within its own directory under the arch
directory. E.g.:
arch/
x86/
init.zig
x86_64
init.zig
etc.
At build time the arch file for the arch being built will then be built to an object and linked with the kmain object, forming the main executable.
This will involve interfacing with the cmos chip, so would be nice to also add a interface to abstract he cmos functionality. Then add the current date and time to the screen under the logo in the center (this is how I have it in DeanOS).
Like for the PIC where the IRQ have named values, do the same for the ISR. This will also need to update all the magic number in all files, e.g. in paging.
ziglang/zig#2627 is preventing us from implementing Directory and DirectoryEntry as packed structs due to byteoffset and size bugs.
The help text for the zig build
debug option is confusing, since it says that it causes the kernel to be built with debug info but this happens irrespective of the option's value. As it stands, our options are
zig build run
will have to be branched out to zig build run-debug
).For option 2 and 3, a value of false will mean we have to strip the debug info from the build with --strip
or the equivalent build.zig
function.
I didn't get 4KB paging working with my original effort (I seem to be cursed).
The kernel should be moved to the higher half to better facilitate user processes in the future. This can be done by putting the initial boot code at 1MiB, setting up paging and jumping to code addressed at 3 GB + 1 MiB with a rudimentary page directory.
This will make the code clean and consistent. Once a style is decided 2 things need to be done:
A checker should be added to CI with a failure returned if it isn't up to standard.
A linter script should be added to the repository so any developer can check a commit before pushing with a mistake.
We'll need a very simple memory allocator that will allocate linearly between given a start and end address, and that doesn't need to care about freeing. This allocator will be used to allocate memory needed before a more sophisticated heap is set up.
Once ziglang/zig#313 is done
We're currently using Zig's normal executable building procedure with zig build-exe
but this doesn't use our build.zig
file. I propose that we start using zig build [steps]
instead and revamp build.zig
. An idea for a build/run workflow is:
zig build
zig build run
, zig build run -Ddebug=true
to make it wait for a debugger connectionzig build debug
This could also coincide with the effort to make adding -curses
to the qemu invocation unnecessary.
We'll need to setup segmentation as it's pretty much required by other features of the architecture. It can follow a flat model where the user and kernel segments overlap entirely, leaving permissions and access management to paging.
The readme could do with some improvement. It will need some more extensive building and running instructions (all the dependencies, possible options to pass to zig build
, exact zig version required etc.) and a list of the project goals. Contribution guidelines would also be good, including the style guide decided upon in #61
It would be a good idea to set up continuous integration for the kernel, which would make sure that the codebase remains in a stable state. It could be used to ensure that each pull request can be built in a number of configurations (architecture etc.) and that all tests pass.
Criteria for a CI service for us to use (in order of priority) are:
Relates to #12
Add the programmable interrupt timer. This then can be used with a scheduler.
Have the ability to read in the keys pressed and handle the control characters, e.g. shift... Also provide an interface to abstract all the functionality of a keyboard.
The syscall handler type could be refactored to add a list of 5 arguments. This would mean each individual handler wouldn't have to extract the arguments from the context itself, and would protect against syscall ABI changes.
From fn (ctx: *arch.InterruptContext) u32
to fn (ctx: *arch.InterruptContext, arg1: u32, arg2: u32, arg3: u32, arg4: u32, arg5: u32) u32
The handler would then be called with
handler(ctx, syscallArg(ctx, 0), syscallArg(ctx, 1), syscallArg(ctx, 2), syscallArg(ctx, 3), syscallArg(ctx, 4));
Create functions that interface with the PIC controllers so abstract the functionalist of the PIC
Here is one example:
https://andrewkelley.me/post/zig-stack-traces-kernel-panic-bare-bones-os.html
When looking at the x86 boot code in pluto.elf
with objdump you can see a lot of NOPs that seem to have no use whatsoever.
$ i686-elf-objdump -d bin/iso/boot/pluto.elf
00102000 <_start>:
102000: b9 00 10 10 00 mov $0x101000,%ecx
102005: 0f 22 d9 mov %ecx,%cr3
102008: 0f 20 e1 mov %cr4,%ecx
10200b: 83 c9 10 or $0x10,%ecx
10200e: 0f 22 e1 mov %ecx,%cr4
102011: 0f 20 c1 mov %cr0,%ecx
102014: 81 c9 00 00 00 80 or $0x80000000,%ecx
10201a: 0f 22 c1 mov %ecx,%cr0
10201d: e9 de df ef bf jmp c0000000 <KERNEL_VADDR_START>
102022: 90 nop
102023: 90 nop
102024: 90 nop
102025: 90 nop
102026: 90 nop
102027: 90 nop
102028: 90 nop
102029: 90 nop
10202a: 90 nop
10202b: 90 nop
10202c: 90 nop
10202d: 90 nop
10202e: 90 nop
10202f: 90 nop
102030: eb fe jmp 102030 <_start+0x30>
An investigation will be needed to determine if this is a build procedure issue or compiler issue.
zig#2616 is preventing us from using a build of zig master so the newly added alignment functions will need to be removed once we can use zig master.
Compiling the architecture-specific code to an object andl inking it with the main object is becoming cumbersome and presenting problems when implementing the x86 GDT and IDT. This issue tracks the work on redesigning the compilation and source structure so that only the minimal boot code is compiled to an object and all other architecture symbols (init
, outb
etc.) are imported in the normal zig way.
The symbol file path given to the gdb command is incorrect. We need to pass the path to the elf file in the same string as the "symbol-file" argument.
Manage the physical memory.
Also, we can decide with the 32 bit OS, handle 4KB chunks where for 64 bit OS handle 4MB chunks, as who has less than 4GB of RAM these days, am I right ;).
Here is an example written in C:
https://github.com/DrDeano/DeanOS/blob/master/src/kernel/kernel/pmm.c
A good tutorial:
http://www.brokenthorn.com/Resources/OSDev17.html
We need a basic bootable kernel for x86. This can be imported from my previous kernel but with the relevant conversions to Zig (Andre Kelley's video on this will be helpful when it comes to the language's intricacies).
This work will include making a master Makefile that will also make the chosen architecture's bootloader.
A discussion point is whether or not the kernel should be in the higher half (with addresses starting at 3GiB + 1MiB) from the very start. I think it should as it makes mapping the kernel into user processes easier, and means you don't have to relocate an executable's code up to the higher half later on.
The kernel will need syscalls so that programs can interact with the kernel. We could choose the same interrupt number as linux, which would be 0x80. The number used to select which syscall is desired could be placed in eax
, and the kernel will perform the appropriate action for that number. Parameters should be passed according to the architecture's ABI, so for x86 we'd follow the cdecl ABI.
Depends on #51
The IDT is needed to handle exceptions and interrupts. Each IDT entry should be associated with a default handler which should check if a higher-level handler has been registered and if so call it, else continue on. This higher-level handler system should be separate from the IDT system.
An example flow:
Interrupt -> Handler in IDT (Stores state on stack, calls default handler) -> Default handler (Checks list/array of pointers to handler functions) -> Specialised handler (Does its stuff) -> Default handler -> Handler in IDT (restores state from stack etc.)
It would be nice to at some point use the super-duper bootloader written by @DrDeano . Perhaps with the stretch goal of migrating some of it to Zig if we can (probably a separate issue).
The latest version of zig brings some bug fixes and improvements that we'll want when building pluto. This will require some syntactic changes.
Paging needs to be implemented to map the higher half kernel to physical memory. This will also require a simple watermark memory manager that takes the end of the loaded kernel code and allocates memory on a linear basis after that (this may require a separate ticket).
4KB paging will be looked at later on.
We'll need a rudimentary cooperative multitasking system that presents an interface to a program, allowing it to have its state stored, relinquish control to another task. When control is returned to the task in the future, its state should be restored.
Control could be relinquished by sending an interrupt since the interrupt handling system already does a lot of the work of saving the CPU state, and this would lay the foundation for a more advanced multitasking and scheduling system, where all task transitions occur via syscalls and timer interrupts.
Preemptive scheduling with a timer interrupt should probably be done in a separate issue.
Depends on #36 if interrupts are the only way for a program to relinquish control.
Logging to serial. Should support multiple levels (debug, warning, info and error). An example log message is [DEBUG] This is the message
.
The logging interface should provide a function for each level which all call a generic function, and the generic function itself.
Options can be passed from build.zig to any zig source file. This can be used for configuration and creating builds for a specific purpose, such as running extensive runtime tests.
The option is added by calling addBuildOption(...)
on a LibObjExeStep
(e.g. the result from addObject(...)
, then used by importing zig-cache/{object name}_build_options.zig.
This is pretty much done with @SamTebbs33 paging. But once the physical memory manager is complete, can integrate this into the paging, so can allocate pages into physical memory.
Once this is done, this can be integrated into the heap memory manager.
The test step in build.zig currently tests all build modes. Let's change it to only test the one supplied by the user but test all in CI.
It would be nice if we could replace the x86 interrupt assembly (irq_asm.s and isr_asm.s) with zig code. This should be possible using naked functions and inline assembly, so we precisely control what happens. The interrupt handler functions could be generated with a pattern similar to this, but something simpler would be nice (perhaps just write them out manually?).
There is a lot of comments in the code of how it works
Instead of the existing system of importing the arch.zig file, have the architecture arch.zig file be added as a package in the build script. This will remove the kernel level arch.zig file. This will also help with the testing.
Is blocked by:
ziglang/zig#855
We can't do this as we have the kernel as a package and as the kernel will import the arch package and arch will import the kernel package, will need to wait for this.
When building the kernel, the entire build process is gone through even if no source files have changed so some investigation is needed. If this turns out to be an issue with zig and it is supposed to only build changed files then a bug report should opened. If we've done something wrong then the required changes should be made.
Only have the the IRQ handles be registered once has this could be a security risk as malicious program could re-register a IRQ handle
The arch interface should present a function that registers an interrupt handler so that arch-agnostic features (like system calls) can be implemented in an arch-agnostic way.
Serial output can be used for logging purposes as we can attach it to stdio from qemu and print debug messages that we may not want to print to the tty. Serial input is out of scope for this issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.