mmastrac / kalos Goto Github PK

AVR supports integer operations, but we need to test and confirm that string operations work as well. This should be a matter of implementing the correct far pointer types for the string system and adding a binding to a serial port write call.

We can move some of our parsing code into scripts if we can expose the kalos_lex functions to scripting code.

The namespace and symbol management in the current parser is very ad-hoc. We have multiple symbols we need to resolve, each it their own ad-hoc namespaces:

1. Builtins
2. Modules/module symbols
3. User vars/consts
4. User functions
5. Additional built-in functions

We should extract all of these into a single place for managing namespaces. As each of these comes along with some "baggage" (ie: modules have a module pointer, vars have a slot index, etc), we need to stash some context information alongside the symbol name.

In addition to simplifying symbol lookup:

• we can also implement block scoping for variables, allowing users to define var and const wherever they find it most appropriate to do so, rather than at the top of the function.
• we can more easily run the parser twice over the source code and pick up function definitions later in the file as forward references

The ITERATOR opcode should have an alternate form that takes a string and iterates over all the characters in the string.

DUP and ITERATOR_NEXT could be made much cleaner using a peek parameter for opcode dispatch. As it stands right now, DUP is handled manually and ITERATOR_NEXT is forced to store its variable in a local (disallowing alternative setting forms that Python allows like for a[3] in x {.

Variables are currently scoped to all following code in a function, but they really should be scoped to a block itself. In addition, it should be legal to shadow variables in nested blocks.

Right now all objects use the the _OBJECT type. We should be creating a new type for each kind of object, and testing those when we do the invoke/dispatch test.

The parser and runtime currently enforce types at runtime, but adding full typing to the system would make scripts more robust overall and improve the runtime speed by avoiding the dynamic dispatch we currently have to do.

Open questions:

• How do we handle casting and type checking? Should we add a let statement?

Indexing and function invocation only work on 'word-like' tokens at this time. This means that trying to index a string or list literal does not work:

assert("abcdef"[2] == "c");

While this is not a great pattern to use, it should compile as it might be useful in certain cases.

List comprehensions are a useful syntactic sugar for creating lists from other iterable things. A list comprehension can be trivially converted into a loop under the hood, but this would require special changes to the parser to switch from a list creation expression to a list comprehension expression.

At this time there are no other parser backtracking or tree peek-ahead, so this infrastructure would need to be built.

The compiler main is currently written in C, but we could provide the appropriate bindings to open, read, and write files, compile IDL and scripts, etc so that the compiler binary could be self-hosted.

This may require implementation of some other features (ie: stringify/hexify probably require #6). The self-hosting portion of the makefile may also require some improvement as getting stuck in a self-hosting loop might be more painful.

• Implement stringify
• Implement idl
• Implement run

Built-in objects should support embedding in the bytecode using a serialization format that is TBD.

We should support:

• Lists
• Strings
• Numbers
• String formats

The parser currently requires that a loop variable be defined outside of the loop, but we could scope the loop variable to the loop itself and allow it to be defined as part of the for:

for var i in range(0, 20) {
  ...
  # i is valid
}

# i is no longer valid

The current self-hosting Makefile targets are brittle and can leave you with a broken tree that is difficult to recover from. Self-hosting should probably make use of "known-good" git hashes that we check out, compile and run over the main tree's binding definitions

We would add:

• Kalos bindings for all compiler functions (compile IDL, compile script, run script).
• A self-hosting kalos compiler file (compiler.kalos) that can dispatch all the current compiler_main.c functions

For any workflow requiring us to generate local code, we want to run the generation from the last known-good hash (or provide a way for the user to specify a known-good hash):

1. git clone the current repo to target/gen/compiler at a known-good hash
2. Run make target/compiler in that folder
3. Run target/gen/compiler/target/compiler with a new self-hosting mode allowing us to pass a custom Kalos script for dispatch generation (ie: target/compiler selfhost script.kalos)
4. Copy the appropriate self-hosting output into the local tree and make test locally to get results

All errors are currently fatal, but we could be adding either error states (like Rust's Result) or try/catch keywords.