mikepopoloski / slang Goto Github PK

There are a bunch of "expected" diagnostics in the parser that could probably be collapsed down.

Instead of just the file name, use the relative path to the file when printing error messages.

The lexer tries to handle non-ASCII characters embedded in the source gracefully. If there are too many of them however, maybe the user accidentally pointed us at a binary file or something. Keep a counter and if we hit some threshold just swallow the rest of the text instead of producing a billion errors.

Add some way to pass options around so we don't hardcode various things like recursion limits.

There are a few preprocessor directives that aren't yet handled. Implement them in Preprocessor::next().

Once refactoring is finished, re-enable all of the various error checking tests.

Use something like premake to generate project files instead of maintaining them by hand.

The SystemVerilog preprocessor supports token pasting (like the C preprocessor). This is implemented in Lexer::concatenateTokens.

There are a bunch of TODOs in the code that cover things like testing all corner cases, better error handling, accounting for trivia in either token, and correctly setting the resulting token's location for proper expansion reporting in diagnostics.

SystemVerilog has a preprocessor feature that enables and disables various sets of keywords. We should take into account that state and downgrade keyword tokens back into simple identifier tokens.

Implemented in Lexer::stringify. It's not always clear what kind of spacing should be in the resulting string; the standard doesn't have much to say about it. Probably we should take a look at what existing Verilog compilers do and try to match them.

See Binder.cpp:147

Simple cleanup to make the code more readable.

BumpAllocator was never designed for types with non-trivial destructors, but we've started using it that way. It should support detecting types that require this and maintain a separate list so that we can call them.

If the closing parenthesis is missing, the macro parser might loop forever. There is a TODO to handle skipping invalid tokens.

Useful as a standalone tool and for checking the preprocessor.

Add some safety to the as() method on SyntaxNodes, which currently just does a blind cast. We know the Kind of the node at runtime, we should be able to check that against the static type given by the template parameter and assert.

This should alleviate some of the null checking we currently do to be safe.

This would be a nice-to-have: the StringTable used to lookup whether an identifier is a keyword could use an offline-computed perfect hash to make it much faster.

For tokens like " and ", they must occur inside a macro definition.

The intrinsic FILE and LINE macros are currently recognized but generate empty tokens ("" and 0 respectively). The LINE macro in particular will require making use of the source manager infrastructure for figuring out which line we're looking at.

Note that this plays into the `line directive as well, which changes the "apparent" line number (like #line in C++).

So far the project has focused on parsing and semantic analysis; the "front end" of the compiler. We need to think about how to take symbols and bound expression trees and generate output from that.

We'll eventually want several backends; LLVM for simulation, netlists for hardware synthesis. For now though, I think we want to go to simplified Verilog for quicker bringup.

This could probably be broken down into a bunch of issues. The ExpressionBinder has basic implementation for arithmetic operators and simple variable lookup, but a bunch of features aren't done yet. Examples of things that make this hard are supporting function calls at compile time (for constant expressions).

We should take a look at what Clang does for constant expression evaluation; in contexts where we know we need a constant expression, the expression binder will keep track of reasons why it was unable to propagate a constant so that we can give useful error messages.

Pretty much everything right now is hardcoded for integrals.

Module symbol binding has been started, and good progress has been made on binding all parameters. Any useful thing we might want to do with the compiler requires at least having the hierarchy bound, so make sure that all works correctly.

Rather than try to greedily create all symbols, the semantic model should only create each symbol and its children when it becomes necessary. This should solve ordering issues between having symbols created and evaluating things like initializers.

Make sure even in weird scenarios that all expanded tokens get correct locations assigned.

SourceManager should be one of the few thread-safe components in the system. Someone just needs to do the work to make it that way.

There are a bunch of TODOs to handle corner cases and report errors appropriately. These should be worked out.

There are cases now where running into a syntax error in a loop will cause it to loop forever fabricating new tokens via expect().

The preprocessor should report an error if you try to redefine a built in directive.

The parser can currently handle 75-80% of the language, and handles the most common parts. There is definitely code out there using the parts we don't yet handle though.

This involves adding new syntax nodes to syntax.txt, adding a new type to SyntaxKind, and then making the parser produce the new nodes. It's pretty straightforward one you do one or two of them.

Currently we only support simple variable lvalues, but there are others like concatenation.

There's a test that's currently commented out that has ridiculously nested code to test overflow cases in the compiler. Since there's no handling right now, it predictably crashes. Whenever we have parser calls that have the potential to recurse on themselves, we should add some guards to check for nesting that's too deep and issue an error if we hit the threshold.

Type symbol binding has only a very bare bones implementation; there are many cases left to handle.

Compilers like GCC and Clang notice the common ifndef define pattern at the top of include files and use it as a marker to skip lexing the entire file if it's already been seen. As an optimization we should do the same.

This is low priority.

If the user redefines a macro, a warning should be reported unless the two macros have exactly the same bodies.

Temporaries produced during constant evaluation should all live in a temporary heap that we can just throw away.

Include file names can actually be specified as a macro that should be expanded; this is fairly straightforward but pretty annoying since a bracketed include name will show up as a whole bunch of tokens in the stream. C++ supports this as well, so we can take inspiration from GCC or Clang here.

Identifier tokens that match a keyword name are turned into the appropriate keyword; make sure this isn't happening for escaped identifiers and write a test to verify it.

Like

`define FOO(type, var) type var;

type, var are keywords, but they need to be parsed as macro arguments and replaceable as macro arguments

See title.

The initial implementation of timescale directives needs error checking for robustness.

There is initial support for this but it's not complete.

Rather than have a ParameterizedModuleSymbol, move that into the instance itself.

After things have settled down a bit, go through and move everything into folders, like the original filter setup in the vsproj.

Most preprocessor directives require that they start on their own line. Currently this isn't checked or enforced.

DiagnosticWriter::formatDiag should take into account the tab stops of the output terminal when positioning the source location caret and any source ranges.

There are a few places where we const_cast in SyntaxNode out of laziness; add another overload to getChild() on each derived type so that we don't need to do this.