ethereum / fe

Emerging smart contract language for the Ethereum blockchain.

Fe is an emerging smart contract language for the Ethereum blockchain.

NOTE: The larger part of the master branch will be replaced with the brand-new implementation, which is currently under development in the fe-v2 branch. Please refer to the branch if you kindly contribute to Fe

Overview

Fe is a statically typed language for the Ethereum Virtual Machine (EVM). It is inspired by Rust and easy to learn -- especially for new developers entering the Ethereum ecosystem.

Features & Goals

• Bounds and overflow checking
• Decidability by limitation of dynamic program behavior
• More precise gas estimation (as a consequence of decidability)
• Static typing
• Pure function support
• Restrictions on reentrancy
• Static looping
• Module imports
• Standard library
• Usage of YUL IR to target both EVM and eWASM
• WASM compiler binaries for enhanced portability and in-browser compilation of Fe contracts
• Implementation in a powerful, systems-oriented language (Rust) with strong safety guarantees to reduce risk of compiler bugs

Additional information about design goals and background can be found in the official announcement.

Language Specification

We aim to provide a full language specification that should eventually be used to formally verify the correctness of the compiler. A work in progress draft of the specification can be found here.

Progress

Fe development is still in its early stages. We have a basic Roadmap for 2021 that we want to follow. We generally try to drive the development by working through real world use cases. Our next goal is to provide a working Uniswap implementation in Fe which will help us to advance and form the language.

Fe had its first alpha release January 2021 and is now following a monthly release cycle.

Getting started

• Build the compiler
• Or download the binary release

To compile Fe code:

1. Run fe path/to/fe_source.fe
2. Fe creates a directory output in the current working directory that contains the compiled binary and abi.

Run fe --help to explore further options.

Examples

The following is a simple contract implemented in Fe.

struct Signed {
    pub book_msg: String<100>
}

contract GuestBook {
    messages: Map<address, String<100>>

    pub fn sign(mut self, mut ctx: Context, book_msg: String<100>) {
        self.messages[ctx.msg_sender()] = book_msg
        ctx.emit(Signed(book_msg: book_msg))
    }

    pub fn get_msg(self, addr: address) -> String<100> {
        return self.messages[addr].to_mem()
    }
}

A lot more working examples can be found in our test fixtures directory.

The most advanced example that we can provide at this point is an implementation of the Uniswap-V2 core contracts.

Community

• Twitter: @official_fe
• Chat: Discord

License

The Fe implementation is split into several crates. Crates that depend on the solidity compiler (directly or indirectly) are licensed GPL-3.0-or-later. This includes the fe CLI tool, yulc, driver, tests, and test-utils.

The remaining crates are licensed Apache-2.0. This includes the parser, analyzer, mir, abi, and common.

fe's Issues

Full event support

We are currently able to emit a Signed event in the guestbook contract, but not much more. For ERC-20, we should be able to emit events of any parameter set or at least just any parameter set of base types.

• add multi-parameter support for events

Vyper -> Fe

Replace all references to Vyper with Fe.

Implement revert reasons

What is wrong?

#74 adds support for revert but it does not yet add support revert reasons. We should support revert reasons.

How can it be fixed

I haven't given that much thought. I guess it boils down to reserving some space for revert reasons and then calling rever(start, end) with the correct memory offsets but I would have to take a closer look.

Add cmake to CI

CI is currently failing with:

failed to execute command: No such file or directory (os error 2)
is `cmake` not installed?

todo
Add cmake and other deps to the environment.

Compilation output should be valid EthPM package

I think it should be a goal for native EthPM integration.

It is looking like the solidity team is going to take this into consideration and we could demonstrate the concept here:

https://docs.ethpm.com/ethpm-spec

In theory, we should be even able to go as far as taking a source-only package as the input for compilation and outputting a package which specifies the input package as a dependency and includes the compiled outputs.

Fully implement expression mapping

The following expressions need to be mapped to Yul:

• Ternary
• BoolOperation
• BinOperation
• UnaryOperation
• CompOperation
• Call
• List
• ListComp
• Tuple
• Str
• Ellipsis

Attempting to compile a contract that includes one of these expressions will give rise to an unimplemented error.

Todo:
Replace each branch in this statement with an appropriate call to a new mapping function that returns an ExtExpression result.

Arithmetic expression mapping for U256 values

The following Fe expressions should be mapped to Yul, like so:

Fe	Yul
a + b	add(a', b')
a - b	sub(a', b')
a * b	mul(a', b')
a / b	div(a', b')

Where a and b are vyp::Exprs and a' and b' are yul::Expressions.

todo:

• call a new function expr_bin_operation here that returns an ExtExpression.
• add a match statement to expr_bin_operation that matches the Add, Sub, Mult, and Div operations and calls more new functions expr_add, expr_sub, expr_mult, and expr_div. (mark all other branches as unimplemented)
• match both the left and right ExtExpression types (i.e. (Base::U256, Base::U256)) in each of the expr_ functions and return a new ExtExpression of type U256 with an expression containing the correct arithmetic function.
• add a test contract with a function that takes two params, adds them together, and returns the result.

Note: the Base::U256 enum value should contain a U256 struct with arithmetic functions attached to it. However, this is not necessary at the moment.

Fits into #21.

Possible introduction of prefix keywords or qualifiers

There are a lot of situations where Rust Vyper could emulate Python Vyper's syntax, but it seems sub-optimal to do that because Python Vyper's syntax seems to have been designed mostly to accommodate the Python parser.

From the point of view of Rust Vyper, it seems more and more like the Python parser is underpowered because it was designed for a dynamic language in which methods and variables don't require much annotation. This is because there aren't many restrictions on how methods and variables can be used in Python.

Here's a concrete example. If Rust Vyper followed Python Vyper's syntax, here are two different ways that things would be annotated to indicate that they're "public":

contract GuestBook:
    guest_book: public(map[address, bytes[100]])

    @public
    def sign(book_msg: bytes[100]):
        ...

Frankly, this is ugly and here are some reasons why:

• Storage variables and methods are being annotated in two entirely different ways. Storage variables use something like a "pseudo-function" called public that "returns" a new type public(map[address, bytes[100]]). On the other hand, methods use the python decorator syntax. Public methods are annotated by placing a @public directive on the line preceding the method declaration.
• Being forced to use the pseudo-function syntax for variable annotations means that having more than one annotation (if we ever want a feature like that) makes things very unreadable and forces the inclusion of lots of nested parentheses.

Now consider this alternative syntax:

contract GuestBook:
    pub guest_book: map[address, bytes[100]]

    pub def sign(book_msg: bytes[100]):
        ...

In my opinion, this is far more readable for the following reasons:

• Both property and method annotations use the same syntax.
• The annotations are at the front of the line where they are most visible.
• The keyword "pub" is shorter but no less expressive.

It seems like we could also elect to keep the decorator syntax in case we also want modifiers (which are a feature that I think was unfairly excluded from Vyper). But the qualification of a method as public or not could use a prefix keyword on the line of the method declaration.

The EVM contracts tests do currently not run in CI

What is wrong

The EVM contracts tests do currently not run under CI.

How can it be fixed

I'm not entirely sure but I speculate the reason why these tests do currently not run in CI is because they depend on building the solc compiler first which takes a long time. Wondering if we can fetch a binary from somewhere instead to skip that step.

Throw DivisionByZero exception

What is wrong

After #36 lands support for basic arithmetic operations including division will be implemented. However, division by zero will return 0 as that is the EVM default behavior. We should throw an exception instead.

How can it be fixed

Need to investigate. Solidity fixed that a long time ago. Related threads ethereum/solidity#888, ethereum/solidity#670

Improve parser error message

What is wrong?

Parsing code that isn't yet supported leaves one with an error that just says "Parser error". E.g. the snippet from #59 is such a candidate.

contract Foo:
    pub def bar() -> u256:
        return 0

How can it be fixed

We drop all previous error information from the ParseError here:

fe/compiler/src/errors.rs

Lines 46 to 50 in 5b9d302

	impl<'a> From<ParseError<'a>> for CompileError {
	fn from(_: ParseError<'a>) -> Self {
	CompileError::static_str("parser error")
	}
	}

However, there's not really a good error message from the parser that we could display so I guess we would have to start there to improve the errors.

Support strings

The following should be completed to support the use of strings in our ERC-20 contract:

• create a new string type
• support the use of strings in storage
• support the use of strings in memory
• support string abi encoding

Decompose Fe -> Yul compiler pass

Currently, we have a single compiler pass that maps Fe ASTs generated by the parser to Yul. This pass is responsible for both context-sensitive analysis and mapping Fe to Yul. As the project grows in complexity, it will become difficult to manage both of these responsibilities in one pass and would result in a codebase that's difficult to audit.

We can improve the situation by introducing a pass before the Fe to Yul mapper that is responsible for both deeper validation of the source program and for providing the mapper pass with enough information to correctly map expressions.

Let's illustrate with the guest book contract:

/* guest_book.vy */
type BookMsg = bytes[100]

contract GuestBook:
    pub guest_book: map<address, BookMsg>

    event Signed:
        idx book_msg: BookMsg

    pub def sign(book_msg: BookMsg):
        self.guest_book[msg.sender] = book_msg

        emit Signed(book_msg=book_msg)

    pub def get_msg(addr: address) -> BookMsg:
        return self.guest_book[addr]

Compilation of the returned expression in get_msg goes roughly like this:

context/mapper pass

• gets the type of self.guest_book from the contract scope and maps it to a Yul expression (its storage index) then returns this information to the caller.
• gets the type information for addr from the function scope and maps it to a Yul expression then returns this information to the caller
• determines the type of self.guest_book[addr] given the type of self.guest_book and maps it to a Yul expression then returns this information to the caller

It should instead go like this:

context pass

• gets the type of self.guest_book from the contract sope and stores it
• gets the type of addr from the function scope and stores it
• determines the type of self.guest_book[addr] and stores it

mapper pass

• maps self.guest_book to a Yul expression given the expression type from the previous pass
• maps addr to a Yul expression given the expression type from the previous pass
• maps self.guest_book[addr] to a Yul expression given the expression type from the previous pass

The second method separates concerns nicely and conforms more closely with standard compiler design.

todo:
Determine what the data structure built up by the context pass should be.

CI doesn't seem to run on PRs

CI doesn't seem to run on PRs. Also, maybe it's worth migrating to Github Actions.

Base type function parameters

It should be possible to pass base type values into functions internally.

• support passing base type values into other contract functions

Ensure we run CI against Windows too

What is wrong?

After landing #63 Windows is the only major platform left that we do not yet test against in CI.

How can it be fixed

1. investigate the failure in #64

No SMT solver found (or it has been forcefully disabled). Optional SMT checking will not be available.
Please install Z3 or CVC4 or remove the option disabling them (USE_Z3, USE_CVC4).

The failure ☝️ is weird because both should actually be disabled.

Support events outside of contracts

What is wrong?

Currently, events are only allowed inside contracts like so:

contract GuestBook:
    
    event Signed:
        idx book_msg: u256

    pub whatever:
        ...

They should be supported outside of the contract, too like so:

event Signed:
    idx book_msg: u256

contract GuestBook:
    
    pub whatever:
        ...

How can it be fixed

🤖

Start language specification

We will need to write a specification so that it's possible to audit this project. It would also just be useful to have in general for ourselves and for users.

This is the specification for c++. It might help give us an idea of how our specification should be structured.

For this issue we should do two things:

• create a new directory named spec with an empty README
• create a couple issues for things that can be easily specified at the moment

Nested maps

We should be able to support maps of the form: map<address, map<address, u256>>

My thinking is that the storage location for the final value (u256 in this case) should be found like so:

self.foo[0x00][0x01]
|------|
    0.) storage index
|------------|
       1.) intermediary storage index
|------------------|
         2.)  storage location of value

The expression attributes for each of the above would be as follows:
0.) type: Map<...>, location: Storage { index: n }
1.) type: Map<...>, location: Storage { index: 0 } with the real index being determined during runtime keccak256(n, 0x00)
2.) type: u256, location: Value

Note: This is a bit strange in that we are using what would normally be a pointer to a segment in storage (self.foo[0x00]) as a new storage index. I don't see a problem with this though.

Function call statement support

We should be able to compile function call statements.

e.g.: self.foo(42, 26, my_var)

This will be very similar to what has been implemented for #78 and #89. We can just discard the return value (if there is one).

Support all numeric types

Currently we just support u256 values. We should add support for for all signed and unsigned values of size 8-256. They should be included in the base type enum under a new variant named Numeric possibly like this:

enum Numeric{
  Int(usize), // doesn't actually need to be implemented for ERC-20
  Uint(usize)
}

enum Base {
  ...
  Numeric(Numeric)
}

The following should be supported for all numeric types:

• use in storage
• use as value
• basic arithmetic
• abi encoding

Tokenizer calculates spans off by one

What is wrong?

Given the following code:

fe/compiler/tests/fixtures/guest_book.vy

Lines 1 to 15 in 8d05a90

	type BookMsg = bytes[100]
	contract GuestBook:
	pub guest_book: map<address, BookMsg>
	event Signed:
	idx book_msg: BookMsg
	pub def sign(book_msg: BookMsg):
	self.guest_book[msg.sender] = book_msg
	emit Signed(book_msg=book_msg)
	pub def get_msg(addr: address) -> BookMsg:
	return self.guest_book[addr]

The tokenizer calculates these tokens.

[Token {
    typ: NAME,
    string: "type",
    span: Span {
        start: 0,
        end: 4
    },
    line: "type BookMsg = bytes[100]\n"
}, Token {
    typ: NAME,
    string: "BookMsg",
    span: Span {
        start: 5,
        end: 12
    },
    line: "type BookMsg = bytes[100]\n"
}, Token {
    typ: OP,
    string: "=",
    span: Span {
        start: 13,
        end: 14
    },
    line: "type BookMsg = bytes[100]\n"
}, Token {
    typ: NAME,
    string: "bytes",
    span: Span {
        start: 15,
        end: 20
    },
    line: "type BookMsg = bytes[100]\n"
}, Token {
    typ: OP,
    string: "[",
    span: Span {
        start: 20,
        end: 21
    },
    line: "type BookMsg = bytes[100]\n"
}, 
...

Read full source

It is placing "type" from 0..4 which is correct but then it puts BookMsg from 5..12 which actually spans BookMs (from the leading whitespace up to the s, missing out the g.

How can it be fixed

It looks like it is miscalculating the whitespace somewhere but I did not dig in further.

Ensure order of function in contract doesn't matter for semantic analyses

What is wrong?

As mentioned in this comment, analyzation of functions currently happens one at a time which causes us not have all needed information available depending on the order in which functions are defined in a contract.

E.g. semantic analyzes works as intended on this snippet:

contract Foo:

    pub def foo():
        revert

    pub def bar() -> u256:
        return self.foo()

But doesn't when the order is the other way around.

contract Foo:

   pub def bar() -> u256:
        return self.foo()

    pub def foo():
        revert

How can it be fixed

Citing @g-r-a-n-t from #78 (comment)

At the moment, function definitions are analyzed one at a time from top to bottom. So, since foo is defined after bar, we can't see foo in bars function body.

We should modify the contract analysis function to instead do an initial sweep on all function definitions that adds their signatures to the scope.

Create an online editor for people to try out Fe

What is wrong?

This doesn't strictly belong here but for the lack of a better place and to make sure to remember it, I'll file it here as a documentation issue.

We want an online playground for people to try out Fe similar to e.g. https://play.rust-lang.org for Rust.
@pipermerriam recently pointed out that it makes sense to prioritize this and get something up early even at a stage where the language isn't yet fully functional.

How can it be fixed

There are generally two possible ways:

1. Frontend + Backend where the frontend sends code over to a backend that runs the code and returns the compiler output back to the frontend

2. Compile the compiler to WASM and have it sit in the frontend directly, eliminating the need for a backend.

yultsur / solc-rust

I see in the README that "This implementation targets the YUL IR used by and developed for Solidity". Do you have anything in mind yet how to do that?

Incidentally we have two projects which could be of help to you:

• yultsur - a library for the Yul AST, to parse, print and have a programmatic builder
• solc-rust - Rust bindings for the Solidity compiler

They didn't receive too much attention in the past year due to lack of time and lack of users, but if you project would utilize them that would mean it makes sense to keep them maintained.

Bind to libyul instead of libsolc

We are currently binding to libsolc using solc-rust and performing Yul -> bytecode compilation through the json interface.

This is problematic because building the whole Solidity compiler is very resource and time intensive (can take up to an hour). It would be nice if we could instead bind to libyul, as this takes a fraction of the time to build.

Memory and storage specification

We should have a section of the specification that describes the following:

• storage layout for each type
• how memory works with regard to function calls
• are reference types copied when passed to a function?
• when is memory allocated and freed?
• how are reference type returned from functions?
• clearly describe when something will be put in storage
• clearly describe when something will be put in memory

Project goals/roadmap (README improvements)

The README was just updated to include more information about the overall goals of the Rust Vyper project and its relationship with the existing Vyper project. Things that still need doing:

• Editing/wording improvements
• Inclusion of any additional goals or information that may have been left out
• Inclusion of a project roadmap with an emphasis on near-term milestones

Syntax of storage and memory handling

Storage and memory

This isn't actionable but it's a thought I wanted to get written down to pick back up when the time comes.
Solidity has two keywords storage and memory that can control where a value ends up being stored.

Example:

    function example1(uint storage value) public {
        // code
    }
    function example2(uint memory value) public {
        //code
    }

The Rust language has a philosophy that often prefers to make things library features rather than syntax features if possible. (That isn't to say rust has a simple syntax, it sure has a lot of syntax rules).

For instance, in Rust, to store a u256 on the heap rather than the stack one would but it in a Box<T> so the type would end up being Box<u256> and no special keyword is needed. I find that to be superior in many circumstances. Not only does it mean less keywords, it also means that the language can evolve easier without introducing breaking changes to its syntax. For instance, at some point a BetterBox<T> may be introduced and one could even end up using both side by side. Alternatively versions of Box<T> could live behind different import paths.

So, to translate the Solidity example into a made up form of future Fe code, this is what I could see it end up looking:

    pub def example1(value: Storage<u256>):
        # code
    pub def example2(value: Memory<u256>):
        # code

Better defined type properties with traits

The module that defines the behavior of types should be tightened up a bit. Instead of generating Yul statements on the types themselves, the types should have various properties attributed to them with traits. The traits would then be used by the compiler to generate various Yul operations. For example, if a given type does not implement the sized trait, it is impossible to perform data operations with it. If it does implement the sized trait, we can use its size value to generate Yul data operations.

The two types of operations that need be refactored are:

• ABI encoding/decoding (#99 via AbiEncoding trait)
• data operations (#93 via the FeSized trait)

Get test suite hooked up to CI (CircleCI)

Looks like CI isn't running tests (maybe just because it's from a fork) on #11

Should we use rstest to get parametric tests?

The rstest library gives us parametric tests roughly comparable to what pytest allows. It can often reduce the amount of boilerplate to test various cases in one single test instead of multiple separate tests.

use rstest::rstest;

#[rstest(input, expected,
    case(0, 0),
    case(1, 1),
    case(2, 1),
    case(3, 2),
    case(4, 3)
)]
fn fibonacci_test(input: u32, expected: u32) {
    assert_eq!(expected, fibonacci(input))
}

@g-r-a-n-t Any hesitation to use that where applicable?

Need to create a logo

Logo

One idea that I had was something like a polygon based rock where an iron band goes through.

E.g.

https://upload.wikimedia.org/wikipedia/commons/thumb/2/25/Banded_iron_formation_Dales_Gorge.jpg/1200px-Banded_iron_formation_Dales_Gorge.jpg

Support for call expressions with parameters

What is wrong?

After #78 lands, Fe will have support for call expressions (e.g. return self.foo()). However it does not handle yet call expressions with arguments such as return self.foo(42). We need to implement that.

How can it be fixed

Probably drilling deeper into what #78 does 😅

Clarification of project goals/roadmap

This issue is intended as a discussion thread for clarification of project goals and roadmap.

So here's a start:

Project goals

Provide a secure and reliable implementation of the Vyper language

• Emphasize thorough testing
  • Favor maintainability over cleverness

Provide a reasonably complete language specification document. Such a document should include:

• Syntax specification in BNF format (or a reasonable extension of BNF) similar to the Python grammar specification
• An operational, small-step semantics to describe correct program evaluation and behavior
• Sections of informal prose to aid in understanding of the grammar, semantics, their relationship and the context in which they operate

Differences from existing Vyper project

• Use of a systems language (Rust) with a strong type system and features intended to
• Emphasis on correctness of implemented features instead of quantity

Project roadmap

Best way to include source location annotations in AST or elsewhere?

For the past week or two, I've been trying to figure out the most idiomatic and DRY way to include source location information in Rust Vyper's AST. The motivation here is that, when some kind of compilation error message needs to be generated, the compiler presumably has some piece of the AST it's working with (an instance of an AST-specific struct or enum) and needs to figure out where in the source code this part of the AST was parsed from. So there needs to be some way to include that info along with or inside of the AST.

So far, I've just been considering ways to include that kind of information inside of the AST.

At first, I was just manually declaring a span: Span field in all AST-related structs or enum variants as can be seen in this version of the ast module. Obviously, this is a bit gross and not DRY at all. You've got to remember to include that field in the struct or enum definition and it also necessitates the existence of methods like this that resolve the variant of an enum and fetch the span field from it.

I initially had a misguided idea of how to fix this, which was to do something like the following:

struct LocatedNode {
    node: Node,
    span: Span,
}

enum Node {
    Module(Module),
    ModuleStmt(ModuleStmt),
    ConstExpr(ConstExpr),
    ...
}

This wouldn't be great because I would need to use this LocatedNode type all over the AST and then I would have to resolve the variant of the type at run-time, adding a bunch of computational overhead and eliminating the power of the type system to prevent invalid ASTs from being represented.

Then yesterday I realized I was being a dummy and completely ignoring the basic feature of generic types that Rust provides (not sure why...maybe it's been a while since I've lived in C++ land and places like it). I ended up just using a struct like this:

struct Spanned<T> {
    node: T,
    span: Span,
}

So now the ast module looks like this which is much simpler and more normalized.

I think that using this simple parameterized type is the correct way to proceed. But I still wonder if AST structs and enums (and functions that work on them) can be defined in an even more general way that removes the need to mention the Spanned type in the type hierarchy at all. Maybe AST structs and enums can be defined using generic type parameters somehow or maybe, as I touched on earlier, source location information can be stored in an entirely separate data structure from the AST.

The separate data structure approach might make a lot of sense conceptually since error message generation only happens once in a while and is more of a secondary function of the compiler. So it doesn't really matter if source location information is conveniently and efficiently available in most cases.

But I don't know, those are just some thoughts. I think I've basically got a good way to move forward and I think I'm just trying to figure out if there are some other tricks I could use to make things even more maintainable or elegant.

Consider replacing Pythonic indents with curly braces

Warning: This may turn out to be a controversial point, since it is about changing one of the core aspects of the syntax. While there seems to be some support from the people behind coq, and @g-r-a-n-t also said that he is open to this change, I am writing this topic to gauge the general sentiment and get the opinion of more people.

Proposed Change

Currently contracts and functions follow the Pythonic indentation rules for their definition.

contract Something:
    pub def foo(from: address) -> address:
        return address

This issue proposes to use curly braces instead so the above syntax would become:

contract Something {
    pub def pass_through(from: address) -> address{
        return address
    }
}

Motivation

The core motivation is that it becomes easier to parse and enables certain features down the road that would otherwise be hard to implement. For instance, the lack of curly braces is one of the main reasons why Python doesn't have rich support for lambdas.

Consider the following code.

contract Something {
    pub def pass_through(from: address) -> address{
        return self.things.filter(|thing| {
            if thing.address == address {
                return True
            } elif self.some_other_check(thing, address) {
                return True
            } else {
                return False
            }
        })
    }
}

Notice that I'm just artificially try to come up with a longer block inside the lambda to demonstrate something that isn't available in Python today.

Notice that I just went for the Rust lambda syntax but something like filter(thing => {...}) (TypeScript syntax) would just work equally fine. It's out of the scope of this issue to discuss lambdas.

In my opinion, Rust as being a very young and modern language seems to have taken a lot of inspiration from Python but improved on the parts that turned out to be holding the language back. I feel that we should be following Rust in this aspect.

Alternatives

My proposal above went into eliminating Pythonic indents all the way through contracts, functions, lamdbas as well as if / else because it felt like this change would naturally expand beyond just functions. However, we could also consider an alternative proposal that would just use curly braces for function bodies even though I have to admit it feels kind of weird.

contract Something:
   pub def pass_through(from: address) -> address{
       return self.things.filter(|thing| {
           if thing.address == address:
               return True
           elif self.some_other_check(thing, address):
               return True
           else:
               return False
       })
   }

Fix parser tests

These macro-generated tests are not compiling. We see the following error for each test function:

error[E0425]: cannot find function `test_guest_book` in this scope
   --> parser/tests/test_parsers.rs:118:9
    |
118 | /         fn $tester_name() {
119 | |             do_with_fixtures!(
120 | |                 assert_fixture_parsed_with!($parser),
121 | |                 $fixture_path,
122 | |             );
123 | |         }
    | |_________^ not found in this scope

The $tester_name identifier in this particular case is set to test_guest_book. If I attempt to only generate the guest book case by deleting the other test cases and then explicitly writing test_guest_book in the macro, it successfully compiles and the test passes. In more detail:

// replace the existing `parse_fixture_tests!` invocation with...
parser_fixture_tests! {
    (
        file_input,
        test_guest_book,
        write_guest_book,
        "fixtures/parsers/guest_book.ron",
    ),
}

// replace `$tester_name` with `test_guest_book` in the macro definition
macro_rules! parser_fixture_tests {
    ($(($parser:expr, $tester_name:ident, $writer_name:ident, $fixture_path:expr,)),+,) => {
        $(
        #[test]
        #[wasm_bindgen_test]
        fn test_guest_book() {
            do_with_fixtures!(
 ...

It looks like this may be related to macro hygiene, but I'm not sure.

Implement constructors

It should be possible to write a constructor that accepts arguments and sets storage values. I'm thinking the constructor should look like Python inits, but am open to doing something else.

def __init__(name: string, symbol: string):
      self._name = name
      self._symbol = symbol
      self._decimals = 18

The first code block in the Yul ERC-20 example is basically where construction will happen. It should look something like this:

code {
      function __init__(name, symbol) {
          ...
       }

      // Call the constructor
      __init__(...) // Call init with string params

      // Deploy the contract
      datacopy(0, dataoffset("runtime"), datasize("runtime"))
      return(0, datasize("runtime"))
}

note: strings are not required for this issue. We can just test with different types that are already supported.

Compile guest book contract to Yul

It should be possible to compile the following Rust Vyper code into Yul and bytecode:

/* guest_book.vy */
type BookMsg = bytes[100]

contract GuestBook:
    pub guest_book: map<address, BookMsg>

    event Signed:
        idx book_msg: BookMsg

    pub def sign(book_msg: BookMsg):
        self.guest_book[msg.sender] = book_msg

        emit Signed(book_msg=book_msg)

    pub def get_msg(addr: address) -> BookMsg:
        return self.guest_book[addr]

TODO list:

• Function-only ABI builder. See: #15
• Basic Yul compiler. See: #16
• Yul -> bytecode compiler and EVM tests. See: #17 (includes: #15 and #16)
• Support for: addresses, bytes, storage maps, and arrays. See: #18
• Event definitions and logging. See: #18 also

Fully implement function statement mapping.

The following function statements need to be mapped to Yul:

• AugAssign
• For
• While
• If
• Assert
• Expr
• Pass
• Break
• Continue
• Revert

Attempting to compile a contract that includes one of these statements will give rise to an unimplemented error.

Todo:
Replace each branch in this statement with a call to a new mapping function that returns a yul::Statement result.

Determine required features for ERC20 contract support

Compilation of an ERC20 token contract is the next big goal. To start, we should complete two things:

• Write an ERC20 token contract in Fe. It should be similar to the OpenZeppelin ERC20 contract.
• Create a list of language features that need to be implemented to support compilation of the contract.

Vyper or something else

The python vyper compiler is continuing with development.

There are already choices in this codebase which make this version of vyper not syntactically compatible with the python one. For this reason I think we have two choices.

1. engage with the python-vyper team to converge on a single language spec.
2. pick a new name.

No urgency if we're good with option 2 (even if we don't end up going that way). More urgency if we want to put in the political work to converge the language syntax.

EthPM, Standard Libary, and Packages ... Oh My!

What is wrong?

One of the powerful features of Python that I've often taken for granted is the standard library.

Similarly, packaging in general is something that isn't yet present in smart contract languages, and would be a compelling feature.

How can it be fixed

EthPM exposes a few different kinds of assets.

1. Source Files
2. Compiled Contracts
3. Deployed Contracts

In theory Fe can support interfacing with all of these, though the source file level is probably the most questionable since that would ultimately require dealing with negotiation of types of source files and compatibility of source files with whatever version of the compiler is being used.

We would also need to hash out what the actual syntax is for these imports. How would someone cleanly reference deployments vs contract interfaces in import statements? Exploration into this will be needed.

Source Files

Interfacing with compatible source files would simply be a matter of establishing a standard for where the Fe-lang compiler will look when attempting to import from an installed package. Nick did some work in the EthPM CLI tool for doing local installs. I don't recall if it was ever strictly specified but there is an established directory structure it uses which would probably be the appropriate starting point for mapping import statements to source files that have been installed this way.

Compiled Contracts

Fe-lang should be able to work with previously compiled assets, assuming they come with an ABI. These can be used to construct a definition of an external interface to define how we can interact with them. One might install a standard ERC20-interface package in order to import it and be able to interact with ERC20 contracts in a generic manner, without needing to embed the interface into the local code.

Deployed Contracts

This is likely the most compelling and complicated feature, since deployments are chain specific, meaning that compilation would have to be done under some sort of context for the chain it's being compiled for. In this version, you could actually import contracts that include both the interface, and the actual address that the contract is deployed to. This functionality would be the foundation for both library code (aka the standard library) and for actually importing and interacting with specific contracts.

For the Library use case, there might be a DateTimeLibrary contract out there which maybe exposes Date and DateTime types. This gives Fe-lang enough information to enforce type safety and understand the various methods that the different types expose. At compile time Fe-lang would need to either do a generic compile which leaves the resulting bytecode unlinked, or to compile against a specific chain which would link the bytecode against the address of the deployed contract.

Use of angle brackets for type parameters

I've been trying to decide on a syntax for map type definitions.

Currently, in python vyper, it looks like this: map(address, bytes[100]). This is another one of those cases where I think the syntax was designed somewhat to accommodate the python parser. I don't much like map(...) because it's not visually distinguished from regular call syntax i.e. fn(..).

An alternative to this could be to use the syntax that python has adopted in the standard typing module. This looks like GenericType[TypeParameter] and so map type definitions would then look like map[address, bytes[100]].

However, I still don't like this because, again, it's not visually distinguished from other common syntactic elements like array indexing or dimensions (bytes[100] as in the example above or some_list[i] to access the ith element).

This is all a question of how to design a syntax for specifying type parameters to some generic type. Both Rust and C++ have a syntax for this that is somewhat distinct from other syntax: GenericType<TypeParameter, ...>. If we used that, map type definitions could look like this: map<address, bytes[100]>. I sort of like this because it's clearly different from other syntax in python and it should already be familiar to people who have used languages like C++ or Rust.

I think such a syntax might also be useful for specifying other meta parameters for a type. This is maybe sort of a wacky idea, but perhaps adopting this syntax could open the door to things like the following:

type uint256 = int<bits=256, min=0, max=2 ** 256 - 1>
type int128 = int<bits=128, min=-(2 ** 127), max=(2 ** 127) - 1>
# up, down, left, right.  min and max could default to 0 and 2 ** bits - 1 respectively
type direction = int<bits=2>
type indexable_str = string<max_len=255>

I remember there was a discussion at one point concerning the way to specify the maximum length of dynamic-length byte arrays in vyper. Maybe something like this could be useful for that:

# Currently, in python vyper, this means a dynamic length array with max length
# of two.  However, this is a bit confusing since the same syntax has a
# different meaning in solidity and in many other languages.  So then let's
# just say that this defines a constant-length array...
bytes1[2]

# ...and then we'd instead specify dynamic-length arrays in this way
bytes1[]<max_len=2>

# ...or using a shorthands like these...
bytes1[]<2>
bytes1[<=2]

# ...which, internally, are re-written to...
bytes1[]<max_len=2>

I'll admit that the examples above are maybe a bit awkward or clunky. But the point is that an extensible syntax like an angle bracket parameter list (extensible in the sense that the list can have one or more items) might be useful in specifying meta info about types. And there already appears to be a need for this.

Compiling a method without parameters gives parser error

What is wrong?

This should compile but does not:

contract Foo:
    pub def bar() -> u256:
        return 0

How can it be fixed

🔪

Type checking for variable declarations and assignments.

The following todo stubs should be replaced with code that checks the validity of the nearby expressions.

• subscript assignments
• name assignments
• variable declarations

Support assert statements

Statements of the form assert <bool expression> should check if the expression is false and revert the transaction if it is.

• type check the bool expression
• map the assert statement to a Yul if statement that reverts if entered

Parser efficiency/semantic improvements

The current use of parser combinators provided by the nom library means that parsing is an O(n^2) time and O(n) space operation. Also, the lack of support for left-recursion means that parsing of things like x + y + z (where evaluation should happen from left-to-right) is a bit awkward and requires some post-processing. For example, parsing the above string with a grammar definition like this:

const_expr: const_term ('+' const_term)*

Means that the parse tree comes out in the wrong order and needs some post-processing before the order of evaluation is faithfully represented. This problem does not exist with a left-recursive grammar:

const_expr: const_expr '+' const_term | const_term

See here for more details.

Future versions of this project should modify existing parser combinators to memoize results and possibly also use memoization functionality to provide support for left-recursion (as is also described here). Or, an entirely different parsing library could be used that supports those features out of the box.

Auto generate Changelog

What is wrong

Manually writing a changelog is error prone and tedious.

How can it be fixed

Even if this isn't a Python project, I believe towncrier currently provides the best system to generate release notes.

If we want a pure rust solution then something like https://github.com/clog-tool/clog-cli can be an alternative (disclaimer: I'm the biased author) but I'm strongly in favor to use towncrier manly because:

• it decouples changelog entries from commits
• it is used in all other snakecharmer projects