filecoin-project / ref-fvm

Reference implementation of the Filecoin Virtual Machine

Reference Filecoin VM implementation (v4; dev)

This repository contains the reference implementation of the Filecoin VM (specs). It is written in Rust, and intended to be integrated via FFI into non-Rust clients (e.g. Lotus, Fuhon), or directly into Rust clients (e.g. Forest). FFI bindings for Go are provided in-repo, and developers are encouraged to contribute bindings for other languages.

See the Project Website for details.

Build requirements

• Install rustup.

Build instructions

$ git clone https://github.com/filecoin-project/ref-fvm.git
$ cd ref-fvm
$ make

Code structure

Here's what you'll find in each directory:

• /fvm
  • The core of the Filecoin Virtual Machine. The key concepts are:
    • Machine: an instantiation of the machine, anchored at a specific state root and epoch, ready to intake messages to be applied.
    • Executor: an object to execute messages on a Machine.
    • CallManager: tracks and manages the call stack for a given message.
    • Invocation container (conceptual layer, not explicitly appearing in code): the WASM instance + sandbox under which a given actor in the call stack runs.
    • Kernel: the environment attached to an invocation container for external interactions.
  • There are two API boundaries in the system:
    1. the boundary between the actor code and the Kernel, which is traversed by invoking Syscalls.
    2. the boundary between the FVM and the host node, represented by Externs.
  • Some parts of the FVM are based on the Forest implementation.
• /sdk
  • Reference SDK implementation to write Filecoin native actors, used by the canonical built-in actors through the Actors FVM Runtime shim.
  • User-defined FVM actors written in Rust can also use this SDK, although it is currently quite rough around the edges. In the next weeks, we expect to sweeten it for improved developer experience.
  • Alternative SDKs will emerge in the community. We also expect community teams to develop SDKs in other WASM-compilable languages such as Swift, Kotlin (using Kotlin Native), and even Go (via the TinyGo compiler).
• /shared
  • A crate of core types and primitives shared between the FVM and the SDK.
• /ipld
  • IPLD libraries. Some of which are based on, and adapted from, the Forest implementation.
• /testing/conformance
  • Contains the test vector runner, as well as benchmarking utilities on top of it.
  • The conformance test runner feeds the test vector corpus located at https://github.com/filecoin-project/fvm-test-vectors into ref-fvm, in order to validate spec conformance.
  • The benchmarking utilities use the criterion Rust library to measure the performance and overhead of ref-fvm across various facets.
  • See the instructions about how to run the tests and the benchmarks.
  • Disclaimers
    • Benchmarks are currently very slow to run, setup and teardown. This is due to using default WASM cache, and will be fixed soon.

License

Dual-licensed: MIT, Apache Software License v2, by way of the Permissive License Stack.

---

actors and vm forked from ChainSafe/forest commit: 73e8f95a108902c6bef44ee359a8478663844e5b

ref-fvm's Issues

[Idea] Upgradable actors

One idea that @Stebalien and I chatted about today is introducing first-class support for upgradable actors in the FVM. This conversation forked from the "how do we upgrade system actors?" question.

Most serious smart contract operators on Ethereum use the proxy pattern to enable contract logic to evolve. There are many ways of doing this, here are some resources:

• https://docs.openzeppelin.com/upgrades/2.7/writing-upgradeable
• https://docs.openzeppelin.com/learn/upgrading-smart-contracts
• https://docs.nucypher.com/en/latest/architecture/upgradeable_proxy_contracts.html

Upgradability might be a bit polarizing. One could argue for either of these views:

1. Immutability of user-deployed code bound to addresses is a basic guarantee of blockchains, and should not be dropped.
2. Code mutability would be a beneficial first-class primitive to introduce at the protocol level; proof of that is that most serious contracts in other platforms resort to establish design patterns to achieve it in user land.

---

One solution would be to allow actors to implement an "Upgradable" trait, which implements the logic to authenticate an upgrade message.

Discussion: Milestone 0 Discussion of evm on wasm decision

Let's use this issue to discuss our decision to support evm on wasm decision:

• What's working
• What's different
• Any changes to our assumptions
• Outstanding items left to validate
• RESULT (go/nogo on supporting evm)

Feature Request: Dropping Privileges

The ability to "drop privileges" (i.e., send all new messages as some "nobody" account) could significantly reduce the risk of "confused deputy" bugs. Even if there isn't a built-in feature, there should be a documented pattern for doing this. This could be useful for, e.g., proxy/event contracts that need to call arbitrary methods on behalf of a user.

However, I'm having trouble thinking of any use-cases where one would want to send a message as "nobody" instead of as a specific unprivileged contract. For the latter, I'd expect the proxy contract to setup a some form of special "gateway" contract that sends arbitrary messages on behalf of the proxy contract and only sends those arbitrary messages (i.e., has no other privileges).

Migrate rust actors into fvm repo as-is

Revisit exit codes structure and semantics

The current VM has a set of system exit codes, reserved (unused) exit codes, and a hardcoded set of actor codes to signal common conditions (e.g. not found, forbidden, illegal state, etc.)

Code 32 and above are actor-specific codes; that is, application-specific codes.

See package: https://github.com/filecoin-project/go-state-types/tree/master/exitcode

We have to figure out how the exit code situation may change with the FVM, and how exit codes will be reflected in the IDL (#29).

@expede has linked to EIP-1066, an attempt to normalise errors in Ethereum by introducing an enumerated set of status codes (with categorisation inspired by HTTP).

---

Original discussion here: filecoin-project/fvm-specs#4 (comment)

specify and prototype FVM actor sandbox

Take an opinionated stab at implementing a prototype of the FVM actor sandbox with a "hello world!" custom actor, as a starting point for iterations.

pull in forest actors and vm

Create minimal fvm-sdk for canonical rust actors to use

Investigation: Using Wasm Reference Types

In addition to ints/floats, wasm now has a special "externref" type for referencing objects outside of the VM.

Possible use-cases:

• IPLD objects/blocks. This could be safer (and likely more performant) than using integers.
• Cross-module calling. E.g., a bigint library could return pointers to bigints as "externrefs" and know that the caller can't forge these references. This would need a bit of help from the runtime but it should be possible.
• Closures?

Pros:

• Unforgable: These could be great as capabilities. Except actually storing them in IPLD could get... tricky.
• Opaque: There's no way we'll introduce non-determinism into modules by passing handles.
• Garbage collected: If the actor has a reference, the object is live.
• They seem to be the "future" for referencing host objects.

Cons:

• Garbage collected: Wasmtime's garbage collector is deterministic, at least, but it's still complex. Luckily, any non-determinism would be in timing, and would not be visible to the Actor itself. I expect a "minimal" VM would just let all references live forever, and we'd probably be fine with that.
• No way to make references affine. For capabilities, being able to say "there exists at most one copy" of a capability would be nice.
• Cannot be stored in linear memory, they'd need to be stored in a WASM table and used indirectly. The language's compiler would need to be able to handle this.
• They're very new.

Conclusion: For now, I'm going to shelve this as it seems cool but too experimental. But I'm documenting it here in case we want to revisit it.

References:

• https://fitzgeraldnick.com/2020/08/27/reference-types-in-wasmtime.html
• https://github.com/WebAssembly/reference-types/blob/master/proposals/reference-types/Overview.md

Cleanup/merge rust-ipld related multiformats libraries

Motivation

We need a CID/multihash library for using both inside and outside of smart contracts, from rust. We currently have two options but neither of them are optimal.

Why Now

Unfortunately, these are base types around which we'll build the rest of the system, so we'd like to nail them down now instead of having to refactor later.

Done

We have a single multihash/cid library:

• Supports nostd.
• Lives in the github.com/multiformats org.
• Used by the ecosystem (i.e., PRs exist to update uses in forest & rust-libp2p).
• Ideally: Can be used without allocations.
• Ideally: Supports seamlessly abstracting over different sized CIDs.

Details

• All implementations:
  • Maximum size specified in the type (efficient in some cases, but it makes some patterns difficult).
    • WIP patch to support dynamically sized CIDs: lurk-lab/sp-multihash#8
  • Allocations are often required for encoding. Probably not something we can easily fix, but we may be able to do something.
• Ours (github.com/multiformats/rust-*):
  • Poor nostd support (leading to large binaries).
  • Uses typenum instead of the new const generics.
• Yatima's (github.com/yatima-inc/sp-*):
  • Uses a special "bytecursor" library that requires allocations.
  • Can't read/write using io::Read/Write.
  • Is a separate implementation (not used by forest, libp2p, etc.).

Parity Gas Accounting

Looking at substrate, it seems like they're doing gas accounting by instrumentation: https://github.com/paritytech/wasm-utils. This would allow using VMs that don't have native gas accounting support, and may be more performant.

Their logic will also perform stack height checks, etc. So we'll likely want to use it anyways.

Dynamic Module Linking

FVM actors will execute inside a sandbox that provides services such as state access, IPLD codecs, chain access, message sending, and more. Actor code will import FVM SDK modules in its source form (e.g. Rust actor code), but it is not clear to me how these libraries will be dynamically linked from the environment so as to not bloat the WASM bytecode with (redundant) statically linked libraries.

See https://github.com/WebAssembly/design/blob/main/Modules.md#imports.

Another angle to consider is the gas accounting from implicit libraries, and how that relates to the upgradability story.

1. We want to be able to upgrade the "actor sandbox" such as to deploy fixes or optimisations. What kinds of changes are consensus-breaking?
2. Is the sandbox code metered through standard gas? I'd assume no.

Feature Request: Circuit Breaker

There are often a lot of concerns around "actor bugs not being fixable without lots of consensus work" - I wonder if there's a way around this via a Circuit Breaker?

Initial (bad) idea:

• Filecoin has a set of elected, trusted parties that were much smaller than the consensus of the whole filecoin system.
• They could agree to publish one thing and one thing only - a version number, signed by all parties, below which an actor would simply NOOP, no matter what the contract. E.g. if the actor is 2.2.3 and the public circuit breaker is 2.4.1, the actor will always NOOP and finish - no gas, no computation.
• The initial published number starts at 0.0.0 and IFF a bug happens that we feel is severe enough, the group gets together and publishes a new version (e.g. 2.4.1) to the chain.

In the case of the circuit breaker, this will grind computation to a halt across the network, BUT it's a fall back in the case of very serious bugs. It would have to be an EXTREMELY tight group of folks - and this would be a means of last resort.

It, of course, breaks the idea of a purely federated trust model, but perhaps actors could ignore it with a command line flag? Unclear.

Implement the FFI layer for the FVM

merge initial v0 FVM architecture

PR in progress here: filecoin-project/fvm-specs#4.

Import forest vm into fvm repo as-is

compiling built-in systems actors to WASM

The built-in system actors need to be compiled down to WASM so they can run inside the FVM.

Ideally, we'd compile the Go actors specs-actors, but the Go toolchain doesn't produce succinct and constrained WASM (desirable in the context of blockchains) because it carries over the Go runtime. That's unfortunate because this is the most battle-tested actors implementation, and were built in the spirit of being an "executable specification" (hence their name), so it doesn't get more canonical than that.

There are two options to evaluate:

1. Switching to the Rust actors, extracting them from the Forest implementation first. Investing in this implementation to ensure they're on par with the Go implementation in terms of testing and auditing.
2. Using the TinyGo compiler, and maybe running post-processing to further trim down the output by pruning unneeded symbols.

Meta: Off-chain data

While users can coordinate storage of large datasets with Filecoin, actors can't directly access this state as it's not universally available.

However, that doesn't mean the data is completely inaccessible. We have several options for accessing state stored in Filecoin sectors:

1. Bounties.
2. Side-chains.

But the important question is: why? Access to off-chain data means actors can make decisions based on very large datasets that couldn't fit on-chain. This is primarily useful when most of the dataset will never be accessed by the chain.

In most cases, a viable alternative is to ask the message sender to provide the necessary data to complete the operation. However, this isn't always possible:

1. The message sender may not have ready access to the data.
2. The storage provider may be more willing to fulfill a bounty than to serve a retrieval request for a few blocks. A bounty can be fulfilled through normal Filecoin messages and doesn't involve directly serving data to a client.
3. The operation may not be directly triggered by a user message (we've discussed things like cron and async tasks in the past).

Feature Request: Data/Computation Bounties

Actors could post "bounties" to some bounty actor requesting some operation/lookup on a CID.

There are many ways to "fulfill" bounties but they generally fall into one of two main categories.

1. Verifiable results.
2. Verifiable computation.
3. Trusted Validators.

Verifiable Computation

Executing a "bounty" would always produce a corresponding "proof" of correct execution.

Halting problem note: careful gas accounting needs to be done to ensure the "always" part. That is, a proof of correctness may need to prove that the operation doesn't halt within some number of operations.

Data re-hydration

In data re-hydration:

1. The requested operation would be executed with access to the necessary state.
2. All touched data (accessed IPLD blocks) would be bundled up into a CAR file.
3. The CAR file would be sent to the bounty actor to re-execute the operation on-chain with access to the blocks in the CAR file.

The upside is that this method is very simple and efficient to compute (for the storage provider). The downside is that a potentially large amount message may need to be sent to the chain.

This variant is likely most useful for operations that need to perform storage-access efficient lookups on very large datasets.

Succinct Cryptographic Verifiable Computation

For example, the bounty could be executed under LURK and the resulting SNARK could then be validated by the chain. However, for this to work, the IPLD dag would likely need to be structured in such a way that computing a SNARK over it is efficient (e.g., hashed with Poseidon, not SHA).

1. Pro: User-friendly.
2. Pro: Storage providers can always prove that they executed the message, even if execution fails.
3. Pro: Efficient on-chain validation.
4. Pro: Small messages.
5. Con: Expensive off-chain computation.

Notes: To ensure that the computation halts, the code would need to be instrumented with gas accounting logic before compiling it into a verifiable computation.

Verifiable Results

Alternatively, bounties could "return" some arbitrary proof. This is a strict super-set of verifiable computation and is significantly more flexible. For example, an actor could:

1. Return a custom, efficient merkle-proof (potentially more compact than a simple trace of accessed IPLD objects).
2. Operate over data encrypted with authenticated homomorphic encryption.

Unfortunately, it may not be possible to prove that some execution failed. That is, storage providers may execute an expensive bounty, then fail to claim it because the proof fails to validate (a DoS vector). Solutions include:

1. Optimistic verifiable results, falling back on verifiable computation to prove that an actor failed to produce a valid proof. When combined with a large deposit, this would remove the DoS vector.
2. Reputation and bounded execution.

Trusted Validators

In the trusted validators model:

1. The party fulfilling the bounty (likely the storage provider) would execute the bounty and perform the data re-hydration described above.
2. A set of validators would then fetch the re-hydrated data, validate the execution, and certify the results on-chain.

The primary downsides are:

1. The need for some set of trusted validators. A mitigating factor is that the client/provider can agree on the set of validators up-front.
2. The storage provider may need to send a significant amount of data to the valdiator.

Switch canonical rust actors to consume fvm-sdk

Switch current prototype to Wasmtime

Investigation: Using Wasm Type Interfaces

• https://hacks.mozilla.org/2019/08/webassembly-interface-types/
• https://github.com/WebAssembly/interface-types

Basically, this is an attempt to make a "better" FFI for cross-language interop. Importantly, it tries really hard to avoid serializing/converting types.

I'm wondering if there's a way to "ipldize" it to get the best of both worlds.

analyse SputnikVM in detail as the basis of the EVM foreign runtime

To finalize our architectural design, we need to fully grasp how flexible SputnikVM in its current form in order to inject our own memory model, storage hooks, address translation, syscalls, etc.

From perusing NEAR Aurora's engine code, there seem to be some pretty well-defined hooks to customise the glue layer between the EVM sandbox and the outer layer (FVM).

There are two traits: Backend and ApplyBackend. The former loads values from the environment, and I suspect the later is called once processing is done to hand off changes that resulted from the call.

https://github.com/aurora-is-near/aurora-engine/blob/008e34cdf87ed1dfe87ba174af6a2bdc38dcfb5d/engine/src/engine.rs#L890-L1076

We need to do a deep dive into this to fully scope what the integration will entail, as well as to identify any uncertainties, risks, and blockers ahead of time.

Links

https://github.com/rust-blockchain/evm
https://github.com/aurora-is-near/sputnikvm

Investigation: Capabilities

Capabilities

A capability is an "object" that grants permission to perform some set of operations. Usually, it is
an object that is acted upon to perform said set of operations. This is in contrast with the more
standard "ambient authority" approach were, e.g., a program acts on some global resource, and the
program's ambient authority (user/group/etc.) is checked against some access control list for the
global resource.

Concretely:

• In an ambient authority system, program A may tell program B to "open" a file at some location. If
  program B isn't careful, it could be tricked into opening a file that program A shouldn't be able
  to access (confused deputy). For program B to open the file, it has to first atomically check if
  the user running program A has access to the file.
• In a capability system, program A will open the file then pass the file descriptor (which is a
  capability) to program B.

The primary benefit of a capability system is that it's easier to reason about security without a
bunch of manual checks.

Background

See wikipedia.

Some examples:

• File Descriptors: A file descriptor grants access to the described file. On most operating systems, file
  descriptors may be transferred between processes, even if the receiving process would not
  otherwise have access to the described file.
• Secret Keys: More precisely, a capability system may be built out of secret keys where
  transferring a key transfers the capability. This type of capability is often called a bearer
  token (knowledge implies permission).
• Objects (called an Object-Capability system): In "safe" languages, objects can be used as
  capabilities where having a reference to an object grants permission to operate on the object.
  • Rust famously allows exclusive/mutable references. These can be seen as capabilities.
  • Microsoft once wrote an OS in C#.
    using the built-in type system to enforce security (instead of the more traditional process/VM
    approach).
• URLs: If knowing a URL grants the bearer (user who knows the URL) a set of capabilities
  (permissions) on a web object, that URL is a "capability". That is, Google Drive's capability system
  is based on URLs.

Some desirable features in capability systems (likely with incorrect terminology):

• Transferable: Capabilities are almost always transferable.
• Conditional: Good capabilities allow imposing additional "conditions". For example:
  • A capability may only be valid during some period of time (may expire).
  • A capability may be combined with a hierarchical permission system to limit who may use the
    capability. For example, Google Drive allows one to generate capability (URL) for a doc that
    may only be used by members of an organization.
• Affine/Linear Capabilities: This type of capability is necessary for, e.g., at-most-once callbacks.

Motivation

A capability system would allow FVM to offer a fool-resistant "framework" for thinking about and
delegating permissions.

Resolution

Punt. For capabilities to work best, the entire system needs to be built around them. This would be a significant departure from the norm.

There's still room for building capability-like systems (e.g., #783) on-top-of the FVM but trying to switch tracks to a complete capability-based VM at this point is likely a non-starter.

Epic: A minimal FVM and SDK

Create a minimal FVM that implements the key concepts in the architecture doc, concretely the Domains, Boundaries, Invocation Container, Syscall handlers, and the Call Stack Manager.

Former Runtime interface: https://github.com/filecoin-project/specs-actors/blob/master/actors/runtime/runtime.go

Breakdown

• #11
• skeleton of FVM to enable divide-and-conquer
• #62
• implement state IO (blockstore)
• implement invocation container
• implement call stack manager
• implement cross-actor sends (via the call stack manager)
• implement exit codes and aborts
• #201
• implement actor creation
• implement actor deletion
• implement randomness access
  • from tickets
  • from beacon
• implement address resolution
• implement caller validation
• implement contextual getters
  • getter for network version
  • getter for base fee
  • getter for current epoch
  • getter for circulating supply
• implement bridging with current crypto syscalls
• export FVM entrypoints to be consumed from node implementations via FFI (actually part of #58)

prune unnecessary dependencies

• actor_interface depends on all of libp2p just for the PeerID type.

[idea] FSI: Filecoin Standard Interface

Developers will build a lot of things on Filecoin. Since early on, we want to encourage a culture of composability and reusability. One way of achieving this is to create a corpus of "Filecoin Standard Interfaces". We expect these to be defined by the community, but frontloading a bunch of pre-existing interfaces would be useful.

These initial interfaces could be extracted from two reusable components that exist in the protocol today:

• Payment channels
• Multisig wallets

The protocol shouldn't care about the implementation, as long as they adhere to a specific interface and behaviour.

In the future, we expect FSIs for:

• data bounties
• markets
• futures
• replication workers
• etc.

bulk state upgrades on protocol upgrades

We should have a strategy to define how to upgrade system actors on protocol upgrades.

---

Hint: this likely does not integrate/interact with #717.

[WIP] Capabilities

TODO

[Idea] IPLD-based Interface Definition Language

FVM actors expose a single entrypoint/mailbox, however in practice they expose multiple behaviours to callers. Callers need to know how to pack/template messages and their arguments to invoke those behaviours. Furthermore, they need to know what to expect in return.

Essentially actors need a way to express their interface in a standard form, which includes both the data model (schema) as well as the behaviour model. This is the role of an IPLD-based Interface Definition Language.

This issue is a placeholder issue to:

1. Analyse, design, prototype, and implement such IDL
2. Its integration with the FVM by (potentially) attaching IDLs to actors in the state tree
3. Designing ways to update/evolve the IDL on-chain
4. Offering SDK APIs for actors to compose and interpret calls according to an IDL, including proxy generation and other developer tooling.

---

References:

• WSDL: https://www.w3.org/TR/wsdl.html
• gRPC IDL: https://grpc.io/docs/what-is-grpc/core-concepts/

finish state of the art VM analysis

For informational purposes. These are the VMs we consider relevant to FVM. Good to have a sense of what they're doing and how they're approaching certain aspects (e.g. WASM/LLVM runtime, EVM support (or not), cross-chain interoperability, async calls, cross-shard calls, state growth) to potentially find room for collaboration.

• NEAR + Aurora (EVM for NEAR)
• Polkadot/Substrate
• CosmWasm (Cosmos)
• SVM (Spacemesh)
• Arwen VM (Elrond)
• Aqua VM (Fluence)
• Mokoto (Dfinity)
• ETH2 execution environments/engines

decide on high-level SDK sugars for method dispatch

Discussion originally starts here: filecoin-project/fvm-specs#4 (comment)

set up CI

Phase 0 burndown list

• Deliverable: Canary FVM implementations running built-in system actors.
• Goal: mainnet gradually running on non-programmable FVM.
• Network impact: consensus non-breaking.
• Delivered by: end of Q4 2021. Jan 2022 (see comments).
• Rough breakdown:
  • Set up filecoin-project/fvm with Steb’s prototype @Stebalien (private for now, while it's incubating)
  • Switch current prototype to Wasmtime @Stebalien
  • Import rust actors into fvm repo as-is @raulk
  • Import forest vm into fvm repo as-is @raulk
  • Create minimal fvm-sdk for canonical rust actors to use
  • Switch canonical rust actors to consume fvm-sdk
  • Compile canonical rust actors to WASM
  • Bind WASM bytecode to built in addresses
  • Evolve the rust VM into the WASM-based FVM
  • Implement the FFI layer for the FVM
  • Integrate the FVM into Lotus via FFI callout, behind a feature flag
  • Instrument WASM execution and emit metrics to a centralised location

Make rust actors use multiformats CIDs

Currently, they use forest CIDs.

Investigation: Use Yatima's IPLD

Well, originally from substrate but extracted into separate crates. This implementation works without the standard library so it should work much better for our use-case.

Eventually I'd like all these implementations merged.

• https://github.com/yatima-inc/sp-cid
• https://github.com/yatima-inc/sp-ipld
• https://github.com/yatima-inc/yatima

Feature Request: Side-Chain Data Access

This is a variant of the "trusted valdiators" approach in #780.

Given some form of sharding/side-chain feature, storage "pools" (groups of storage providers) with high-bandwidth links to each other could make sector state available to actors.

First, let's start with a simple synchronous execution model:

1. Some actor A1 on some chain C1 sends an async message M1 to some actor A2 in some storage pool chain C2.
2. Actor A2 would be "owned" by some storage provider S1 in the storage pool.
3. When S1 next mines a block, it would process message M1 making the relevant sectors available to the VM. A2 would then send a message back to A1 (or some other actor).
4. When propagating the block, S1 would push the accessed IPLD blocks to the rest of the storage providers for validation.
5. Finally, C2 would be stamped back into C1, and the results of the computation would become available.

Unfortunately, there are some downsides to this approach:

1. An actor/message can only reference sectors stored by a single storage provider in the pool.
2. Step 4 may forcibly "push" data that other storage providers in the pool already have cached.

An optimization would be use something like the multi-epoch async execution system described in #779.

1. A1 would send M1 to A2.
2. All storage providers would asynchronously execute M1 on A2, fetching data from members of the storage pool as necessary. This async execution may take many epochs and only completes when all participants are done (recorded via on-chain signaling).
3. The results would be committed and subsequent messages sent. E.g., an message (M2) back to A1.

Given the semi-trusted environment in the storage pool, waiting a potentially arbitrary amount of time before "committing" the result of the operation is likely acceptable.

Feature Request: Closures

Use-cases:

1. Async callbacks (and, in general, asynchronous messaging).
2. Programmable conditions. E.g., "this voucher is only valid when....".
3. Capabilities. A closure can be used to grant transferable access to some privileged endpoint by closing over the current "authority" in addition to some state.

Actor to actor

On-chain, this is relatively easy. The naive implementation would be to keep a table mapping closure "handles" (numbers) to saved state, the closure's function, and the actor that's actually allowed to call the closure.

Let's assume that actor A is sending a closure to actor B:

1. To create the closure, actor A would add the closure information (along with actor B's address) to this closure table, resulting in a closure handle H.
2. Actor A would then send H to actor B.
3. To call the closure, actor B send a message back to actor A with the closure handle H and some parameters.
4. Actor A would lookup and call closure H, verifying that actor B is "authorized" to invoke the closure.
5. If the closure is "run once", actor A would remove it from the table.

But what if actor B needs to send a message to some actor C, and wants actor C to invoke actor A's closure? In this case, we have two mostly equivalent options:

1. Provide some way for actor A to delegate/forward the closure. In this case, actor B would need to tell actor A to authorize actor C to call H.
   1. Pro: Actor C can now directly call the closure on actor A.
   2. Con: Actor B needs to call actor A when forwarding the closure.
2. Create a new closure.
   1. Pro: Actor B can forward the closure without involving actor A
   2. Pro: Actor B can "wrap" the closure in it's own logic.
   3. Con: When C invokes the closure, it'll have to call B which will, in turn, call A.
   4. Con: Actor B needs to "remember" the closure it gave to C.

There are still two open questions here:

1. What about pure functions? Calling back into the original actor just to execute a pure function is expensive. Furthermore, tracking these pure functions could limit their usefulness. It would be nice to be able to dynamically load up some WASM module and invoke a function on it.
2. Garbage collection is tricky. There's no good way for actor A to know that actor B has "dropped" the closure.

User to Actor

It would be really useful to specify complex conditions (ideally as pure functions) in messages sent from the user to the chain. This could be done in multiple steps (load the condition onto the chain, then send a message that refers to that condition) but it would be convenient, and likely cheaper, to be able to include a small lambda inline in messages.

IPLD state objects with schema CIDs

Explore the idea of having actor contracts specify the CID of the schema that encodes their state.

Advantages:

1. Makes the state introspectable from the outside (e.g. state explorers) -- this is a major painpoint in the Ethereum tooling ecosystem.
2. System actor migrations can be implemented outside of WASM by patching the state objects.

Precompiles & Wasm Modules

The EVM has two features to allow extension with native code, usually for performance:

1. Custom opcodes: The EVM makes it relatively easy to add new opcodes. These are usually used for new crypto primitives, etc. They're more difficult to implement than precompiles, but get to skip the calling overhead.
2. Precompiles: Special contracts with network-defined calling costs that are implemented in the client's native language instead of in the EVM. These are easy to implement, but introduce some overhead because they need to be called like any other contract.

In Wasm, we have two options:

1. Runtime provided function calls (same as any other syscall).
2. Precompiles (same as any other actor).

Given that we're planning on allowing deployed Wasm modules to import other deployed Wasm modules, option 1 likely makes the most sense.

However, we can take this even further. Given that LLVM can compile Wasm to native code, we can:

1. Try out new user libraries and see what works and what doesn't.
2. If we find that a specific library could benefit from running natively, the network can designate that library as a "precompile" and specify the gas cost.
3. Implementations can then compile the Wasm contract down to native code. In practice, this may not help much as the Wasm VM will likely run at near native performance.
4. Implementations can replace the native code with optimized assembly, if desired.

Importantly, this means promoting a library to "native" doesn't require a full network upgrade. At worst, some implementations may start syncing more slowly if they don't follow steps 3 & 4. However, even an implementation running 10x slower should be able to stay in sync.

prototype with various WASM engines

• wasmer
• wasmtime
• wasm3 (interpreter)

potential gas rearchitecture

User-programmable actors will substantially alter the mechanics of actor execution, as well as the dynamics of messages/transactions on-chain, as well as that of the fee market.

The gradual rollout plan is as follows:

1. Phase 0: FVM on Lotus supporting only system actors. Gas continues to be charged explicitly at syscalls following the current gas schedule/structure.
2. Phase 1: Adapt the gas architecture/schedule to account for real WASM execution costs of actors.
3. Phase 2: Enable user programmability.

This issue is an epic to discuss and track:

1. What execution/gas instrumentation/tracing do we want to include in Phase 0 to collect as much real-world data as possible to feed into the subsequent gas model?
2. Systematic analysis of gas system impact of user-programmability, and cross-interactions with system messages. Systematic solution design.
3. Choice of path forward.
4. Calculation of new gas schedule according to new architecture/policy.
5. Implementation of the new gas system.
6. Test and validation/confirmation using a collection of user-defined actors.
7. Rollout.

@Kubuxu would like to help with much of this. At the very least, we should start thinking about (1) ASAP, so we can factor it in the implementation for Phase 0.

Feature parity with Agoric

We should make sure we can do everything Agoric can do, but without depending on JavaScript. They have a lot of good ideas: https://agoric.com/documentation/.

They've thought through:

1. Async versus sync calling (where actors within a VAT can sync call, and async calls can be made between VATs).
2. Capabilities, futures, etc.

Define how Wasm bytecode will be represented as IPLD data

Define how code (WASM, EVM) will be represented in IPLD.

Figure out how to deal with floating point operations

The plan is to forbid floating point operations until making them completely deterministic is well understood and tested. Unfortunately, almost all standard libraries include some floating point operations. Ideally the optimizer would just remove any reference to floating point operations, but we can't necessarily rely on that.

Options:

1. Take a hard-line and forbid floating point operations entirely. Unfortunately, if this instruction happens to get compiled into the program in some "dead" section, the actor will get rejected on deploy.
2. Trap on floating point operation and raise a runtime error. This will provide less ahead-of-time safety, but'll make the system a lot easier to use.

Bind WASM deployables to built in addresses

Exploration: Multi-epoch, parallel, async actors

The largest problem plaguing blockchain systems is throughput. While sharding is the most general solution, it:

1. Doesn't take advantage of potential parallelism on within a single machine. While a single machine can execute multiple shards, cross-shard communication introduces additional overhead.
2. Doesn't allow for operations that take longer than a single epoch. While intermediate state can usually just be re-serialized, this isn't always convenient.
3. Requires accurate GAS estimation up-front.

However, async actors can make multi-epoch, efficient(ish), flexible, parallel execution possible. This kind of execution model is likely most useful for high-throughput side-chains and shards.

The basic idea is to schedule messages as "tasks" (messages applied to special async actors) on multiple "lanes". Each async actor would:

1. Execute at most one message at a time.
2. Be unable to synchronously call any other actor (at least while executing as a task). Or, at least, be unable to make a synchronous call that mutates state.
3. Yield a set of outbound messages on return.

Each message would specify:

1. Some maximum gas budget where the gas budget. Likely limited to a few blocks.
2. Some expiration epoch after which the message will expire without executing.
3. Possibly some minimum epoch? This could be used as a form of "cron".

There are two general approaches to schedule tasks: per-actor queues or a message pool.

1. The queue model is the standard causal actor model. That is, messages are dropped into a per-actor queue and executed in FIFO order. After processing a message, all "outbound" messages are deterministically enqueued into the destination actors in-order.
2. The pool model is what we currently use for normal messages. All messages would enter into a single pool and would be scheduled based on some form of bidding system.

The queue model is much nicer to program against, but may be expensive and/or easily DoSed because it precludes bidding. The pool model allows bidding and message pricing, but is harder to program against as messages can be executed in any order and/or may be dropped.

One possible solution is a mixed model where both bounded queues and pools are available.

WASM bundle strategy for built-in actors

We will soon be having actors code that depends on the fvm sdk, compiled to WASM targets. Questions:

1. Do we want a WASM bundle per actor and version tuple?
2. Do we want a monolithic WASM bundle?

Feature Request: First-Class Events

An event is effectively the inverse of a method:

• Method: The caller calls a method defined by the callee.
• Event: The sender sends an event defined by the sender.

In the current Filecoin actors, an actor A can register for an event from actor B and ask to be called back on a specific method (usually with some params). However, if not done carefully, this can be used to trick actor B into invoking an arbitrary method on some privileged actor.

With first-class events, there would be no confusion.

There are a couple of possible implementations:

1. A new wasm-level function and a new send primitive. This, unfortunately, extends runtime interface.
2. A "receive event" endpoint (kind of like "send"). In this case, we'd probably want to reserve a set (1024?) of methods.
3. A top-level field in the parameters to specify the type of message.

The event would then need to specify:

1. Some event identifier/number (relative to the sender). This is analogous to a method number.
2. The event parameters.

Compile canonical rust actors to WASM