VIP: stateful singleton modules with ownership hierarchy about vyper

[pcaversaccio]

Having intensively discussed the different tradeoffs with @charles-cooper over the last months I think this proposal is the "better" one of the two (#3723). The "ownership hierarchy" concept is pretty common in languages that deal with memory management and resource allocation (e.g. C, C++, or Rust) and has shown its virtues. I think this can be carried over to contract-oriented programming.

Some open questions:

• Python doesn't allow for private instance variables. I think the module design could benefit a lot from such a feature. Using a private variable allows encapsulating a module contract in a way that an importing contract can't change a variable in a bad way and the imported contract can expose a getter, or an internal setter if it's ok to modify.
• Proxies are completely oblivious to the storage trie changes that are performed by the constructor. Upgradability might be a desirable feature (unfortunately) for some use cases, and the __init__ logic using owns or seals can be circumvented since dedicated initialiser functions are usually called. What are your thoughts on this?
• Should it be possible to remove seals mode for certain specific functions? I.e. a more granular control access structure than completely switching off any writes from other modules.

[charles-cooper]

@DanielSchiavini suggests that ownership should be the default, whereas borrowing needs to be marked

[charles-cooper]

@DanielSchiavini suggests that ownership should be the default, whereas borrowing needs to be marked

as an example:

# library1.vy
import Library2

library2: borrows(Library2)

def __init__(self):
    pass

# contract
import Library1
import Library2

library1: Library1[library2]
library2: Library2

def __init__():
    self.library2.__init__()
    self.library1.__init__()

[charles-cooper]

Having intensively discussed the different tradeoffs with @charles-cooper over the last months I think this proposal is the "better" one of the two (#3723). The "ownership hierarchy" concept is pretty common in languages that deal with memory management and resource allocation (e.g. C, C++, or Rust) and has shown its virtues. I think this can be carried over to contract-oriented programming.

i kind of agree (at least at this moment in time -- the two approaches both have their merits and i have gone back and forth on them many times). typically when people put state in a module, they intend for it to be global. this is especially familiar for people coming from a python background.

put another way, the multiple instantiation paradigm is more elegant, but actually more error prone if you consider the global lock use case. it's too easy to forget to tie two instances together when the library designer intended for a piece of state to be global (which is the default design attitude for somebody who is writing a module).

Some open questions:

• Python doesn't allow for private instance variables. I think the module design could benefit a lot from such a feature. Using a private variable allows encapsulating a module contract in a way that an importing contract can't change a variable in a bad way and the imported contract can expose a getter, or an internal setter if it's ok to modify.

i think private variable declarations hurt composability. more generally, my current design philosophy is that importers should set constraints, not the importees. this design philosophy maximizes composability.

• Proxies are completely oblivious to the existence of constructors. Upgradability might be a desirable feature (unfortunately) for some use cases, and the __init__ logic using owns or seals can be circumvented since dedicated initialiser functions are usually called. What are your thoughts on this?

i will need to think about this more, but i think that delegatecall use cases are kind of orthogonal to this proposal.

• Should it be possible to remove seals mode for certain specific functions? I.e. a more granular control access structure than completely switching off any writes from other modules.

i don't think so. if a library designer or (probably more commonly) a contract author seals another module, i would assume they mean it.
EDIT: i misread this -- i don't think it's particularly useful to seal only specific parts of a module.

[pcaversaccio]

put another way, the multiple instantiation paradigm is more elegant, but actually more error prone if you consider the global lock use case. it's too easy to forget to tie two instances together when the library designer intended for a piece of state to be global (which is the default design attitude for somebody who is writing a module).

Exactly - generally, it's not a straightforward exercise to immediately understand the implications of multiple instances and stateful actions. Singletons optimise better IMHO for safety & reasonability, which again, is an important design principle for any Vyper feature.

i think private variable declarations hurt composability. more generally, my current design philosophy is that importers should set constraints, not the importees. this design philosophy maximizes composability.

Hmm, I don't think private variables hurt composability. On the contrary, applied correctly it can even be an enabler for better composability without shooting yourself. But I don't wanna abuse this issue for that discussion 😄; so let's stick with the as-is situation for the moment.

[charles-cooper]

@DanielSchiavini suggests that ownership should be the default, whereas borrowing needs to be marked

after ruminating on this for a few days, i favor a system which marks ownership and additionally requires annotation of write dependencies as is proposed in #3723. borrowship may also be marked, although it this is a relatively small detail and can be added or removed in the future. in the following examples, i renamed the keywords owns: and borrows to initializes: and uses:, respectively. so the above example would look like

# Library1.vy
import Library2 as library2

uses: library2

def __init__():
    pass

# contract
import Library1 as lib1
import Library2 as lib2

initializes: lib2
initializes: lib1[library2 := lib2]

def __init__():
    lib2.__init__()
    lib1.__init__()

more formally, the ownership hierarchy as exposed to the user is therefore:

- NO_OWNERSHIP
- WRITES
- INITIALIZES

as an implementation detail, i settled on using the walrus operator (:=) for dependency annotation, since the assignment operator (=) is not allowed inside of brackets.

as a larger example, i wrote up the token example using this syntax here: https://gist.github.com/charles-cooper/fb5caff4eee8bbf92ed86cefaa39a855

[fubuloubu]

How about require instead of uses (which is a weaker verb and a protected keyword in solidity, ensuring less of a need for it as a state variable name)?

Also, is initializes required when you end up initializing the module anyways?

Your example could look like:

import Library1
import Library2

def __init__():
    Library2()
    Library1(library2=Library2)

extends may also be a nicer word for initializes too

[charles-cooper]

How about require instead of uses (which is a weaker verb and a protected keyword in solidity, ensuring less of a need for it as a state variable name)?

i don't think we need to restrict ourselves to solidity protected keywords, we should rather choose the word which best represents the semantics. the biggest "dent" to UX (if you can call it that) here is that programmers won't be able to have state variables named uses. that said, just to have them all in one place -- the current possibilities/suggestions for the keyword names are:

• ownership:

  • owns
  • initializes
  • controls
  • extends

• borrowship:

  • borrows
  • uses
  • requires
  • writes

Also, is initializes required when you end up initializing the module anyways?

Your example could look like:

import Library1
import Library2

def __init__():
    Library2()
    Library1(library2=Library2)

i considered not requiring the initializes: statement. however, the initializes: statement serves two important benefits.

one, it lets the programmer control where the library goes in the storage layout. i think this is important, since the other options are to (somewhat arbitrarily) either choose storage layout order depending on import order, or where the initializations occur in the source code. this way the storage layout is clear from the order in which storage variable declarations and initializes: statements are laid out in the source code.

second, it allows compile-time resolution of Library1's dependencies at the ownership declaration site. that's one reason why i chose the bracket notation -- it looks like a compile-time type- parametrization.

the dependency resolution could be done in source code, but it starts to get weird once source code is not just straight-line, e.g.:

def __init__():
    if block.number % 2 == 0:
        Library2()
        Library1(param1, param2, library2=Library2)
    else:
        Library1(param2, param1, library2=Libraryyy2)  # probably a user typo, need to throw an error even though the first call to Library() is well-formed.
        Library2()  # how does this affect storage layout?

extends may also be a nicer word for initializes too

added to the list above, although i am not really a fan of extends as it has connotations for people coming from OOP (especially Java/C#) background.

[charles-cooper]

implemented in #3729