A Tool for Scrutinizing OpenGov Proposals (Don't Trust, Verify) about runtimes

[xlc]

MVP is done https://xlc.github.io/chopsticks-web/

[liamaharon]

@PieWol is interested in working on this.

Probably a good place to start would be familiarising with the Chopsticks codebase and this proposal. Ask questions to clear up existing ambiguity about the spec and what we want from a V0.1 of this tool.

Once spec is clear, estimate how long it'll take and let us know an hourly rate you'd charge for this. Finally we can put forward a proposal to the treasury to fund the implementation and also the already done research/discovery work.

Obviously we cannot guarantee treasury will approve funding, but given a clear and reasonable proposal you should expect support from the fellowship to get this funded and implemented.

[xlc]

Some more points:

There could be multiple ways to deliver this:

1. A TS library that allow us to write quick script to invoke it. e.g. Using zx
2. A CLI tool and we use shell script to invoke it
3. A webpage and we have nice GUI

I suggest do 1 first, and we could build 2 or/and 3 based on 1. From UX point of view, 3 will be best as there is one less tool to be installed locally and anyone including non technical people can also use it.

The inputs should be something like:

type Input = {
  networkName: string
  referendumNumber?: number
  proposalHex?: string
  proposalHash?: string
}

And the tool should fetch the proposal from onchain if received a referendum number or the hash.

It should decode the proposal, and determine if it needs to be whitelisted first, and generate a whitelist call if needed.

And it should execute the call on the provide chain, display the storage diff, which will include the events.

It should detect if there is outgoing XCM generated from the proposal, and figure out the networks (using a hardcoded list), and launch a Chopsticks network for the dest networks, and run the XCM, and display the outcome.

It should highlight if a call is scheduled to be dispatched in future, and print out the block number and call hash and bytes.

And the tool should allow user to enter the number and call bytes to dry run the scheduled call.

There could be a mode to allow the tool to automatically do that, but need to handle recurring calls properly.

The output should be displayed in human readable format. Chopsticks already have the ability to decode and generate a nice HTML for viewing.

Some more pointers:

Dry run preimage implementation in Chopsticks https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/plugins/dry-run/dry-run-preimage.ts

And dru run extrinsic https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/plugins/dry-run/dry-run-extrinsic.ts

Generate html diff https://github.com/AcalaNetwork/chopsticks/blob/master/packages/chopsticks/src/utils/generate-html-diff.ts

storage diff decoder https://github.com/AcalaNetwork/chopsticks/blob/master/packages/core/src/utils/decoder.ts

There are two ways to interact with a Chopsticks instance:

• Run an instance and use JSON RPC. There are dev RPCs available to dry run or produce blocks. You can use whatever tools to interact with the instance. Useful for say if you want to use Rust to implement this.
• Create a Blockchain instance using setup and interact with the Blockchain instance. This unlocks the full potential of Chopsticks. Also you can do this in a webpage.

[xlc]

xcm is fully supported

[PieWol]

Thank you @liamaharon and @xlc for all the pointers.
Let's do the CLI first. I'll need some more time to familiarize myself with the Chopsticks code and to evaluate the effort of the Spec we come up with.

For the CLI it sounds like almost everything is already provided by Chopsticks.

Need to add:

• fetching of proposals
• check for whitelist calls
• event scanning (e.g. scheduled calls and Sent XCM events)
• ability to detect and execute scheduled calls
• a hardcoded list of networks that shall be supported
• automatically boot up a new instance of needed networks
• aggregate storage diff of all the used networks in a human readable format
• enable comparison between expected outcome vs. actual outcome

Given the spec item

Exposes local RPCs allowing devs to connect to and play with post-upgrade state using Polkadot.js

I guess we need to:

Create a Blockchain instance using setup and interact with the Blockchain instance. This unlocks the full potential of Chopsticks. Also you can do this in a webpage.

How about we also add "flagging concerning behaviour" to the Spec?
For example xcm calls are given from chain A to chain B and C resulting in B and C sending XCM calls back to Chain A within the same block time. Now the order of execution on chain A will be uncertain.

My apologies if this case is actually already defined and I just didn't know about it yet.

[h4x3rotab]

MVP is done https://xlc.github.io/chopsticks-web/

This is awesome! Tried on our parachain and it worked like a charm

[flipchan]

@xlc can chopstick verify XCM calls? that would be really useful on the sender and receiver parachains. Given the user knows the predicted output of the xcm call.

[xlc]

Given the spec item

Exposes local RPCs allowing devs to connect to and play with post-upgrade state using Polkadot.js

I guess we need to:

Create a Blockchain instance using setup and interact with the Blockchain instance. This unlocks the full potential of Chopsticks. Also you can do this in a webpage.

You can but you don't need to.

For example xcm calls are given from chain A to chain B and C resulting in B and C sending XCM calls back to Chain A within the same block time. Now the order of execution on chain A will be uncertain.

How is that relevant to this particular issue?

[PieWol]

How is that relevant to this particular issue?

Misunderstanding. It's not relevant.

[xlc]

Any progress on this? We had too many failed fellowship proposals. If no meaningful amount work is already done, I will just spend a few days to get something out.

[PieWol]

hey @xlc ,
My progress is here . Feel free to take over and start cooking. No idea what you mean by failed fellowship proposal as I am just doing this in my free time.

[xlc]

This is one failed fellowship proposal that could be avoided with this tool https://collectives.subsquare.io/fellowship/referenda/51

[xlc]

I am building a version at https://github.com/xlc/chopsticks-web

Currently it loads preimages, you can copy & paste it to the dry run panel and run it.

The diff viewer style is a mess but the core functionality works

[liamaharon]

This is awesome, thank you @xlc for this initial effort + detailing a roadmap.

Once the roadmap has been delivered and we add documentation for the tool and how to use it to check opengov proposals, I think we can close this issue.

Ideally the remainder of the roadmap could be delivered by a community member. I will ask around again.