zeitgeistpm / zeitgeist

An evolving blockchain for prediction markets and futarchy.

Zeitgeist: An Evolving Blockchain for Prediction Markets and Futarchy

Zeitgeist is a decentralized network for creating, betting on, and resolving prediction markets, allowing traders to create complex financial contracts on virtually anything. The platform's native currency ZTG is used to sway the direction of the network, and as a means of last-call dispute resolution in the decentralized court.

Modules

• asset-router - Routes all asset classes to the respective pallets and provides a garbage collection for destructible assets.
• authorized - Offers authorized resolution of disputes.
• court - An implementation of a court mechanism used to resolve disputes in a decentralized fashion.
• global-disputes - Global disputes sets one out of multiple outcomes with the most locked ZTG tokens as the canonical outcome. This is the default process if a dispute mechanism fails to resolve.
• macros - Contains macros shared by the other modules.
• market-commons - Contains common operations on markets that are used by multiple pallets.
• neo-swaps - An implementation of the Logarithmic Market Scoring Rule as constant function market maker, tailor-made for decentralized combinatorial markets and futarchy.
• orderbook - An order book implementation.
• parimutuel - A straightforward parimutuel market maker for categorical markets.
• prediction-markets - The core implementation of the prediction market logic for creating and resolving markets.
• swaps - An implementation of the Balancer CFMM that allows any user to create pools, provide liquidity or swap assets.
• primitives - Contains custom and common types, traits and constants.
• rikiddo - The module contains a completely modular implementation of our novel market maker Rikiddo. It also offers a pallet that other pallets can use to utilize the Rikiddo market maker. Rikiddo can be used by the automated market maker to determine swap prices.

How to Build and Run a Zeitgeist Node

Zeitgeist node comes in two flavors, one for standalone self-contained execution and another for Kusama/Polkadot parachain integration.

To build the standalone version for testing, simply point to the top directory of this project and type:

cargo build --release

The standalone version uses the runtime defined for Zeitgeist's testnet Battery Station in runtimes/battery-station and is run in --dev mode by default.

To build the parachain version, execute the following command:

cargo build --features parachain --release

By default, the parachain version will connect to the Zeitgeist main network, which launched as a parachain of Kusama and has since been migrated to Polkadot. The runtime of the main network is defined in runtimes/zeitgeist.

To connect to Zeitgeist's testnet Battery Station, which runs as a parachain of Rococo, run:

cargo run --features parachain --release -- --chain=battery-station

Optimized binaries (--release) are usually used for production (faster and smaller), but this behavior is optional and up to you.

Using Docker

We publish the latest standalone and parachain version to the Docker Hub, from where it can be pulled and ran locally to connect to the network with relatively low effort and high compatibility. In order to fetch the latest docker image, ensure you have Docker installed locally, then type (or paste) the following commands in your terminal.

For parachain Zeitgeist node:

docker pull zeitgeistpm/zeitgeist-node-parachain

For standalone, non-parachain Zeitgeist node:

docker pull zeitgeistpm/zeitgeist-node

To connect your Zeitgeist parachain node using Docker, follow the tutorial at our documentation site.

Alternatively you can run a non-parachain node, which is usually only necessary for testing purposes, by executing the following command:

docker run zeitgeistpm/zeitgeist-node -- <node-options-and-flags>

zeitgeist's Issues

Review PredictionMarkets module

PredictionMarkets need a review now that it's sat for a little while.

Primarily I need to check that

• [] All logic is implemented.
• [] Make issues for any unimplemented logic.

Set up GitHub Actions for tests

Add reversion for the extrinsics in Swaps

Currently, the extrinsics in Swaps are written with the assumption that they will revert (like in Ethereum) if the checks don't all pass. However, this not the case for Substrate unless it is made explicitly so. We need to enable this feature on all the extrinsics in Swaps.

Handle the imbalance created in PredictionMarkets::reject_market

Store pallet parameter_types! in primitives crate

The pallet configuration of the Zeitgeist pallets in the runtime, test and benchmark environment currently uses hard-coded values. If one value changes, for example MaxAssets, some tests and benchmarks become invalid, leading to invalid checks of correctness and potentially invalid automatic generation of weights.

Extend the primitives create, such that it contains the pallet configuration values of the Zeitgeist pallets. Update the runtime, tests and benchmarks to fetch the values from the primitives crate

Upgrade to v3.0.0 Substrate

Yay! A new Substrate release.

Now we can refactor the code to use the new proc macros for runtime pallets.

Benchmark pallets

This is the overarching issue for both #51 and #7

Benchmarking the extrinsics in Substrate pallets allow us to understand their computation cost at different states in order to find a formula to price them accordingly. For information regarding benchmarking please consult the Substrate documentation.

Once we have set up the benchmarking, we can set up a cloud instance that we can run the benchmarks on in order to get a standard pricing.

Rename "zrml" to pallets

ZRML is an archaic name back when Substrate pallets were called runtime modules and there things like the SRML (Substrate Runtime Module Library) and ORML.

Now the ergonomic way to name things is simply "pallet" so we should migrate all zrml -> pallet namespaces.

deploy_swap_pool_for_market can corrupt the storage

zrml/prediction_markets/lib.rs @ fn deploy_swap_pool_for_market: calls create_pool which modifies storage and can return an error afterwards, thus leaving behind a corrupted storage. Use the attribute #[frame_support::transactional] or implement a variant that does not lead to such situations.

add an RPC endpoint for getting the market share id based on market id and outcome index

All shares on Zeitgeist are identified by the hash of the market id and the outcome index.

For example, in Binary markets there are 3 outcomes:

• Yes = 0
• No = 1
• Invalid = 2

The hash of market with an index of 3 for the outcome "No" would be blake2_256(3, 1)

Would be easiest to provide an RPC endpoint for this so we could call api.rpc.market_share_id(3, 1) (?)

or I can try it from the javascript

Add parameters to event documentation

The events currently only contain a description, missing the information about the parameters an event emits. Add this information to every event, following the example of the balances pallet from paritytech.

Implement Categorical markets

[Swaps] [RPC] Add an RPC endpoint that will return the price history for a range of blocks

In order to create a historical price chart on the front-end we will need a way to fetch the historic prices of assets that are held in swap pools. One way of doing this would be to add an RPC endpoint that returned a Vec<u128> of prices for a range of blocks. The advantage of housing this logic inside the node's RPC interface is so that it requires only 1 pass between clients.

Do not require users to wrap the native currency

Currently, in order to make the Balances currency compatible with runtime logic that uses the Shares trait we require that the native currency (ZTG) is wrapped as a "share". We need to remove this requirement and make the native currency from Balances able to use the same interface as the shares be default. One way to do this would be to use the ORML MultiCurrency trait.

Related to #26

Implement the advisory committee

Probably just clone the Collective pallet

Categorical markets with less than 2 options can be created

Issue: See title

zeitgeist/zrml/prediction-markets/src/lib.rs

Lines 249 to 298 in 1355f47

	#[pallet::weight(50_000_000)]
	pub fn create_categorical_market(
	origin: OriginFor<T>,
	oracle: T::AccountId,
	end: MarketEnd<T::BlockNumber>,
	metadata: Vec<u8>,
	creation: MarketCreation,
	categories: u16,
	) -> DispatchResult {
	let sender = ensure_signed(origin)?;
	Self::ensure_create_market_end(end)?;
	ensure!(
	categories <= T::MaxCategories::get(),
	"Cannot exceed max categories for a new market."
	);
	let status: MarketStatus = match creation {
	MarketCreation::Permissionless => {
	let required_bond = T::ValidityBond::get() + T::OracleBond::get();
	T::Currency::reserve(&sender, required_bond)?;
	MarketStatus::Active
	}
	MarketCreation::Advised => {
	let required_bond = T::AdvisoryBond::get() + T::OracleBond::get();
	T::Currency::reserve(&sender, required_bond)?;
	MarketStatus::Proposed
	}
	};
	let market_id = Self::get_next_market_id()?;
	let market = Market {
	creator: sender.clone(),
	creation,
	creator_fee: 0,
	oracle,
	end,
	metadata,
	market_type: MarketType::Categorical(categories),
	status,
	report: None,
	resolved_outcome: None,
	};
	<Markets<T>>::insert(market_id.clone(), Some(market));
	Self::deposit_event(Event::MarketCreated(market_id, sender));
	Ok(())
	}

Solution: Add range check 2 <= categories <= T::MaxCategories::get()

Benchmark PredictionMarkets

The PredictionMarkets pallet needs to get benchmarked.

Explore the use of anonymizing tech

Are there any use cases we can explore for ring signatures? I can think of two possible uses: ring signature for the oracles from a group of oracles in order to possibly prevent DoS or plausible deniability and ring signature for the reporter.

[Swaps] Do something with the exit fee

zeitgeist/zrml/swaps/src/macros.rs

Line 36 in 4f3e71d

// todo do something with exit fee

When someone leaves a pool, they can get charged an exit_fee (currently set to 0 on the testnet). However, we don't do anything with this exit fee and should pay it somewhere.

add more complete events to the Swaps pallet

The Swaps pallet does not have events being fired for all the functions, but it should. We want events to bubble up any important data that would otherwise require some difficulty to compute on the front-end or SDK.

For example, in join_pool we bubble up the event JoinPool but it only contains info about the pool_id and sender - things we should already know. We should add a new field here that also bubbles up the exact amounts of asset_amount_in.

We should also apply this same philosophy for bubbling up useful information in events for all the extrinsics in the Swaps pallet.

Change "Shares" to ORML tokens and multicurrency

This is quite the refactor, but I think should ultimately be done to align Zeitgeist to other Polkadot ecosystem standards.

Change the binary markets to be a categorical market with only two outcomes

We need to implement categorical markets, markets that can have up to 8 different outcomes attached to them. When we do this we should change the current Binary markets that are used to really just be a simple categorical market with only 2 outcomes.

related #15

Use FixedU128 for arithmetic in Swaps

See #36 for related discussion. It might make sense to migrate the arithmetic operations in Swaps to use Fixedu128 and checked math instead for extra safety. Doing this would involve using our own pow approximation which can be modified from the one we use now https://github.com/zeitgeistpm/zeitgeist/blob/master/zrml/swaps/src/fixed.rs#L50

determine what should happen to markets that are not reported on

Low TransactionByteFee enables low-cost DOS

Attack vector
This issue was already partially described in #83. The blocks can be filled with transactions that contain arbitrary data for relatively low cost. This is due to the configuration of TransactionByteFee, which is set to 1 per Byte (1e-10 ZTG). To give in an example of an well reviewed system, Polkadot: It uses a TransactionByteFee of 1,000,000 per Byte (1e-06 DOT).

Potential solutions
Determine a reasonable transaction fee per byte value and adjust it in the runtime configuration.

sell_complete_set can corrupt the storage

Issue: See title

zeitgeist/zrml/prediction-markets/src/lib.rs

Lines 687 to 696 in 1355f47

	for asset in Self::outcome_assets(market_id, market).iter() {
	// Ensures that the sender has sufficient amount of each
	// share in the set.
	ensure!(
	T::Shares::free_balance(*asset, &sender) >= amount,
	Error::<T>::InsufficientShareBalance,
	);
	T::Shares::slash(*asset, &sender, amount);
	}

Potential solutions: transact after every asset has been checked or use transactional attribute.

Automatically determine initial swap pool weights / set initial prices

Issue
Currently the creator of a swaps pool has to specify the initial weights (see here and here), which determine the price ratio between the assets. In addition to the circumstance, that deploy_swap_pool_for_market does implicitly add the market currency to the assets, but not extend the weights, i.e. expects the user to know that this implicit expansion happens and to manually add the weight, leads to an unintuitive and error-prone situation.

Potential solution
TBD. How do we determine the initial weights? Should the user be able to modify them?

Incorrect error string

zeitgeist/zrml/prediction-markets/src/lib.rs

Line 315 in 89a0e48

ensure!(now < end.saturated_into(), "End timestamp must be before now.");

This error string should say "End timestamp must be after now.". Or better, changed out for an Error type.

create diagrams for different processes

• market creation
• dispute process
• reporting process

implement disputes

Add the treasury pallet

The treasury will play an important part of Zeitgeist governance. It should get implemented ASAP

Add timestamp as an alternate to `end_block` when creating a market

Adopt a mission statement

https://missionprotocol.org/

Implement **simple** global disputes

This would later be replaced by the full-fledged court system.

Battery Park panicked at "Should never happen"

Mar 10 13:39:24.009  INFO 🙌 Starting consensus session on top of parent 0xfaf13721e8e471db4a34fd0a1e6c2221a8e13119021675c174d78b4193d59d3f
Mar 10 13:39:24.036 ERROR panicked at 'Should never happen', /home/volt/Code/zeitgeistpm/zeitgeist/zrml/prediction-markets/src/lib.rs:830:18
Mar 10 13:39:24.036  WARN Encountered consensus error: ClientImport("Execution(Wasmi(Trap(Trap { kind: Unreachable })))")  
Mar 10 13:39:25.478  INFO 💤 Idle (1 peers), best: #1015579 (0xfaf1…9d3f), finalized #0 (0x0959…d912), ⬇ 12 B/s ⬆ 12 B/s   

Separate the "court" from the "conditionals"

Inspired by Gnosis, we should separate the prediction markets pallet into two:

1. The Conditionals pallet will handle the creation of conditional markets. These conditions can be compose, merged, or split.
2. The Court handles resolutions and disputes.

Add a static weight to all extrinsics before testnet launch

We don't need to have all the custom pallets benchmarked yet, but we need to at least put a static weight on them to avoid getting spammed. Adding #[weight = 10_000] should be fine for now.

Implement Scalar markets

Use serde for custom structs/enums

Currently serde is included but not used in the pallets. In a std-environment, use serde to serialize and deserialize the data structures.

Add pre-commit hooks

By adding pre-commit hooks, whose usage is voluntary, it can be assured that certain checks and actions are performed before commits are accepted. For example, the hooks could ensure that the project is properly formatted by executing "cargo fmt" before every commit.

allow editing market details before it has started?

Once a market is in the "Active" phase it should not allow any edits to its metadata or end time. However, we might consider allowing creators to edit while it's still in the "Proposed" phase.

[CI] Add Docker builds for commits added to the main branch

We should add a CI job to build and publish Docker images on new commits to the main branch.

Implement the logic of PredictionMarkets::report_overdue

The function is unimplemented.

Battery Park panics with "denom is zero"

There's a bug in the math for get_spot_price that's tripping the panic. The panics there also need to be removed.

Some dispatchable functions allow arbitrarily sized vectors

Attack vector
Some dispatchable functions take an arbitrarily sized vector as at least one of their arguments, that is not checked before inserting it into the storage database. Transactions (that invoke normal dispatches, i.e. "user" functions) can include about 3,75MB of arbitrary data per block.
Since the fee for 1 Byte is one unit of currency and the weight calculation for storage insertions does not depend on the amount of data inserted, abusing this could lead to virtually free (in terms of transaction fees) storage population.

Having the option to fill the block in that cheap way introduces the next vulnerability: DOS attacks. Because including such a transaction will cost only a fraction of the function call itself using the current configuration (~3,75mil fee for arbitrary data inclusion in contrast to 50+ mil weight for function execution and therefore 50+ mil fee), the blocks can be filled virtually for free and thus stall the execution of other extrinsics. This is covered in issue #86.

Note: The fee per byte issue and the possibility to insert arbitrarily sized data structures in transactions are two separate issues that should each be tackled non-exclusively. Having a reasonable fee per byte would raise the cost, but still allow to insert arbitrary amounts of data into the storage.

Vulnerable functions

• prediction_markets/create_categorical_market
• prediction_markets/create_scalar_market

Potential solutions

• Introduce a maximum vector size that is checked during the execution of the function
• Use an array with a fixed size
• If the data contained within those vectors does not need to be processed in the substrate runtime, using an external reference to an off-chain storage is also a valid solution. For example, the data could be stored in IPFS and referenced by using the corresponding CID / Hash. An advantage this method offers is, that it enables the reasonable storage of relatively big amounts of data. A disadvantage is the complexity of implementing the solution, i.e. the front end code becomes a little more complex and a backend storage solution is required.

Upgrade to Substrate 2.0.1

determine what should happen to the shares of the outcome that didn't win

probably just programmatically destroy them

remove invalid as a tradable outcome

Add RPC for swaps

• get_spot_price
• pool_shares_id

Benchmark Swaps

The Swaps pallet needs to be benchmarked.