Comments (28)
Kusama is already done. Polkadot should be finished after the next runtime upgrade.
from runtimes.
Ultimately another good check is to run a warp sync (if the state is not migrated warp sync should not be working).
I just tried and it still works.
from runtimes.
Will do (tomorrow most likely), I think since block time is longer on polkadot, keeping the same config as kusuma should be fine.
from runtimes.
@cheme could you please post some status on what needs to be done for Kusama? Aka does the migration works on its own or is that driven by some offchain thingy?
After Kusama was successful, we should also directly start doing it for Polkadot.
from runtimes.
Looks ready to go with next kusuma runtime.
This line set at 1 (switch to hybrid state when new runtime released) :
runtimes/relay/kusama/src/lib.rs
Line 146 in 94b2798
Start of migration added to Unreleased
runtimes/relay/kusama/src/lib.rs
Line 1731 in 94b2798
Unrealeased set as runtime migration https://github.com/polkadot-fellows/runtimes/blob/94b2798b69ba6779764e20a50f056e48db78ebef/relay/kusama/src/lib.rs#L1654C35-L1654C35
Not that if there is many migration runing together, might be an idea to lower the limit per block:
runtimes/relay/kusama/src/lib.rs
Line 2714 in 94b2798
here up to 4800 item or 408000 octet which in case of max out blocks can add :
db_weight_reads_writes(1, 1) = (25_000 + 100_000) * 1000.
(using rocksdb cst).
125_000_000 * 4_800 + 408_000 * 1_139 + some_fix_weight
600 * 10^9 + 464_712_000
out of 2_000_000_000_000, so if I check correctly about 1 third of a block weight.
Note that in case of relay chain, consuming weight is not a must have from my point of view.
runtimes/relay/kusama/src/lib.rs
Line 1468 in 94b2798
Warning if the start line get remove from unreleased, the line 146 must be set to 0 (to avoid hybrid state).
from runtimes.
@bkchr the weight use in each block is an important point that should be in release note I think.
As a relay chain we could also ignore this weight to prevent any issue.
from runtimes.
actually I did use 2 * 10 ^ 12 for block weight but on relay it may be 6*10^12 so would not be worrying then
from runtimes.
The migration will run entirely on chain and doesn't require any external interactions?
from runtimes.
chain and doesn't require any exte
on chain, no possible external interactions.
from runtimes.
Kusama is done? I queried StateTrieMigration.MigrationProcess
and got:
{
"progress_top": {
"name": "Complete",
"values": []
},
"progress_child": {
"name": "ToStart",
"values": []
},
"size": 239581990,
"top_items": 881284,
"child_items": 1373
}
The Events also look fine. Just ~480 blocks to migrate everything? Looks like it did about 4800 keys in some blocks, nice 😳
from runtimes.
4800 was the limit indeed.
This rpc can be call (paritytech/cumulus#1424) to double check the state did migrate correctly (requires runing it locally as it is unsafe).
Ultimately another good check is to run a warp sync (if the state is not migrated warp sync should not be working).
(cannot run these right now I am ooo)
from runtimes.
@cheme so did it finished successfully?
from runtimes.
Yes, but if warp sync did pass, all is fine.
(warp sync is a guaranteed fail during migration, and the counter on chain do state the migration is finished).
from runtimes.
@cheme can you prepare the changes for Polkadot?
from runtimes.
Ping @cheme
from runtimes.
BTW, we also need to migrate the system chains.
from runtimes.
#170
systems chain will be another beast.
I was never really happy with the rpc calls process manual migration, last time I thought about it I was thinking of just running the automatic process, maybe with an exclusion list (first scan offchain for big values and process them one by one, but then there is still the issue of big values being added to the chain between the scan and the start of the migration, but realistically speaking and with a bit of knowledge of the system chain logic we can probably be confident this would not happen (big problematic values should be rather rare and no sane chain would create them randomly).
But the manual rpc approach can probably do the job, just the amount of energy to manage that worries me (and also the fact that we rely on a dedicated external trusted entity).
With automatic, we need to contact someone competent to know where can be big problematic values, do a scan of the state to find the existing ones and add a skip list of them ( actually a process first then skip list).
For this automatic approach new code would be needed in the migration pallet (the skip list related code) though.
cc @kianenigma
from runtimes.
Ohh fuck, have overseen this! Sorry!
I was never really happy with the rpc calls process manual migration
I thought this was also just some bot doing this? Or what you mean by manual in this case?
from runtimes.
I thought this was also just some bot doing this? Or what you mean by manual in this case?
yes manual by a bot, still need to run the bot (need slashable fee deposit, also I am not sure anymore if it should target a specific account (seems like a liability: should be open to everyone, but I don't remember how we ensure a single call is done per blocks).
Maybe it is fine.
from runtimes.
I mean just opening this for one account sounds fine to me. I mean we are speaking here about a one time migration.
but I don't remember how we ensure a single call is done per blocks
This could be done with a storage value that is set to true
when the call was done and removed at on_finalize
.
from runtimes.
Kusama is done? I queried
StateTrieMigration.MigrationProcess
and got:{ "progress_top": { "name": "Complete", "values": [] }, "progress_child": { "name": "ToStart", "values": [] }, "size": 239581990, "top_items": 881284, "child_items": 1373 }
The Events also look fine. Just ~480 blocks to migrate everything? Looks like it did about 4800 keys in some blocks, nice 😳
So the kusama state is 240 MiB? interesting. I am still missing a tool like paritytech/polkadot-sdk#449, I wonder if there is an ecosystem tool for this that I am not aware of?
from runtimes.
@cheme I would not use the automatic migration on a system chain, as any error will likely cause the parachain to stop.
Maybe on a kusama system chain, but 100% not for Polkadot. I wrote a TS bot that still should work fine to trigger the migrations one by one, and it should all be free. Have we ever used the signed migration? ref: https://github.com/paritytech/polkadot-scripts/blob/master/src/services/state_trie_migration.ts
from runtimes.
@cheme I would not use the automatic migration on a system chain, as any error will likely cause the parachain to stop.
Maybe on a kusama system chain, but 100% not for Polkadot. I wrote a TS bot that still should work fine to trigger the migrations one by one, and it should all be free. Have we ever used the signed migration? ref: https://github.com/paritytech/polkadot-scripts/blob/master/src/services/state_trie_migration.ts
Long time ago we did some with @PierreBesson when doing rococo and westand (I think statemine). But for me it is a bit too long ago.
from runtimes.
After Polkadot is done, we need to work on the parachains. As said above, I don't see any real problem in using the offchain bot.
from runtimes.
I added a list to the issue description. Please tick of the ones that are done.
from runtimes.
@cheme can you please also add it for the missing ones?
from runtimes.
@cheme can you please also add it for the missing ones?
is there new ones? (I mean asset-hub and collectives were done).
from runtimes.
Yea there are some remaining it looks like. Asset-Hub Kusama has it behind a feature-gate that was never enabled it looks like?
from runtimes.
Related Issues (20)
- Remove emulated test macros HOT 3
- [Ambassadors] Reduce `MaxRank` to 3 HOT 1
- `Fail when author pushes new code` should check if the user is not a fellow HOT 1
- Optimistic Project Funding HOT 22
- Implement Unbrick Collective HOT 3
- [CI] Publish unoptimized runtime blobs for debugging HOT 2
- [People] Add new registrar HOT 11
- PeopleKusama, PeoplePolkadot and Encointer don't have price for parent delivery set up HOT 2
- Add fee estimation support for XCMs over P<>K and P<>E bridges
- Treasury Configuration(s) for Multi-Chain Spends HOT 1
- [bridges][AHs] Include ExportMessage fees in XCM message from AH to BH HOT 2
- Update CI for Polkadot People Chain HOT 2
- Remove `chain-spec-generator`
- Check `merge bot` and required pipelines HOT 8
- Add test for coretime Runtime API HOT 1
- Coretime Runtime API: Refactor `burn_at_relay`
- Review bot error: Reviews may only be requested from collaborators HOT 6
- Field `image` on people chain is unusable HOT 5
- Enable StorageWeightReclaim for all the runtime HOT 2
- Consider using `RuntimeGenesisConfig` instead of directly constructing json for the preset genesis patch
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from runtimes.