Problem
In setting up a new test cluster, @joeaba encountered a capitalization overflow while taking an accounts hash prior to creating a snapshot. The full backtrace / some relevant discussion here on Discord:
...
thread="solAccountsLo10" one=1i message="panicked at accounts-db/src/accounts_hash.rs:849:26:
summing capitalization cannot overflow" location="accounts-db/src/accounts_hash.rs:849:26" version="1.18.2 (src:13656e30; feat:3352961542, client:SolanaLabs)"
The cluster was created with the following solana-genesis
command:
solana-genesis --hashes-per-tick 12500 --bootstrap-validator genesis-keypair.json vote-pubkey.json stake-pubkey.json \
--ledger /home/sol/ledger --cluster-type development --faucet-pubkey faucet.json --faucet-lamports 100000000000000000 \
--bootstrap-validator-lamports 10000000000000 --bootstrap-validator-stake-lamports 250000000000000 \
--primordial-accounts-file accounts.json
The creation of the above genesis file yielded the following capitalization:
Capitalization: 600000000 SOL in 604 accounts
However, here are the lamports that we explicitly added:
18 446 744 073 709 551 615 <== U64::MAX
100 000 000 000 000 000 <== --faucet-lamports
250 000 000 000 000 <== --bootstrap-validator-stake-lamports
10 000 000 000 000 <== --bootstrap-validator-lamports
Note that 500M of the 600M in capitalization stems from an item discovered in solana-labs#35266. Technically, we don't need/want those accounts in a development cluster, but even with 600M capitalization, I don't think we should have overflowed.
Proposed Solution
Recreate the problem and figure out why the overflow happened. Technically, minting too many lamports could cause an overflow. But, even with those extra accounts that got us up to 600M capitalization, we should still have been operating at about 1/30th of u64::MAX:
600M SOL * (1 billion lamports / 1 SOL) / u64::MAX ~= 1 / 30
Joe mentioned that the bootstrap validator was reproducing this. Steps to reproduce:
- Use
solana-keygen
to create a bunch of keys to pass to above solana-genesis
invocation
- Create the following
accounts.json
file and pass it via --primordial-accounts-file
:
{
"tvc2ZFRKhdDgGg3aDsmKGtUWoQdn8gDnhH48KmpEdoj": {
"balance": 25000000000000000,
"owner": "11111111111111111111111111111111",
"data": "",
"executable": false
},
"tvc3NTVL8WdHuUcXEoNcCheQh8S52wjcFEcCXN6AdBj": {
"balance": 25000000000000000,
"owner": "11111111111111111111111111111111",
"data": "",
"executable": false
},
"tvc4VQteaHCQA8hPEyZjfz31JNPR8ny8baNyQYSkt4C": {
"balance": 25000000000000000,
"owner": "11111111111111111111111111111111",
"data": "",
"executable": false
}
}
- This issue showed on v1.18.2, but if trying to reproduce with master, back out the commit that was introduced with solana-labs#35266
- Here were the bootstrap validator args; nothing special here but pasted for completeness:
Starting validator with: ArgsOs {
inner: [
"solana-validator",
"--dynamic-port-range",
"8000-8020",
"--gossip-port",
"8001",
"--identity",
"/home/sol/identity/genesis-keypair.json",
"--ledger",
"/home/sol/ledger",
"--limit-ledger-size",
"--log",
"/home/sol/logs/solana-validator.log",
"--rpc-port",
"8899",
"--expected-genesis-hash",
"BEQ6Kc7BLfSUTh9nfMBW7fn5L8jfA22nCXuBna4NY3ue",
"--wal-recovery-mode",
"skip_any_corrupted_record",
"--full-rpc-api",
"--no-wait-for-vote-to-start-leader",
"--enable-rpc-transaction-history",
"--snapshot-interval-slots",
"5000",
"--vote-account",
"/home/sol/identity/vote-pubkey.json",
"--rpc-faucet-address",
"127.0.0.1:9900",
"--trusted-validator",
"tvc2ZFRKhdDgGg3aDsmKGtUWoQdn8gDnhH48KmpEdoj",
"--halt-on-trusted-validators-accounts-hash-mismatch",
"--no-untrusted-rpc",
"--expected-shred-version",
"3541",
"--entrypoint",
"104.154.224.20:8001",
"--no-genesis-fetch",
"--no-snapshot-fetch",
"--accounts",
"/mnt/accounts",
"--enable-cpi-and-log-storage",
],
}