scroll-tech / go-ethereum Goto Github PK

This project forked from ethereum/go-ethereum

Scroll's fork of the official Go implementation of the Ethereum protocol

License: GNU Lesser General Public License v3.0

Go 91.02% Ruby 0.01% Shell 0.10% C 4.35% Makefile 0.06% JavaScript 2.85% Assembly 0.61% NSIS 0.15% HTML 0.07% Python 0.04% Dockerfile 0.01% Solidity 0.10% M4 0.17% Sage 0.20% Java 0.20% Rust 0.06%

go-ethereum's People

Contributors

Stargazers

Watchers

Forkers

icemelon jules noel2004 maogongyin qvd06 kadrkn brianluis123 pashapitlord x-razor polat1181 thamdestech satman81 thegaram robsberrry gitgoldenhammer eaddaa kriptodahscgo mukeshmeravi unknownsolloet wandaanggara dashapavlyk borisnalxanov tamaratarasevych0 dimsdsdvsd vasiook tutkakt vasiliksa ezotmaxi supono0 subarno01 galaxynode dewa1990 hss1995 katakkuri88 neiky07 degenbrouillard kaantaskin61 cybermuslimindonesia apolinaria19520 voleur27 niaangelina husoam agfirlana bokir20 lifeisdaijoubu devanlee12 nitrrobot smartbitcoinaw lalatrade main320 finest101994 tecnodes-network unifralabs buildaos devan12 suisei880 denyucil conan95 nodestannft caesarcarol micro-sd dragontryy helenistic endorphinel armyoflove bestxeosx-gm manipulation44 qiaoxrcn make69node mosin112 uzmanmuzaffer jack4666 bunga90 balutakriting rachmawa celengsu danangjuniar above-labs zclim as6549639 prayogo00 oomkomariah578 katsuhiromato caocao9 serlyoi mili137106 tolooe mghre7 aircraftcarier afs00n rakeshsay bradyjoestar seve789 duxin0817060114 mehdisal1985 llamanodes stewars20 yopi93 adev060796 bunnazan

go-ethereum's Issues

TODO: keep track of state_after account_proof

We lose the tracking of state_after account_proof.

Simple way:
...

Complicated way:
...

How to refactor and simplify the codes....

fix CaptureState

Previously we capture some state-info depending on stacks

but this is incorrect because in the "after" state the stacks have already changed.

we should move CaptureStateAfter logic into CaptureState

move CaptureStateAfter logic to CaptureState

improve

go-ethereum/core/vm/logger.go

Lines 187 to 221 in 468db1d

    
           if !l.cfg.DisableStorage && (op == SLOAD || op == SSTORE) { 
        
           	// initialise new changed values storage container for this contract 
        
           	// if not present. 
        
           	if l.storage[contract.Address()] == nil { 
        
           		l.storage[contract.Address()] = make(Storage) 
        
           	} 
        
           	// capture SLOAD opcodes and record the read entry in the local storage 
        
           	if op == SLOAD && stack.len() >= 1 { 
        
           		var ( 
        
           			address = common.Hash(stack.data[stack.len()-1].Bytes32()) 
        
           			value   = l.env.StateDB.GetState(contract.Address(), address) 
        
           		) 
        
           		l.storage[contract.Address()][address] = value 
        
           		storage = l.storage[contract.Address()].Copy() 
        
           		extraData = types.NewExtraData() 
        
           		if err := traceStorageProof(l, scope, extraData); err != nil { 
        
           			log.Warn("Failed to get proof", "contract address", contract.Address().String(), "key", address.String(), "err", err) 
        
           		} 
        
           	} else if op == SSTORE && stack.len() >= 2 { 
        
           		// capture SSTORE opcodes and record the written entry in the local storage. 
        
           		var ( 
        
           			value   = common.Hash(stack.data[stack.len()-2].Bytes32()) 
        
           			address = common.Hash(stack.data[stack.len()-1].Bytes32()) 
        
           		) 
        
           		l.storage[contract.Address()][address] = value 
        
           		storage = l.storage[contract.Address()].Copy() 
        
           		extraData = types.NewExtraData() 
        
           		if err := traceStorageProof(l, scope, extraData); err != nil { 
        
           			log.Warn("Failed to get proof", "contract address", contract.Address().String(), "key", address.String(), "err", err) 
        
           		} 
        
           	} 
        
           }

(maybe can combine with 1?)

record "after_state" for sstore (need to think of a way to keep track of address & value, because they were in stack but now stack change)

USDC test suite could not parse event

yarn test gives a lots of output of:

Events emitted during test:
---------------------------

Warning: Could not decode event!
...
---------------------------

create trace from block + storage proof statelessly

currently we generate block trace in l2geth. But the trace is very heavy due to structLog per step.

while In theory, if we have Block + Storage Witness, we can re-execute the block statelessly. So prover server can re-construct the full trace locally without network overhead.

So we plan to try to implement this feature. it gives us more freedom in future architecture design.

The first step:

(1) write a command line tool, the input is a trace json https://github.com/scroll-tech/integration-test/#a-complete-guide-on-how-to-generate-trace . The tool traverse the storage trace in this json, and insert the trie node into trie database (backed by a local in-memory db). And then re-execute the txs in the block (of course the per step StructLog will be be used here), we can get a new trace json. Finally we assert the new trace json is identical to the old one.

Use this branch #101. The branch can use two types of storage backend: zktrie and old mpt. Finally we plan to use the zktrie backend. For this issue, both zktrie and old mpt is ok. But i am afraid whether it works for old mpt since the mpt is very complext and so the "storage trace" may not contain enough witness sibling nodes.
I suggest use zktrie to implement this feature, whenever storage trace misses some needed sibling nodes, we treat it as a bug that should be fixed.

add recent change-logs (by 2022-0624)

Implement roller communication logic

Diff with the current geth codebase's vm.interpreter.Run function

Hi, just found that our EVM's interpreter's Run function is a bit different than the current codebase of Geth. In order to improve performance, they removed the code in this place that https://github.com/scroll-tech/go-ethereum/blob/zkrollup/core/vm/interpreter.go#L183-L188. (Current Geth: https://github.com/ethereum/go-ethereum/blob/master/core/vm/interpreter.go#L181-L185)

Since this part is the core function of EVM, maybe we should align with them in the next release?

FYI:

A new tag about l2geth

Merge zkrollup branch into master and give a tag.

Redundant adding accounts with their balance when creating the Genesis Block

System information

Geth version: geth version
OS & Version: OSX
Commit hash : zkrollup-smt

Expected behaviour

The geth adds two predefiend accounts and their balance from in the genesis.json.

Actual behaviour

After call adding two predefiend accounts and their balance from in the genesis.json, geth also calls "Writing default main-net genesis block" and creates extra 8893 accounts.

!!! These accounts can get additional Tokens on our layer.

Steps to reproduce the behaviour

Here is the genesis.json I used for testing (from new_trie:debug/huai).

{
    "config": {
      "chainId": 53077,
      "homesteadBlock": 0,
      "eip150Block": 0,
      "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
      "eip155Block": 0,
      "eip158Block": 0,
      "byzantiumBlock": 0,
      "constantinopleBlock": 0,
      "petersburgBlock": 0,
      "istanbulBlock": 0
    },
    "nonce": "0x0",
    "timestamp": "0x61bc34a0",
    "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000221cb8Cd97223442754493A31184CF81F418E7280000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "gasLimit": "0x47b760",
    "difficulty": "0x1",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "coinbase": "0x0000000000000000000000000000000000000000",
    "alloc": {
      "221cb8Cd97223442754493A31184CF81F418E728": {
        "balance": "0x56BC75E2D63100000"
      },
      "6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf": {
        "balance": "0x0"
      }
    },
    "number": "0x0",
    "gasUsed": "0x0",
    "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "baseFeePerGas": null
  }

Here are some logs I print for debug.

INFO [03-05|15:53:18.464] Maximum peer count                       ETH=50 LES=0 total=50
INFO [03-05|15:53:18.465] Set global gas cap                       cap=50,000,000
INFO [03-05|15:53:18.465] Allocated cache and file handles         database=/Users/siyuan/workspace/go/src/github.com/scroll-tech/go-ethereum/testchain/geth/chaindata cache=16.00MiB handles=16
INFO [03-05|15:53:18.509] Writing custom genesis block
INFO [03-05|15:53:18.566] Genesis Block Alloc Account number:      LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
Genesis Block Alloc Account number 2
INFO [03-05|15:53:18.566] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=0
INFO [03-05|15:53:18.566] Call AddBalance # Target Address:        0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=nil LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.566] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.566] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.566] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.566] Call createObject
ERROR[03-05|15:53:18.566] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.566] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.566] The new Object Address/Balance           0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=0
INFO [03-05|15:53:18.566] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0x221cb8Cd97223442754493A31184CF81F418E728=100,000,000,000,000,000,000
INFO [03-05|15:53:18.566] Call AddBalance # Target Address:        0x221cb8Cd97223442754493A31184CF81F418E728=nil                         LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.566] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.566] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.566] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.566] Call createObject
ERROR[03-05|15:53:18.566] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.566] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.566] The new Object Address/Balance           0x221cb8Cd97223442754493A31184CF81F418E728=0
INFO [03-05|15:53:18.568] Successfully wrote genesis state         database=chaindata hash=be088c..4ae275
INFO [03-05|15:53:18.568] Allocated cache and file handles         database=/Users/siyuan/workspace/go/src/github.com/scroll-tech/go-ethereum/testchain/geth/lightchaindata cache=16.00MiB handles=16
INFO [03-05|15:53:18.597] Writing custom genesis block
INFO [03-05|15:53:18.597] Genesis Block Alloc Account number:      LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
**Genesis Block Alloc Account number 2**
INFO [03-05|15:53:18.597] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0x221cb8Cd97223442754493A31184CF81F418E728=100,000,000,000,000,000,000
INFO [03-05|15:53:18.597] Call AddBalance # Target Address:        0x221cb8Cd97223442754493A31184CF81F418E728=nil                         LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.597] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.597] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.597] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.597] Call createObject
ERROR[03-05|15:53:18.597] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.597] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.597] The new Object Address/Balance           0x221cb8Cd97223442754493A31184CF81F418E728=0
INFO [03-05|15:53:18.597] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=0
INFO [03-05|15:53:18.597] Call AddBalance # Target Address:        0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=nil LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.597] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.597] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.597] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.597] Call createObject
ERROR[03-05|15:53:18.597] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.598] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.598] The new Object Address/Balance           0x6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf=0
INFO [03-05|15:53:18.599] Successfully wrote genesis state         database=lightchaindata hash=be088c..4ae275
INFO [03-05|15:53:18.669] Starting Geth on Ethereum mainnet...
INFO [03-05|15:53:18.669] Bumping default cache on mainnet         provided=1024 updated=4096
INFO [03-05|15:53:18.671] Maximum peer count                       ETH=0 LES=0 total=0
INFO [03-05|15:53:18.672] Set global gas cap                       cap=50,000,000
INFO [03-05|15:53:18.672] Allocated trie memory caches             clean=614.00MiB dirty=1024.00MiB
INFO [03-05|15:53:18.672] Allocated cache and file handles         database=/Users/siyuan/workspace/go/src/github.com/scroll-tech/go-ethereum/testchain/geth/chaindata cache=2.00GiB handles=5120
INFO [03-05|15:53:18.729] Opened ancient database                  database=/Users/siyuan/workspace/go/src/github.com/scroll-tech/go-ethereum/testchain/geth/chaindata/ancient readonly=false
INFO [03-05|15:53:18.729] Writing default main-net genesis block
INFO [03-05|15:53:18.773] Genesis Block Alloc Account number:      LOG15_ERROR= LOG15_ERROR="Normalized odd number of arguments by adding nil"
**Genesis Block Alloc Account number 8893**
INFO [03-05|15:53:18.773] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0x2F13657526B177CaD547C3908c840Eff647b45D9=1,170,685,000,000,000,000,000
INFO [03-05|15:53:18.773] Call AddBalance # Target Address:        0x2F13657526B177CaD547C3908c840Eff647b45D9=nil                           LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.773] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.773] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.773] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.773] Call createObject
ERROR[03-05|15:53:18.773] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.773] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.773] The new Object Address/Balance           0x2F13657526B177CaD547C3908c840Eff647b45D9=0
INFO [03-05|15:53:18.773] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0xB2BFAa58B5196c5CB7F89de15f479D1838de713D=21,000,000,000,000,000,000
INFO [03-05|15:53:18.773] Call AddBalance # Target Address:        0xB2BFAa58B5196c5CB7F89de15f479D1838de713D=nil                        LOG15_ERROR="Normalized odd number of arguments by adding nil"
ERROR[03-05|15:53:18.773] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.773] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.773] GetOrNewStateObject function call createObject because of stateObject == nil
INFO [03-05|15:53:18.773] Call createObject
ERROR[03-05|15:53:18.773] [Error-Log] getDeletedStateObject function: <nil>
ERROR[03-05|15:53:18.774] [Error-Log] TryGet function: Key not found in the MerkleTree
INFO [03-05|15:53:18.774] The new Object Address/Balance           0xB2BFAa58B5196c5CB7F89de15f479D1838de713D=0
INFO [03-05|15:53:18.774] Genesis Block ToBlock function Call AddBalance for Address/Balance (in Alloc): 0xeC2CB8B9378dff31aeC3c22e0e6DAdFf314AB5Dd=2,000,000,000,000,000,000,000
INFO [03-05|15:53:18.774] Call AddBalance # Target Address:        0xeC2CB8B9378dff31aeC3c22e0e6DAdFf314AB5Dd=nil                           LOG15_ERROR="Normalized odd number of arguments by adding nil"

For example: Account "0xbEA0AfC93aae2108a3FAc059623BF86FA582a75e" is not defined in the genesis file but still has 1700 eth(So many).

// Not in our genesis.json
> eth.getBalance("0xbEA0AfC93aae2108a3FAc059623BF86FA582a75e")
ERROR[03-05|16:17:25.845] [Error-Log] getDeletedStateObject function: not covered yet
WARN [03-05|16:17:25.845] 1700000000000000000000
1.7e+21

When submitting logs: please submit them as text and not screenshots.

Update go.mod and imports so that we can build remotely

TODO: MT Read/Write Optimaztion

Goals:

Lowering the cost of MT writing.
Reducing the cost of MT reads.
Evaluating the impact when L2geth is subjected to high-volume read operations

TestGenerateBlockAndImportEthash should pass

go-ethereum/miner/worker_test.go

Lines 207 to 210 in 97faa8c

    
           func TestGenerateBlockAndImportEthash(t *testing.T) { 
        
           	t.Skip("FIXME later. it should pass") 
        
           	testGenerateBlockAndImport(t, false) 
        
           }

currently failed with log:

=== RUN   TestGenerateBlockAndImportEthash

    worker_test.go:265: failed to insert new mined block 1: invalid receipt root hash (remote: 46be8924226c54f771d30283b863add0b365f39298253ce44601c035db6323ee local: db85ddb0e660e1c591341cb76012b4ae5a384b62322e196e387bbf3ba8dcab59)
--- FAIL: TestGenerateBlockAndImportEthash (0.01s)

FAIL

Priority

P0 TestGenerateBlockAndImportEthash
P0 TestTransactionIndices

add code_hash to account_proof

Unable to get the value from the contract Storage

System information

Geth version: geth version
OS & Version: OSX
Commit hash : zkrollup-smt: ff17af3

Expected behaviour

Get the value of the corresponding value from the Storage layer of the contract.

Actual behaviour

Unable to get the value from the corresponding Storage Slot. The result oh that thing returns a zero value.

> eth.getStorageAt("0x4b5d1fB97BC071aa419aA4CF811c84C1d9DF85B0",0)
"0x0000000000000000000000000000000000000000000000000000000000000000"

Steps to reproduce the behaviour

Here is the solidity code used in this reporting issue.

pragma solidity >=0.7.0 <0.9.0;

/**
 * @title Storage
 * @dev Store & retrieve value in a variable
 */
contract Storage {
    
    uint256 number;
    
    function set_balance(uint256 num) public {
        number = num;
    }
    
    function get_number() public view returns (uint256){
        return number;
    }
  
}

This contract declares one uint256 variable and two functions: the set_balance function used for storing the data and the get_number function used for retrieving the data.

We use Remix to connect to the local geth.

We first construct the transaction by calling the set_balance function with 100 as the input.

Then, we call the get_number function to check the value of the number. We can observe that the get_number function returns 0 rather than the correct result, 100.

If we call eth.getStorageAt("0x4b5d1fB97BC071aa419aA4CF811c84C1d9DF85B0",0) in Geth console, it also retruns 0.

Backtrace

I think the cause of this problem may be in the GetCommittedState function in the state_object.go file. 1. https://github.com/scroll-tech/go-ethereum/blob/zkrollup-smt/core/state/state_object.go#L249

_if len(enc) > 0 {
		_, content, _, err := rlp.Split(enc)
		if err != nil {
			s.setError(err)
		}
		value.SetBytes(content)
	}

Since we no longer need the rlp encoding anymore, I tried replacing the contents of if with value.SetBytes(enc). According to the figure below, this looks valid. But I don't know if this modification will have other effects.

Transaction not found

System information

Geth version: geth version
OS & Version: Windows/Linux/OSX
Commit hash : (if develop)

Expected behaviour

Actual behaviour

ERROR[05-18|10:24:05.075] Transaction not found number=170 hash=04ffca..2f0a50 txhash=83ae4c..985800

Steps to reproduce the behaviour

Backtrace

This happens occasionally when debugging with Goland

[backtrace]

When submitting logs: please submit them as text and not screenshots.

Disable the consensus module in the miner

When call GetBlockResultByHash api.

json: cannot unmarshal non-string into Go struct field BlockTrace.blockTrace.coinbase of type common.Address

add storage value in storage_proof

Meta issue on trace

executionResults.Sender.Nonces in are not correct? They are fetched after the whole block execution

go-ethereum/core/blockchain.go

Lines 1376 to 1381 in 40e8f08

    
           evmTrace.Sender = &types.AccountProofWrapper{ 
        
           	Address:  from, 
        
           	Nonce:    state.GetNonce(from), 
        
           	Balance:  state.GetBalance(from), 
        
           	CodeHash: state.GetCodeHash(from), 
        
           }

traces cost too much memory

During the erc20 stress test, memory usage kept increasing and did not decrease after new block was produced.

suggest:
performe the stress test on a high mem cap machine.
Investigate this issue

disable some unavailable opcodes

like some precompile

mysterious errors occurs in USDC test suite

keystore: testchain.zip
genesis.json:

{
  "config": {
    "chainId": 53077,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "clique": {
      "period": 15,
      "epoch": 30000
    }
  },
  "nonce": "0x0",
  "timestamp": "0x61bc34a0",
  "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000221cb8Cd97223442754493A31184CF81F418E7280000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  "gasLimit": "0x6691b7",
  "difficulty": "0x1",
  "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "coinbase": "0x0000000000000000000000000000000000000000",
  "alloc": {
    "221cb8Cd97223442754493A31184CF81F418E728": {
      "balance": "0x56BC75E2D63100000"
    },
    "6CCf3e5FEbdEe9a3B263Eb3eAd5A19fa95Dc8ADf": {
      "balance": "0x56BC75E2D63100000"
    },
    "90F8bf6A479f320ead074411a4B0e7944Ea8c9C1": {
      "balance": "0x56BC75E2D63100000"
    },
    "FFcf8FDEE72ac11b5c542428B35EEF5769C409f0": {
      "balance": "0x56BC75E2D63100000"
    },
    "E11BA2b4D45Eaed5996Cd0823791E0C93114882d": {
      "balance": "0x56BC75E2D63100000"
    },
    "d03ea8624C8C5987235048901fB614fDcA89b117": {
      "balance": "0x56BC75E2D63100000"
    },
    "95cED938F7991cd0dFcb48F0a06a40FA1aF46EBC": {
      "balance": "0x56BC75E2D63100000"
    },
    "3E5e9111Ae8eB78Fe1CC3bb8915d5D461F3Ef9A9": {
      "balance": "0x56BC75E2D63100000"
    },
    "28a8746e75304c0780E011BEd21C72cD78cd535E": {
      "balance": "0x56BC75E2D63100000"
    },
    "ACa94ef8bD5ffEE41947b4585a84BdA5a3d3DA6E": {
      "balance": "0x56BC75E2D63100000"
    },
    "1dF62f291b2E969fB0849d99D9Ce41e2F137006e": {
      "balance": "0x56BC75E2D63100000"
    },
    "610Bb1573d1046FCb8A70Bbbd395754cD57C2b60": {
      "balance": "0x56BC75E2D63100000"
    },
    "2F560290FEF1B3Ada194b6aA9c40aa71f8e95598": {
      "balance": "0x56BC75E2D63100000"
    }
  },
  "number": "0x0",
  "gasUsed": "0x0",
  "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
  "baseFeePerGas": null
}

geth cmdline: ./build/bin/geth --datadir testchain --mine --unlock 0,1,2,3,4,5,6,7,8,9,10,11,12 --allow-insecure-unlock --miner.etherbase 0x221cb8Cd97223442754493A31184CF81F418E728 --maxpeers 0 --http --http.corsdomain https://remix.ethereum.org --http.api personal,eth,net,web3,debug

then run yarn test from centre-tokens

Expected ERROR: Failed to trace data

add balance in account_proof

architecture: 1 write node, multiple read nodes

need to tweak consensus

trace: add coinbase address's account proof

Deducing the I/O cost by increasing the capacity of the underlying database's caches

Currently, Geth's cache setting for the underlying LevelDB is based on the Default parameter. We can see here that it is about 256MB. Considering the original Geth will be deployed on devices with average performance, this default setting is relatively low.

Since the performance of our Sequencer is much higher, increasing the size of these caches will significantly reduce the underlying I/O cost.

Related codes:

backlog: remove mem trace

after scroll-tech/scroll-prover#32 is used,

we can set vm.LogConfig{EnableMemory: true} to false. i guess doing this can reduce blockResult size 5-10times.

(I remember maybe we have to change 2 place? One is for the realtime websocket , the other is getBlockResultByHash fetching/regerating for old block trace? )

clear out any leftover accessList from previous executions

from the discussion in privacy-scaling-explorations/zkevm-chain#5

I find that we should follow https://github.com/ethereum/go-ethereum/blob/f49e29833084d65cae0d2eba71e5788e6d9996c0/core/state/statedb.go#L1002 (currently we don;t have this clearing)

We should also keep track of

bug fixes on https://github.com/ethereum/go-ethereum
PSE's changes to their L2geth: https://github.com/privacy-scaling-explorations/go-ethereum

fix `code_hash` json annotation.

need to fix
rename code_hash annotation to codeHash in https://github.com/scroll-tech/go-ethereum/blob/zkrollup/core/types/l2trace.go#L62

also need to update:

Redundant/Useless TrieDB instances

Since no functions in trie\database.go are used as the current middle layer for MT to submit data to Disk, all instances of TrieDB are theoretically useless.
Also, since the structure of MT nodes is different from that of the original MPT, the functions of reference counting in trie\database.go are invalid.

But there are still a lot of functions calls to trie\database.go in the codebase.

For example, in the core/blockchain.go , NewBlockChain function creates an instance of trie\database.go\Database. And this instance is called multiple times in the core/blockchain.go , such as:

triedb := bc.stateCache.TrieDB()

Even some function about reference counting of nodes is called, we can find it in this line: https://github.com/scroll-tech/go-ethereum/blob/zkrollup-smt/core/blockchain.go#L1243.

Fortunately, since MT has not submitted any data to this instance, its dirties array is always empty. Therefore, the relevant call is always a direct return, which is why the current code works without problems.

If we want to support both MPT and MT, these areas may need some attention.

Determine which tests to run and set up CI

add coinbase address's account_proof in trace for each block

Remove secp256k1 package

This package is causing linker errors in the sequencer repo, due to duplicate C symbols for the crypto functions.

Uniswap v3 test partial failed on zkrollup branch

Reproduce

testing on commit f853248
with https://github.com/Uniswap/v3-core

v3-core modificiation

+++ b/hardhat.config.ts
@@ -4,7 +4,13 @@ import '@nomiclabs/hardhat-waffle'
 import '@nomiclabs/hardhat-etherscan'

 export default {
+  defaultNetwork: "localhost",
   networks: {
+    localhost: {
+      url: "http://127.0.0.1:8545",
+      gas: 0x6691b7,
+      gasPrice: 1e9,
+    },
     hardhat: {
       allowUnlimitedContractSize: false,
     },

after start geth node, run npx hardhat test --network localhost

Part failed test

  10) Tick
       #getFeeGrowthInside
         subtracts upper tick if below:
     AssertionError: Expected "15" to be equal 13
      at Context.<anonymous> (test\Tick.spec.ts:70:54)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  11) Tick
       #getFeeGrowthInside
         subtracts lower tick if above:
     AssertionError: Expected "15" to be equal 13
      at Context.<anonymous> (test\Tick.spec.ts:85:54)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  12) Tick
       #getFeeGrowthInside
         subtracts upper and lower tick if inside:
     AssertionError: Expected "15" to be equal 9
      at Context.<anonymous> (test\Tick.spec.ts:110:54)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  13) Tick
       #getFeeGrowthInside
         works correctly with overflow on inside tick:
     AssertionError: Expected "15" to be equal 16
      at Context.<anonymous> (test\Tick.spec.ts:135:54)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  14) Tick
       #update
         does not flip from nonzero to greater nonzero:

      AssertionError: expected true to equal false
      + expected - actual

      -true
      +false

      at Context.<anonymous> (test\Tick.spec.ts:145:100)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  18) Tick
       #update
         reverts if total liquidity gross is greater than max:

      AssertionError: Expected transaction to be reverted
      + expected - actual

      -Transaction NOT reverted.
      +Transaction reverted.



  19) Tick
       #update
         nets the liquidity based on upper flag:
     AssertionError: Expected "0" to be equal 7
      at Context.<anonymous> (test\Tick.spec.ts:170:48)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  20) Tick
       #update
         reverts on overflow liquidity gross:

      AssertionError: Expected transaction to be reverted
      + expected - actual

      -Transaction NOT reverted.
      +Transaction reverted.



  21) Tick
       #update
         assumes all growth happens below ticks lte current tick:
     AssertionError: Expected "0" to be equal 1
      at Context.<anonymous> (test\Tick.spec.ts:180:55)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  22) Tick
       #update
         does not set any growth fields if tick is already initialized:
     AssertionError: Expected "0" to be equal 1
      at Context.<anonymous> (test\Tick.spec.ts:191:55)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  23) Tick
       #update
         does not set any growth fields for ticks gt current tick:

      AssertionError: expected false to equal true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test\Tick.spec.ts:206:45)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  24) Tick
       #cross
         flips the growth variables:
     AssertionError: Expected "0" to be equal 6
      at Context.<anonymous> (test\Tick.spec.ts:248:55)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  25) Tick
       #cross
         two flips are no op:
     AssertionError: Expected "0" to be equal 1
      at Context.<anonymous> (test\Tick.spec.ts:268:55)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  26) TickBitmap
       #isInitialized
         is flipped by #flipTick:

      AssertionError: expected false to equal true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test\TickBitmap.spec.ts:26:67)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  27) TickBitmap
       #isInitialized
         is not changed by another flip to a different tick on another word:

      AssertionError: expected false to equal true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test\TickBitmap.spec.ts:39:69)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  28) TickBitmap
       #flipTick
         flips only the specified tick:

      AssertionError: expected false to equal true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test\TickBitmap.spec.ts:46:70)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  29) TickBitmap
       #flipTick
         reverts only itself:

      AssertionError: expected false to equal true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test\TickBitmap.spec.ts:66:70)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

Get inconsistent Contract storage data caused by stale/incorrect Snapshot

System information

Geth version: geth version
OS & Version: OSX
Commit hash : zkrollup-smt: ff17af3

Expected behaviour

Get the correct value of the persistent data in the contract.

Actual behaviour

The query result is not fully correct. Please see the description below for the details.

Steps to reproduce the behaviour

Here is the solidity code used in this experimental study.

// SPDX-License-Identifier: GPL-3.0

pragma solidity >=0.7.0 <0.9.0;

/**
 * @title Storage
 * @dev Store & retrieve value in a variable
 */
contract Storage {
    
    uint256 number;
    
    mapping(string => uint256) balances;

    function set_balance(uint256 num) public {
        number = num;
        balances["hsy"] = num;
        balances["ysh"] = num + 1;
    }
    
    function get_number() public view returns (uint256){
        return number;
    }

    function get_hsy() public view returns (uint256){
        return balances["hsy"];
    }

    function get_ysh() public view returns (uint256){
        return balances["ysh"];
    }
    
}

This contract has two variables, an uint256 variable named number, and a mapping variable named balances. The set_balance function will set the input value to number and balances["hsy"], and set the input value+1 to balances["ysh"]. In addition, we have three get functions to query their values, respectively.

When we call the set_balance function with 100 as input, we expect the values of these variables to be: number == 100, balances["hsy"] == 100, balances["ysh"] == 101.

However, we can observe from the following figures that the query results are incorrect. Either all results are 100 or all results are 101.

Backtrace

TL;DR: This problem may be caused by the inconsistency between the Snapshot and the actual value.

We call the set_balance function with 100 as an input. To locate the problem, I printed the relevant logs as follows. Each of the three variables we mentioned occupies a separate slot, so we can see the assignment of the three slots. And according to the following logs, it seems that the SetState function runs well and assgins the correct value to the corresponding slot.

INFO [03-12|21:52:49.772] Set State:                               0x0000000000000000000000000000000000000000000000000000000000000000=0x0000000000000000000000000000000000000000000000000000000000000064
INFO [03-12|21:52:49.774] Set State:                               0xbaded3bf529b04b554de2e4ee0f5702613335896b4041c50a5555b2d5e279f91=0x0000000000000000000000000000000000000000000000000000000000000064
INFO [03-12|21:52:49.777] Set State:                               0x37910dd9fc389b0cf2923975f06457bb9f45771422beae4687205a0988af00de=0x0000000000000000000000000000000000000000000000000000000000000065

But when the query functions are called we get a different value. The value of 0x37910dd9fc389b0cf2923975f06457bb9f45771422beae4687205a0988af00de should be 0x65 but gets 0x64 instead.

INFO [03-12|22:01:58.959] Get State:                               0x0000000000000000000000000000000000000000000000000000000000000000=0x0000000000000000000000000000000000000000000000000000000000000064
INFO [03-12|22:01:59.675] Get State:                               0xbaded3bf529b04b554de2e4ee0f5702613335896b4041c50a5555b2d5e279f91=0x0000000000000000000000000000000000000000000000000000000000000064
INFO [03-12|22:02:00.457] Get State:                               0x37910dd9fc389b0cf2923975f06457bb9f45771422beae4687205a0988af00de=0x0000000000000000000000000000000000000000000000000000000000000064

This means that the problem is probably still caused by the getState function.

We find that in the GetCommittedState function, the final value is actually obtained from the if segment snap!=nil.

https://github.com/scroll-tech/go-ethereum/blob/zkrollup-smt/core/state/state_object.go#L229

	if s.db.snap != nil {
		if metrics.EnabledExpensive {
		if _, destructed := s.db.snapDestructs[s.addrHash]; destructed {
			return common.Hash{}
		}
		enc, err = s.db.snap.Storage(s.addrHash, crypto.Keccak256Hash(key.Bytes()))
	}

We can find in the stack at the breakpoint here that the values in snapshot are all 100 (0x64).

I made an additional attempt to remove this snap!=nil snippet and try to get the data from the trie each time. Then, we can get the correct data as the following firgure shows.

// Incorrect logic here. Only used for testing.
if s.db.snap != nil || s.db.snap == nil || err != nil {
if meter != nil {
	// If we already spent time checking the snapshot, account for it
	// and reset the readStart
	*meter += time.Since(readStart)
	readStart = time.Now()
}
if metrics.EnabledExpensive {
	meter = &s.db.StorageReads
}
if enc, err = s.getTrie(db).TryGet(key.Bytes()); err != nil {
	s.setError(err)
	return common.Hash{}
}
}

So the reason of this problem shall be the incorrect snapshot data. This problem will disappear after rebooting Geth(No Snapshot). But for a continuously running geth node, it is a problem that needs to be solved.

I found that this problem can be caused by the error in the s.snapStorage value at the time of calling stateDB commit function. https://github.com/scroll-tech/go-ethereum/blob/zkrollup-smt/core/state/statedb.go#L972

We can observe from the following logs that the sanpshot of all the Slots is updated with the same value (The logs below shows the hash of the Slot position). This is incorrect.

INFO [03-13|11:48:48.553] Update Snapshot                          0x290decd9548b62a8d60345a988386fc84ba6bc95484008f6362f93160ef3e563=0000000000000000000000000000000000000000000000000000000000000065
INFO [03-13|11:48:59.593] Update Snapshot                          0xa601d8e9cd2719ca27765dc16042655548d1ac3600a53ffc06b4a06a12b7c65c=0000000000000000000000000000000000000000000000000000000000000065
INFO [03-13|11:49:00.301] Update Snapshot                          0x43e5a6d45b6b646cf39869ca05026a8e8998e1cf7e9606beb8ca83e16c4a4e49=0000000000000000000000000000000000000000000000000000000000000065

The process of debugging in the middle is too long, so I'll omit it here and go straight to the conclusion:

I found this issue could be related to the updateTrie function in state_object.go.. https://github.com/scroll-tech/go-ethereum/blob/zkrollup-smt/core/state/state_object.go#L322

The following is the function call stack when this problem is encountered.

     worker.go/commitTransactions->commitTransaction->commitTransaction
-> stete_processor.go/ApplyTransaction -> applyTransaction
-> stete_transasition.go/ApplyMessage -> TransitionDb
-> vm/evm.go/Call -> vm/interpreter.go/run -> vm/logger.go/CaptureStateAfter
-> statedb.go/GetStorageProof ->  StorageTrie
-> state_object.go/updateTrie

Modify MPT to output the trace

Need to follow the MPT circuit design at https://hackmd.io/@3otaqoTqRnyDPRPeE85dqw/SJ-JKxaGF

Define protocol spec for sequencer/roller communication

Define tracking struct and api about evm.

The "real" coinbase address do not being included in block for clique (PoA) mining

It is noticed that when geth is mining a block under clique mode, it has applied a coinbase address (i.e. one of the authorized address) so the gas bouns for miner would be added into this address. However, this address would not being put in the coinbase field of a block (it is always zero).

Lacking of the information of this address, it means the block and trace we have obtained is in fact not complete. For example, the world state could not be correctly rebuilt for we have just missed the balance change of the coinbase address.

PR #126 would fix part of this issue (among the storage traces) but it may not enough since we have not considered the gas bonus in evm/state circuit with the unspecified coinbase address. Maybe we should:

Change the coinbase field in block to the "real" coinbase used in clique mode, OR
Add another field to specify the address

SMT: Ideds about SMT Implementation Optimization

Hi friends, I have a small question about the SMT codebase.

Since the logic of mt.updateWord and mt.updateVarWord functions are basically the same, I will take mt.updateWord as an example.

To my understanding, in the tree update process, the function call chains when facing a new node are as follows:

First, we call mt.updateWord and then call mt.Update.
proof, err := mt.Update(k, v, kPreimage, vPreimage[:])
In the mt.Update function, we first go through the MT by the mt.GetNode function.
n, err := mt.GetNode(nextKey)
If the key is not found, we return to the mt.updateWord function, then call MT. Add function to add a new node to the MT.
In the mt.Add function, we call the mt.addLeaf function to add the lead node.
newNodeLeaf := NewNodeLeaf(kHash, vHash, kPreimage, vPreimage)

However, we can see that there are two parts that should be duplicated in the current codebase.

There are three hash calls may be duplicated.
We can observe that the mt.Update and mt.Add have the same agruments when we call MT.updateWord function.

        // mt.Update and mt.Add have the same agruments.
     	proof, err := mt.Update(k, v, kPreimage, vPreimage[:])
	if err == ErrKeyNotFound {
		err = mt.Add(k, v, kPreimage, vPreimage[:])
		return nil, err
	}

Hence, the values of kHash, vHash, and path should be the same both in Update and Add.

func (mt *MerkleTree) Update(k, v *big.Int, kPreimage *smt.Byte32, vPreimage []byte) (*CircomProcessorProof, error) {
        ...
	kHash := smt.NewHashFromBigInt(k)
	vHash := smt.NewHashFromBigInt(v)
	path := getPath(mt.maxLevels, kHash[:])
        ...
}

func (mt *MerkleTree) Add(k, v *big.Int, kPreimage *smt.Byte32, vPreimage []byte) error {
...
	kHash := smt.NewHashFromBigInt(k)
	vHash := smt.NewHashFromBigInt(v)
	newNodeLeaf := NewNodeLeaf(kHash, vHash, kPreimage, vPreimage)
	path := getPath(mt.maxLevels, kHash[:])
...
}

MT's duplicate visits. We already went through the MT in the mt.Update function, but we go through the MT in the mt.addLeaf again.

         // mt.Update function accesses the MT from the root node.
	nextKey := mt.rootKey
	siblings := []*smt.Hash{}
	for i := 0; i < mt.maxLevels; i++ {
		n, err := mt.GetNode(nextKey)
                 ...
        }

        // mt.addLeaf function also starts from the mt.rootKey
	newRootKey, err := mt.addLeaf(tx, newNodeLeaf, mt.rootKey, 0, path)

I think the mt.GetNodefunction called in both mt.Update and mt.addLeaf is costly, since it will call mt.db.Get (key [:]), which is currently called leveldb every time it is invoked.

I have some simple ideas:

We can reuse the kHash, vHash, and path computed in mt.Update function.
We could establish a memory database for MT, then we could go through the memory instead of the disk.

After finishing all the main workflow functions, we may need to think about how to reduce the potential number of levelDB calls, since the disk I/O is costly.

Please let me know if I have misunderstood anything.

Write a change log to record the changes applied to Geth

We need to create a change log in the repo to record which change we did to Geth repo, better to have one entry corresponding to each commit.

optimize code data in trace

as suggested in #128 (review)

backlog: updates needed for next circuit release

rebase traceing memory size updates from upstream ethereum#24867
disable memory trace (to allow bigger blocks)

Need to do them when integrating with NEXT scroll-dev-MMDD circuit release

add refund_counter to trace

ethereum@b5a129e

Define extractTrace interface in the Geth for Sequencer

This is the interface for Sequencer to get the block trace from Geth.

we must optimize "EnableMemory" in trace

I traced a random recent block (block 14742200, 30M gas).

When EnableMemory: false, debug_traceBlockByNumber finished within 15 seconds, the trace file size is less than 130MiB.

But when EnableMemory: true, debug_traceBlockByNumber never runs successfully even on a 64GiB mem server. The debug_traceBlockByNumber rpc call usually fails(return nothing) after more than 2 mintues, 40-50GiB mem used.

curl -s -H "Content-Type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"debug_traceBlockByNumber", "params": ["0xe0f2b8",{"DisableStack":false,"DisableStorage":false,"EnableMemory":true,"EnableReturnData":true,"Timeout":"10m"}], "id": 99}' 127.0.0.1:8545 > tmp.trace

(I used ethereum/go-ethereum, not scroll-tech/go-ethereum)

Rev: 9b99f2e

	if !l.cfg.DisableStorage && (op == SLOAD \|\| op == SSTORE) {
	// initialise new changed values storage container for this contract
	// if not present.
	if l.storage[contract.Address()] == nil {
	l.storage[contract.Address()] = make(Storage)
	}
	// capture SLOAD opcodes and record the read entry in the local storage
	if op == SLOAD && stack.len() >= 1 {
	var (
	address = common.Hash(stack.data[stack.len()-1].Bytes32())
	value = l.env.StateDB.GetState(contract.Address(), address)
	)
	l.storage[contract.Address()][address] = value
	storage = l.storage[contract.Address()].Copy()

	extraData = types.NewExtraData()
	if err := traceStorageProof(l, scope, extraData); err != nil {
	log.Warn("Failed to get proof", "contract address", contract.Address().String(), "key", address.String(), "err", err)
	}

	} else if op == SSTORE && stack.len() >= 2 {
	// capture SSTORE opcodes and record the written entry in the local storage.
	var (
	value = common.Hash(stack.data[stack.len()-2].Bytes32())
	address = common.Hash(stack.data[stack.len()-1].Bytes32())
	)
	l.storage[contract.Address()][address] = value
	storage = l.storage[contract.Address()].Copy()

	extraData = types.NewExtraData()
	if err := traceStorageProof(l, scope, extraData); err != nil {
	log.Warn("Failed to get proof", "contract address", contract.Address().String(), "key", address.String(), "err", err)
	}
	}
	}

	func TestGenerateBlockAndImportEthash(t *testing.T) {
	t.Skip("FIXME later. it should pass")
	testGenerateBlockAndImport(t, false)
	}

	evmTrace.Sender = &types.AccountProofWrapper{
	Address: from,
	Nonce: state.GetNonce(from),
	Balance: state.GetBalance(from),
	CodeHash: state.GetCodeHash(from),
	}

scroll-tech / go-ethereum Goto Github PK

go-ethereum's People

Contributors

Stargazers

Watchers

Forkers

go-ethereum's Issues

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

Backtrace

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

Backtrace

Reproduce

v3-core modificiation

Part failed test

System information

Expected behaviour

Actual behaviour

Steps to reproduce the behaviour

Backtrace

Recommend Projects

Recommend Topics

Recommend Org