20211009_workshop_geth_pending_logs

Task

Learn how to start working with Geth

Download Geth
Start Geth node
Deploy a contract
Work with Geth JavaScript console
Subscribe to Geth events

Steps

Get Geth client sources and compile

Geth sources are available at github at https://github.com/ethereum/go-ethereum
```
git clone https://github.com/ethereum/go-ethereum
cd go-ethereum
# make geth
make all
```
After succesful compilation the Geth binaries can be found at build/bin/
```
export PATH=$PATH:"$PWD/build/bin"
```
Note
the recommended steps to build on various OS are in the documentation at https://geth.ethereum.org/docs/install-and-build/installing-geth#build-go-ethereum-from-source-code

Start Geth client on localhost

To start the Geth and then to "play" with the private network we need

To create an account(s) that can be pre-funded and used for transfers and contract calls (see https://geth.ethereum.org/docs/interface/managing-your-accounts)

mkdir data
DATADIR="$PWD/data"
geth account new --datadir "$DATADIR" --password <(echo 'P422wORd')
# note down the public address of the created key

# each account created stores a key in keystore under $DATADIR
tree "$DATADIR"/keystore

Note	Supplying the password directly as part of the command line is not recommended

To define a genesis block and define the balance of the accounts

The genesis block is a json configuration that can be generated with puppeth binary or copied as it is as an example from documentation.

puppeth --network=myprivatenetwork
# console asks interactively about actions to take
# we want only the genesis block being generated

...

INFO [...] Administering Ethereum network           name=myprivatenetwork
WARN [...] No previous configurations found         path=$HOME/.puppeth/myprivatenetwork

What would you like to do? (default = stats)
 1. Show network stats
 2. Configure new genesis
 3. Track new remote server
 4. Deploy network components
> 2

What would you like to do? (default = create)
 1. Create new genesis from scratch
 2. Import already existing genesis
> 1

Which consensus engine to use? (default = clique)
 1. Ethash - proof-of-work
 2. Clique - proof-of-authority
> 1

Which accounts should be pre-funded? (advisable at least one)
> 0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9
> 0x

Should the precompile-addresses (0x1 .. 0xff) be pre-funded with 1 wei? (advisable yes)
> n

Specify your chain/network ID if you want an explicit one (default = random)
>
INFO [...] Configured new genesis block

What would you like to do? (default = stats)
 1. Show network stats
 2. Manage existing genesis
 3. Track new remote server
 4. Deploy network components
> 2

 1. Modify existing configurations
 2. Export genesis configurations
 3. Remove genesis configuration
> 2

Which folder to save the genesis specs into? (default = current)
  Will create myprivatenetwork.json, myprivatenetwork-aleth.json, myprivatenetwork-harmony.json, myprivatenetwork-parity.json
> data/genesis
INFO [...] Saved native genesis chain spec          path=data/genesis/myprivatenetwork.json
INFO [...] Saved genesis chain spec                 client=aleth path=data/genesis/myprivatenetwork-aleth.json
INFO [...] Saved genesis chain spec                 client=parity path=data/genesis/myprivatenetwork-parity.json
INFO [...] Saved genesis chain spec                 client=harmony path=data/genesis/myprivatenetwork-harmony.json

> ^C

We defined the 'proof-of-work' to be used and made the address of the created account to be pre-funded.

The configuraiton of the genesis block is exported under data/genesis (puppeth stores the configuration under $HOME/.puppeth/myprivatenetwork).

Initialize Geth with the genesis block configuration

geth init --datadir "$DATADIR" "$DATADIR"/myprivatenetwork.json

Starting Geth

see more at https://geth.ethereum.org/docs/interface/private-network

geth \
  --datadir "$DATADIR" \
  --networkid 21110 -nodiscover \
  --unlock 0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9 --password <(echo 'P422wORd') --allow-insecure-unlock \
  -http --http.corsdomain '*' --ws --ws.origins='*' \
  --mine \
  --miner.threads=1 \
  --miner.etherbase=0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9

`--datadir "$DATADIR"`	data directory with accounts
`-networkid <id>`	network identifier as defined in the genesis file (`chainId`, use the same)
`-nodiscover`	do not discover (not necessary, only one node here)
`--unlock <address> --password <password-file> --allow-insecure-unlock`	need to unlock account to be able to transfer from it
`-http --http.corsdomain '' --ws --ws.origins=''`	permit HTTP and WebSocket RPC + do not worry about CQRS (may connect eg. from Remix)
`--http.api web3,eth,debug,personal,net`	possible to restrict APIs available on HTTP RPC calls in console
`--mine`	to mine proof-of-work (defined in genesis block json configuration)
`--miner.threads=1`	to run at one CPU core
`--miner.etherbase=<address>`	to assign the rewards for mining to this address

Note	unlock other accounts could be done in console with `web3.personal.unlockAccount(<account>, <password>)`

Note	as a shortcut of running geth at localhost one may use `geth --dev`

Connect to JavaScript console of the started Geth node

# via HTTP RPC
geth attach http://localhost:8545
# or via local IPC (inter-process-communication)
geth attach "$DATADIR/geth.ipc"
# the other way of communication are websockets

List available accounts and transfer funds

eth.accounts
eth.getBalance(eth.accounts[0])
eth.sendTransaction({from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", to:eth.accounts[1], value: "22", password: "P422wORd"})
web3.fromWei(eth.getBalance(eth.accounts[1]), "ether")

Create a simple Solidity contract that emit log events

EchoContract is available at link:./contracts/EchoContract.sol
Build the contract

More details on building the Solidity contracts at https://docs.soliditylang.org/en/latest/installing-solidity.html
```
docker run -v $PWD/contracts:/sources ethereum/solc:stable -o /sources/output --abi --bin /sources/EchoContract.sol
```
Note
mounting the volume with podman may throw what(): boost::filesystem::status: Permission denied use :Z at -v $PWD/contracts:/sources:Z

Compilation output can be found at directory contracts/output.

Deploy contract

You can use Geth or use some GUI like Remix Ethereum for the same task.

cat contracts/output/EchoContract.abi
abi = eth.contract(<abi-file-content--copy&paste>)

cat cat contracts/output/EchoContract.bin
bytecode = '0x<bin-file-content--copy&paste>'

deploy = {from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", data:bytecode, gas: 2000000}
undefined
contract = abi.new("TEST", deploy)

# check the contract address
contract.address

Send a transaction to the contract to call it

eth.sendTransaction({from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", to:contract.address, value: "0"})

Subscribe to log events

One can use a the JSONRPC API (a stateless, remote procedure call protocol) or the pub/sub subscription to listen to log events in Geth.

Note	You can see the Geth sources where the filter creation done.

Pub/Sub WebSocket notification

We can go with JSON-RPC notifications connecting via websockets. As an example we can be listening to mined log events. Geth default WS port is 8546.

wscat -c ws://localhost:8546

# for latest logs
> {"id": 1, "method": "eth_subscribe", "params": ["logs", {}]}
# for pending logs
> {"id": 1, "method": "eth_subscribe", "params": ["logs", {"FromBlock":"pending","ToBlock":"pending"}]}
< {"jsonrpc":"2.0","id":1,"result":"0xb7982a3cc6c82c3147d49076a7cae610"}

# to get the log emitted there has to be called a contract with a event log emitted
# run from other console 'geth attach ...'
eth.sendTransaction({from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", to:contract.address, value: "0"})

< {"jsonrpc":"2.0","method":"eth_subscription","params":{"subscription":"0xb7982a3cc6c82c3147d49076a7cae610","result":{"address":"0x6181123d2c0b58ee9d65b89dab64d4611af21b26","topics":["0x030cacc26fa09af329d921d39728c2e63d3e54b638293f4d3d4b474dd96750f3","0x00000000000000000000000072b152b9ccc40be2a04f4075a1457cf8f753f7c9"],"data":"0x0000000000000000000000000000000000000000000000000000000000000030","blockNumber":"0xafe","transactionHash":"0x0c31eaaa4e19cf590228102d55295b3787c0757f7c12d8824426b78475a84265","transactionIndex":"0x0","blockHash":"0x8f257f83d21ecf61246474d6d7d919d273c37661f06de16361803b5663d4ca81","logIndex":"0x0","removed":false}}}

Note

The log shows the event emitted from Echo contract. First topic is the name of the function hashed with keccac-256.

keccac_256("Echo(address,uint256)") resutls in hash 030cacc26fa09af329d921d39728c2e63d3e54b638293f4d3d4b474dd96750f3 which is the value of the topic value.

The second value `0x00000000000000000000000072b152b9ccc40be2a04f4075a1457cf8f753f7c9` is the
`from` address padded to 32 bytes with `0`s.

The data contains the value of the counter variable as a hex number 0x30 is printf %d '0x30' which is 48.

In the similar way we can listen to the logs from the Geth console with the eth object.

geth attach http://localhost:8485

eth.filter("pending", function(err,txn) { console.log('Pending txn: ' + txn);})
eth.filter("latest", function(err,block) { console.log('Latest block: ' + block);})

eth.sendTransaction({from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", to:contract.address, value: "0"})
> Pending txn: 0xb8d5970948cf1177bdf9670029fa26e0f313e20a8752932de6636b3dc848cfb7
> Latest block: 0x771adf4c6dd52936801dd13b62311f9522546f8d91b99d432ccdb085d3b80025

# transaction receipt method shows data only when the block is mined
eth.getTransactionReceipt('0xb8d5970948cf1177bdf9670029fa26e0f313e20a8752932de6636b3dc848cfb7')

JSON-RPC polling mechanism

curl -X POST localhost:8545 -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_newFilter","params":[{}],"id":1}'
{"jsonrpc":"2.0","id":1,"result":"0xda402a379872cbd2dfc8016a4c60ff88"}

# any next call returns only the new changes
curl -X POST localhost:8545 -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getFilterChanges","params":["0xda402a379872cbd2dfc8016a4c60ff88"],"id":1}'

Stop mining and start listening to log events

# console [1]
geth attach "$DATADIR"/geth.ipc
> miner.stop()

# console [2]
wscat -c ws://localhost:8546
> {"id": 1, "method": "eth_subscribe", "params": ["logs", {"FromBlock":"pending","ToBlock":"pending"}]}

# console [1]
eth.sendTransaction({from:"0x72B152b9cCC40BE2a04F4075a1457cF8f753F7c9", to:contract.address, value: "0"})

# console [2]
# mining is stopped the pub/sub shows only pending data
< {"jsonrpc":"2.0","method":"eth_subscription","params":{"subscription":"0x48cd156d84db63fc06c53f7967d4250","result":{"address":"0x6181123d2c0b58ee9d65b89dab64d4611af21b26","topics":["0x030cacc26fa09af329d921d39728c2e63d3e54b638293f4d3d4b474dd96750f3","0x00000000000000000000000072b152b9ccc40be2a04f4075a1457cf8f753f7c9"],"data":"0x0000000000000000000000000000000000000000000000000000000000000035","blockNumber":"0xce1","transactionHash":"0x35937de17d4c3e00515d7ebb3e7416f3ba0e1db8e48068fc81598fe54afb81a7","transactionIndex":"0x1","blockHash":"0x0030ab214b44d0a63abda188ac994326c07eeb5d3df04212ba2e3524a313e49f","logIndex":"0x1","removed":false}}}

# console[1]
# mining can be started again
> miner.start(1)

A bit to code

Filter subscription API: https://github.com/ethereum/go-ethereum/blob/v1.10.9/eth/filters/api.go#L298
Pending logs subscription: https://github.com/ethereum/go-ethereum/blob/v1.10.9/eth/filters/filter_system.go#L204
Events processing loop: https://github.com/ethereum/go-ethereum/blob/v1.10.9/eth/filters/filter_system.go#L443
RPC call to code with reflection: https://github.com/ethereum/go-ethereum/blob/v1.10.9/rpc/service.go#L184

Our coding experiments

We did some changes around filters and pending transaction logs here https://github.com/ochaloup/go-ethereum/commits/filter-with-txn-data-instead-of-hashes

wscat -c ws://localhost:8546

> {"id": 1, "method": "eth_subscribe", "params": ["newFilteredTransactions", {"method": "0xabcdefg"}]}

ochaloup / 20211009_workshop_geth_pending_logs Goto Github PK