[istanbul nodes] Failed to prepare header for mining - err="invalid extra data format" about qubernetes

[libby]

Hey @yuyat what does your qubernetes.yaml look like? Are you generating the API resources or using the examples?

[yuyat]

Hi @libby, I am generating API resources.
2 nodes + istanbul + tessera

qubernetes.yaml:

namespace:
  name: quorum-test
# number of nodes to deploy
nodes:
  number: 2
service:
  # NodePort | ClusterIP
  type: ClusterIP
quorum:
  # supported: raft | istanbul
  consensus: istanbul
  # base quorum data dir as set inside each container.
  Node_DataDir: /etc/quorum/qdata
  # This is where all the keys are store, and/or where they are generated, as in the case of quorum-keygen.
  # Either full or relative paths on the machine generating the config
  Key_Dir_Base: 7nodes
  Permissioned_Nodes_File: 7nodes/permissioned-nodes.json
  Genesis_File: 7nodes/istanbul-genesis.json
  # related to quorum containers
  quorum:
    Raft_Port: 50401
    # container images at https://hub.docker.com/u/quorumengineering/
    Quorum_Version: 2.2.3
  # related to transaction manager containers
  tm:
    # (tessera|constellation)
    Name: tessera
    Tm_Version: 0.9.2
    Port: 9001
    Tessera_Config_Dir: 7nodes

# generic geth related options
geth:
  Node_WSPort: 8546
  NodeP2P_ListenAddr: 21000
  network:
    # network id (1: mainnet, 3: ropsten, 4: rinkeby ... )
    id: 10
    # public (true|false) is it a public network?
    public: false
  # general verbosity of geth [1..5]
  verbosity: 9

[libby]

I think he issue is it is using the 7nodes/istanbul-genesis.json and not generating a fresh one.
Try updating the config paths (genesis.json, permissioned-nodes.json) to paths you wish to generate. Then run ./quorum-init to generate the appropriate genesis.json, permissioned-nodes.json, etc. .
note: a minimum network size for Istanbul should be 3f +1 == 4.
Example config (change to 4 ;-) but I tested this):

#namespace:
#  name: quorum-test
# number of nodes to deploy
nodes:
  number: 2
service:
  # NodePort | ClusterIP
  type: NodePort
quorum:
  # supported: (raft | istanbul)
  consensus: istanbul
  # base quorum data dir as set inside each container.
  Node_DataDir: /etc/quorum/qdata
  # This is where all the keys are store, and/or where they are generated, as in the case of quorum-keygen.
  # Either full or relative paths on the machine generating the config
  Key_Dir_Base: out/config
  Permissioned_Nodes_File: out/config/permissioned-nodes.json
  Genesis_File: out/config/genesis.json
  # related to quorum containers
  quorum:
    Raft_Port: 50401
    # container images at https://hub.docker.com/u/quorumengineering/
    Quorum_Version: 2.2.3
  # related to transaction manager containers
  tm:
    # container images at https://hub.docker.com/u/quorumengineering/
    # (tessera|constellation)
    Name: tessera
    Tm_Version: 0.9.2
    Port: 9001
    Tessera_Config_Dir: out/config
  Data_Dir: /var/lib/docker/geth-storage

# generic geth related options
geth:
  Node_WSPort: 8545
  NodeP2P_ListenAddr: 30303
  network:
    # network id (1: mainnet, 3: ropsten, 4: rinkeby ... )
    id: 1101
    # public (true|false) is it a public network?
    public: false
  # general verbosity of geth [1..5]
  verbosity: 9
  # additional startup params to pass into geth/quorum
  Geth_Startup_Params: --rpccorsdomain=\"*\"

[yuyat]

I tried to modify the paths for the configuration files and set the minimum number of nodes to 4, but the problem persists. Is it possible to perform tests with two nodes, assuming that no errors will be tolerated?

The 7nodes / istanbul-genesis.json file is modified with each run of ./quorum-init. Its content is similar to if they are generated in another path. I think the problem comes in the generation of the extraData field.
This field is emptied with each execution of ./quorum-init (regardless of the file path), but I think it should contain the list of validators.

ERROR[09-23|10:30:03.582] Failed to decode message from payload address=0x280aF08C5A4bafD8a4035e826e45031608883960 err="unauthorized address"

Catch up round address=0x280aF08C5A4bafD8a4035e826e45031608883960 old_round=6 old_seq=1 old_proposer=nil new_round=7 new_seq=1 new_proposer="&{validators:[] policy:0 proposer:<nil> validatorMu:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} selector:0xb23930}"

[libby]

@yuyat can you please provide your qubernetes.yaml file?

[yuyat]

Sure. I changed the paths of the configuration files again to 7nodes/*

namespace:
  name: quorum-test
# number of nodes to deploy
nodes:
  number: 4
service:
  # NodePort | ClusterIP
  type: ClusterIP
quorum:
  # supported: raft | istanbul
  consensus: istanbul
  # base quorum data dir as set inside each container.
  Node_DataDir: /etc/quorum/qdata
  # This is where all the keys are store, and/or where they are generated, as in the case of quorum-keygen.
  # Either full or relative paths on the machine generating the config
  Key_Dir_Base: 7nodes
  Permissioned_Nodes_File: 7nodes/permissioned-nodes.json
  Genesis_File: 7nodes/istanbul-genesis.json
  # related to quorum containers
  quorum:
    Raft_Port: 50401
    # container images at https://hub.docker.com/u/quorumengineering/
    Quorum_Version: 2.2.3
  # related to transaction manager containers
  tm:
    # (tessera|constellation)
    Name: tessera
    Tm_Version: 0.9.2
    Port: 9001
    Tessera_Config_Dir: 7nodes

# generic geth related options
geth:
  Node_WSPort: 8546
  NodeP2P_ListenAddr: 21000
  network:
    # network id (1: mainnet, 3: ropsten, 4: rinkeby ... )
    id: 10
    # public (true|false) is it a public network?
    public: false
  # general verbosity of geth [1..5]
  verbosity: 9

[libby]

@yuyat when generating new kubernetes api resources, don't use the 7nodes static resources. Here is an example qubernetes.yaml you can use, which will generate the necessary resources into the default out directory. Copy and paste this into your qubernetes.yaml file and it should work after you run ./quorum-init:

#namespace:
#  name: quorum-test
# number of nodes to deploy
nodes:
  number: 4
service:
  # NodePort | ClusterIP
  type: NodePort
quorum:
  # supported: (raft | istanbul)
  consensus: istanbul
  # base quorum data dir as set inside each container.
  Node_DataDir: /etc/quorum/qdata
  # This is where all the keys are store, and/or where they are generated, as in the case of quorum-keygen.
  # Either full or relative paths on the machine generating the config
  Key_Dir_Base: out/config
  Permissioned_Nodes_File: out/config/permissioned-nodes.json
  Genesis_File: out/config/genesis.json
  # related to quorum containers
  quorum:
    Raft_Port: 50401
    # container images at https://hub.docker.com/u/quorumengineering/
    Quorum_Version: 2.2.3
  # related to transaction manager containers
  tm:
    # container images at https://hub.docker.com/u/quorumengineering/
    # (tessera|constellation)
    Name: tessera
    Tm_Version: 0.9.2
    Port: 9001
    Tessera_Config_Dir: out/config
  storage:
    # Host (requires hostPath) || PVC (Persistent_Volume_Claim - tested with GCP).
    Type: Host
    ## hostPath
    Data_Dir: /var/lib/docker/geth-storage

# generic geth related options
geth:
  Node_WSPort: 8545
  NodeP2P_ListenAddr: 30303
  network:
    # network id (1: mainnet, 3: ropsten, 4: rinkeby ... )
    id: 1101
    # public (true|false) is it a public network?
    public: false
  # general verbosity of geth [1..5]
  verbosity: 9
  # additional startup params to pass into geth/quorum
  Geth_Startup_Params: --rpccorsdomain=\"*\"

[libby]

@yuyat did that work for you? If so I'm going to close this issue.

[yuyat]

Hi @libby, unfortunately not.

The first problem is because I can't set Key_Dir_Base to out /config.
out /config does not contain the keys, and I can not move or generate keys in this folder because it is removed with each run.

For that reason, I changed the configuration as follows:

#namespace:
#  name: quorum-test
# number of nodes to deploy
nodes:
  number: 4
service:
  # NodePort | ClusterIP
  type: NodePort
quorum:
  # supported: (raft | istanbul)
  consensus: istanbul
  # base quorum data dir as set inside each container.
  Node_DataDir: /etc/quorum/qdata
  # This is where all the keys are store, and/or where they are generated, as in the case of quorum-keygen.
  # Either full or relative paths on the machine generating the config
  Key_Dir_Base: 7nodes
  Permissioned_Nodes_File: out/config/permissioned-nodes.json
  Genesis_File: out/config/genesis.json
  # related to quorum containers
  quorum:
    Raft_Port: 50401
    # container images at https://hub.docker.com/u/quorumengineering/
    Quorum_Version: 2.2.3
  # related to transaction manager containers
  tm:
    # container images at https://hub.docker.com/u/quorumengineering/
    # (tessera|constellation)
    Name: tessera
    Tm_Version: 0.9.2
    Port: 9001
    Tessera_Config_Dir: out/config
  storage:
    # Host (requires hostPath) || PVC (Persistent_Volume_Claim - tested with GCP).
    Type: Host
    ## hostPath
    Data_Dir: /var/lib/docker/geth-storage

# generic geth related options
geth:
  Node_WSPort: 8545
  NodeP2P_ListenAddr: 30303
  network:
    # network id (1: mainnet, 3: ropsten, 4: rinkeby ... )
    id: 1101
    # public (true|false) is it a public network?
    public: false
  # general verbosity of geth [1..5]
  verbosity: 9
  # additional startup params to pass into geth/quorum
  Geth_Startup_Params: --rpccorsdomain=\"*\"

This worked and created the configuration files inside out /config, but the problem persists, extraData is generated empty, in the same way when the 7nodes / istanbul-genesis.json was overwritten.

[libby]

@yuyat when prompted you need to 'Y' to delete and regenerate the out dir and the necessary resources. I'm going to close this issue and ask you to move this conversation over to slack as it is a conversation about how to use setup the project. Please ping me on and we can pick up the conversation there: https://go-quorum.slack.com/ signup link: http://bit.ly/quorum-slack