nervosnetwork / mercury

Building on top of ckb-indexer, Mercury provides handy integration features for Nervos CKB.

Mercury

About Mercury

Mercury is a tool that handles application development on CKB. In the Nervos ecosystem, analogically speaking, while CKB is the Linux kernel, Mercury is Ubuntu. Mercury is the service layer providing interfaces for CKB. The support for CKB core interfaces and other practical functionalities of Mercury can significantly reduce the workload for developers. For developing wallet applications, Mercury offers the interfaces to get balance of an address and to assemble transactions for transferring CKBytes, sUDT or xUDT. For exchanges, Mercury provides functions such as aggregating digital assets and fetching blocks.

Mercury is the bridge between CKB and applications. It provides useful RPC services for DApps that are built upon Lumos and applications that are built upon ckb-sdk (java /go), such as wallets and exchanges. Furthermore, Mercury fetches data from CKB, processes the data, and implements efficient functions based on the core interfaces of CKB.

So far, Mercury has implemented a series of practical interfaces for wallet and exchange applications. More features are in development.

Long-term Support(LTS)

The v0.2 release has been designated as the LTS release. We suggest using the 0.2 version. The migration instructions for upgrading from other versions to the latest version are here.

Usage

There are three ways to use Mercury.

1. Quick Experience

The Mercury official provides public servers for a quick experience of Mercury.

For version 0.2, The request url for mainnet is https://Mercury-mainnet.ckbapp.dev/0.2 , for testnet is https://Mercury-testnet.ckbapp.dev/0.2 .

For version 0.3, the request url for mainnet is https://Mercury-mainnet.ckbapp.dev/0.3 , for testnet is https://Mercury-testnet.ckbapp.dev/0.3 .

For version 0.4, the request url for mainnet is https://Mercury-mainnet.ckbapp.dev/0.4 , for testnet is https://Mercury-testnet.ckbapp.dev/0.4 .

For example, you can use the following command to call a Mercury API method to view the version.

echo '{
    "id": 1234,
    "jsonrpc": "2.0",
    "method": "get_mercury_info",
    "params": []
}' \
| tr -d '\n' \
| curl -H 'content-type: application/json' -d @- https://Mercury-testnet.ckbapp.dev

Attention
Public servers do not guarantee high availability and high performance. If you want to use Mercury in a production project, please deploy and run Mercury on yourself.

2. Run Mercury Locally

• Step 1. Run a CKB node. If you already have a running node, skip this step.

  • run a mainnet node
  • run a testnet node

• Step 2. Edit mainnet_config.toml or testnet_config.toml according to mainnet or testnet. These config files are located in ./devtools/config/. The instrcution of config file is here.

• Step 3. Download the latest version of Mercury from the release page.

• Step 4. Create mercury tables if not exists.

$ psql mercury -U mercury -f ~/path/to/mercury/devtools/create_table/create_table.sql

• Step 5. Run Mercury.

# mainnet
$ Mercury -c ~/path/to/mercury/devtools/config/mainnet_config.toml run

# testnet
$ Mercury -c ~/path/to/mercury/devtools/config/testnet_config.toml run

Recommended Hardware

4 Cores - 8G Memory - 500G Disk and above.

If you use a standalone server to run the Postgres server, a 50G Disk is enough.

Expected Synchronization Duration

If Mercury connects with a synced CKB node, it takes 5-7 hours to catch up the mainnet tip or 10-14 hours to catch up the testnet tip.

3. Run Mercury via Docker

• Step 1. Edit docker_compose_config.toml according to your set. This config file is located in ./devtools/config/.

• Step 2. Edit ./docker-compose.yml to modify the runtime environment of CKB.

environment:
    CKB_NETWORK: mainnet

or

environment:
    CKB_NETWORK: testnet

• Step 3. Build Mercury images from the Dockerfile.

$ docker build -t Mercury .

• Step 4. Run Mercury via docker.

$ docker-compose up -d

or

$ docker run -d -p 8116:8116 -v {user_config_path}:/app/devtools/config Mercury:latest

Recommended Hardware

4 Cores - 8G Memory - 500G Disk and above.

Expected Synchronization Duration

The docker environment runs CKB node and Mercury from the genesis block. It takes 12-15 hours to catch up mainnet tip or 24-30 hours for testnet tip.

SDK Support

For now, two SDKs have supported Mercury: ckb-sdk-java and ckb-sdk-go.

License

Mercury is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The main branch is built and tested regularly, considered as production-ready; The dev branch is the work branch to merge new features, and it is not stable. The CHANGELOG is available in Releases in the main branch.

Minimum Supported Rust Version policy (MSRV)

The Mercury crate's minimum supported rust version is 1.56.1.

---

Documentation

• Mercury API Documentation
• Mercury Config Documentation
• Mercury Layout Documentation
• Mercury Setup Instructions

mercury's Issues

Need to update the ckb version in docker-compose file.

There is a hardfork of ckb mainnet on version v0.103.0 , but the ckb version in mercury docker-compose file is still v0.101.0 .

A way to query for a deposited Nervos DAO balance.

What would you like to be added:

A way to query for a deposited Nervos DAO balance of an address using the Mercury RPC.

Why is this needed:

Wallets and exchanges that want to support the Nervos DAO would benefit from a way to query for a Nervos DAO balance that they can display to their users.

DB error error returned from database: bigint out of range

What happened:

➜  mercury git:(main) ✗ curl -X POST http://localhost:8116 -H 'Content-Type: application/json' -d @- <<BODY
{
  "id": 42,
  "jsonrpc": "2.0",
  "method": "get_block_info",
  "params": [
    {
      "block_number": null,
      "block_hash": null
    }
  ]
}
BODY
{"jsonrpc":"2.0","error":{"code":-32000,"message":"Mercury Rpc Error code -11005, error DB error error returned from database: bigint out of range"},"id":42}%  

What you expected to happen:

Return latest block

How to reproduce it (as minimally and precisely as possible):

Start postgres:

docker run -p 5432:8432 -v /home/kuzi/projects/mercury/devtools/create_table/create_table.sql:/docker-entrypoint-initdb.d/create_tables.sql -e POSTGRES_USER=postgres -e POSTGRES_DB=mercury -e POSTGRES_PASSWORD=123456 postgres:alpine

Use this mainnet config: https://gist.github.com/Kuzirashi/7be03450748ed41e3aba81a589d33aa1

Anything else we need to know?:

Environment:

• Mercury version or commit hash (git log -1): a010462
• OS (e.g: cat /etc/os-release): Ubuntu 20.04
• Kernel (e.g. uname -a): Linux DESKTOP-H3DBUQR 4.19.104-microsoft-standard SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Others:

Improper spelling of "frozen" in terminology.

What happened:

In the terminology used and in the API, the key "freezed" is used. This is the incorrect spelling in modern usage.

What you expected to happen:

The correct spelling is "frozen".

• Mercury version or commit hash (git log -1): v0.2.0 (stable)

Sync block witch ckb_v0.100.0 failed.

What happened:
Error happens when syncing blocks:

sync block [3864136, 4462438, 638972, 377823, 1909804, 1076613, 4558490, 5157811, 2223359, 3343764] error Mercury Rpc Error DecodeJson("missing field `uncles_hash`")

What you expected to happen:
Hope it supports ckb_v0.100.0 . By the way I do not want to resync database with ckb_v0.43.0, all our nodes have upgraded to v0.100.0 now.

How to reproduce it (as minimally and precisely as possible):
Just run it with a ckb_v0.100.0 node.

Anything else we need to know?:
No

Environment:
Runs in Docker version 20.10.7, build f0df350

• Mercury version or commit hash (git log -1): v0.2.0-beta.2
• Docker Image: nervos/mercury:v0.2.0-beta.2

Thanks for any helps.

Strange "Free" value by getting balance

Can anybody explain to me, why such things happen?

Address: ckt1qyq8fh2x2vjtz6hapzgpgwqm8nmu37tm845szcjh7j
(FULL format: ckt1qjda0cr08m85hc8jlnfp3zer7xulejywt49kt2rr0vthywaa50xwsaxagefjfvt2l5yfq9pcrv700j8e0v7kjcl97c7)

First screenshot before I made faucet for 10000 CKB:

Second after I made faucet for 10000 CKB, and some amount added to "occupied" value:

If reason in the minimum amount of 61 CKB for cell, so why I could transfer to that amount?
Anyway, explorer shows right (in my logic) balance:

https://explorer.nervos.org/aggron/address/ckt1qjda0cr08m85hc8jlnfp3zer7xulejywt49kt2rr0vthywaa50xwsaxagefjfvt2l5yfq9pcrv700j8e0v7kjcl97c7

Thank you in advance!

Mercury free balance does not reflect all capacity available in an a DAO cell that is in the withdrawal phase and mature.

The free balance being returned by Mercury does not seem to reflect all of the capacity available in this specific instance with a DAO cell in the withdrawal phase that is mature.

There are seven cells currently on ckb1...mkus:

CKB	Cell
200 CKB	DAO (deposit phase)
200 CKB	DAO (deposit phase)
200 CKB	DAO (deposit phase)
200 CKB	DAO (withdrawal phase mature)
1227.37 CKB	Basic Cell
139.20 CKB	Basic Cell
200 CKB	Basic Cell

Balances from Mercury are:

key	value
Free	1664.57737955 CKB
Occupied	408 CKB
Frozen	294CKB

To calculate the free amount of 1664, Mercury includes the 98 CKB from the DAO cell in the withdrawal phase but excludes the 102 CKB occupied capacity by that cell. 200 - 102 + 1227 + 139 + 200 = 1664

Mercury is generating transactions using the DAO cell in the withdrawal phase since it is mature. The 102 CKB required on the cell should not be subtracted in this case because this cell can be spent the same as a basic cell. 200 + 1227 + 139 + 200 = 1766

This amount of 1766 is relevant to developers because they need a way to represent what balance is available for the user to send using Mercury to generate transactions.

Raw requests are below:

echo '{
  "id": 42,
  "jsonrpc": "2.0",
  "method": "get_balance",
  "params": [
    {
      "item": {
        "type": "Address",
        "value": "ckb1qzda0cr08m85hc8jlnfp3zer7xulejywt49kt2rr0vthywaa50xwsq0nmjcs850x9c46thjshlw9nlwsq3ztpsc75mkus"
      },
      "asset_infos": [
        {
          "asset_type": "CKB",
          "udt_hash": "0x0000000000000000000000000000000000000000000000000000000000000000"
        }
      ],
      "tip_block_number": null
    }
  ]
}' \
| tr -d '\n' \
| curl -H 'content-type: application/json' -d @- https://Mercury-mainnet.ckbapp.dev/0.4

{
  "jsonrpc": "2.0",
  "result": {
    "balances": [
      {
        "ownership": "ckb1qzda0cr08m85hc8jlnfp3zer7xulejywt49kt2rr0vthywaa50xwsq0nmjcs850x9c46thjshlw9nlwsq3ztpsc75mkus",
        "asset_info": {
          "asset_type": "CKB",
          "udt_hash": "0x0000000000000000000000000000000000000000000000000000000000000000"
        },
        "free": "0x26c1a782e3",
        "occupied": "0x97fde9800",
        "frozen": "0x6d8606600"
      }
    ],
    "tip_block_number": "0x7b123d"
  },
  "id": 42
}

echo '{
    "id": 2,
    "jsonrpc": "2.0",
    "method": "get_cells",
    "params": [
        {
            "script": {
                "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
                "hash_type": "type",
                "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3"
            },
            "script_type": "lock"
        },
        "asc",
        "0x64"
    ]
}' \
| curl -H 'content-type: application/json' -d @- \
https://mainnet.ckb.dev/indexer

{
  "jsonrpc": "2.0",
  "result": {
    "last_cursor": "0x409bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce801f3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3000000000079ef5b0000000100000000",
    "objects": [
      {
        "block_number": "0x743803",
        "out_point": {
          "index": "0x0",
          "tx_hash": "0xfad306ca6b3bbf1d3d066455b5e44a8ddaf1c157268315dc9a7f312348180066"
        },
        "output": {
          "capacity": "0x4a817c800",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": {
            "args": "0x",
            "code_hash": "0x82d76d1b75fe2fd9a27dfbaa65a039221a380d76c926f378d3f81cf3e7e13f2e",
            "hash_type": "type"
          }
        },
        "output_data": "0x0000000000000000",
        "tx_index": "0x1"
      },
      {
        "block_number": "0x74380c",
        "out_point": {
          "index": "0x0",
          "tx_hash": "0xe8d3ef8fe0679a160a4306eb7fd63ea2537d368b812a69890b1c38e95893a32e"
        },
        "output": {
          "capacity": "0x4a817c800",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": {
            "args": "0x",
            "code_hash": "0x82d76d1b75fe2fd9a27dfbaa65a039221a380d76c926f378d3f81cf3e7e13f2e",
            "hash_type": "type"
          }
        },
        "output_data": "0x0000000000000000",
        "tx_index": "0x4"
      },
      {
        "block_number": "0x75db78",
        "out_point": {
          "index": "0x0",
          "tx_hash": "0x368db5ab9d88ad1d398b514c99c7e923f107189d2cc713fa0a768ba4d48b8087"
        },
        "output": {
          "capacity": "0x4a817c800",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": {
            "args": "0x",
            "code_hash": "0x82d76d1b75fe2fd9a27dfbaa65a039221a380d76c926f378d3f81cf3e7e13f2e",
            "hash_type": "type"
          }
        },
        "output_data": "0x0000000000000000",
        "tx_index": "0x4"
      },
      {
        "block_number": "0x75db87",
        "out_point": {
          "index": "0x0",
          "tx_hash": "0x13dbd3261a8d3fe322327fbdb10e484980596a9432413e9b70cc1e317bef2354"
        },
        "output": {
          "capacity": "0x4a817c800",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": {
            "args": "0x",
            "code_hash": "0x82d76d1b75fe2fd9a27dfbaa65a039221a380d76c926f378d3f81cf3e7e13f2e",
            "hash_type": "type"
          }
        },
        "output_data": "0x41f8720000000000",
        "tx_index": "0x5"
      },
      {
        "block_number": "0x75db87",
        "out_point": {
          "index": "0x1",
          "tx_hash": "0x13dbd3261a8d3fe322327fbdb10e484980596a9432413e9b70cc1e317bef2354"
        },
        "output": {
          "capacity": "0x1c93b224de",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": null
        },
        "output_data": "0x",
        "tx_index": "0x5"
      },
      {
        "block_number": "0x782d3a",
        "out_point": {
          "index": "0x1",
          "tx_hash": "0xf0e7b75739b5aa028dfccbe8a73991bbebd6bc987bcba3c081c5a2146e7a712c"
        },
        "output": {
          "capacity": "0x33dbd7405",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": null
        },
        "output_data": "0x",
        "tx_index": "0x2"
      },
      {
        "block_number": "0x79ef5b",
        "out_point": {
          "index": "0x0",
          "tx_hash": "0xefc4098d5880f9b1ffa3fb59e9b18bbd456354f0a6273b45701390250cf3c5ba"
        },
        "output": {
          "capacity": "0x4a817c800",
          "lock": {
            "args": "0xf3dcb103d1e62e2ba5de50bfdc59fdd00444b0c3",
            "code_hash": "0x9bd7e06f3ecf4be0f2fcd2188b23f1b9fcc88e5d4b65a8637b17723bbda3cce8",
            "hash_type": "type"
          },
          "type": null
        },
        "output_data": "0x",
        "tx_index": "0x1"
      }
    ]
  },
  "id": 2
}

Syncing will be stuck after it was interrupted once

What happened:
I first stopped the mercury container while it is syncing, then when I start it again I found it log not updating any more, after enable debug log I found the mercury just keep requesting get_block_by_number API again and again.

Here is the log of mercury: http://tmp.link/f/6183a1c91695d

What you expected to happen:
Keep syncing after restart.

How to reproduce it (as minimally and precisely as possible):
Stop it while syncing or try it with my docker-compose configs here https://github.com/linkdesu/ckb-env .

Anything else we need to know?:
Why I stopped the container while it is syncing? Becasue I found the config log_path is a path to directory but not a file.
And when I config it to the directory of where the config.toml, in my container it is /data, it will not write log any more, I think it is may be a small bug, but it can be avoid by config, so I just metion it here.

Environment:

• Mercury version or commit hash (git log -1): v0.2.0-beta.3
• OS (e.g: cat /etc/os-release): Ubuntu 20.04.3 LTS
• Kernel (e.g. uname -a): Linux ubuntu-server 5.4.0-81-generic
• Others: Docker version 20.10.10, build b485636

Thanks for your help.

当mercury 启动失败时，没有任何错误信息？

版本号：mercury:v0.2.0-beta.4
部署好mercury 后，它一直在重启，而且没有生成日志目录，没有任何日志的情况下，该怎么判断它是什么原因导致启动失败？
有什么方法把启动失败的信息打印出来？

[log_config]
log_level = "error"
auto_split_log_file = true
log_path = "./free-space/mainnet/log"

free-space/mainnet/ 目录没有生产log 文件夹

Mercury is about to remove built-in Ckb Indexer RPCs

CKB is currently embedding Ckb Indexer internally, and the upcoming CKB version will support all Ckb Indexer Rpcs. At that time, users will no longer need the original Ckb Indexer or Mercury's built-in indexer RPCs support.

Based on the above, after the new ckb version is released, Mercury will remove the indexer RPCs.

Migrating from PostgreSQL 10 to a newer version

The database version in Mercury's current configuration is PostgreSQL 10. The newer version improves the experience and efficiency of building and scanning indexes, and it is necessary for us to migrate to the new version of PostgreSQL.

Remote Nodes

Hi, can anybody provide a list of remote mercury nodes for testnet and mainnet? Thx!

Integrated Omni Lock

Omni Lock is a full-featured Lock, and Mercury intends to gradually support it.

Blog: Omnilock, a Universal Lock that Powers Interoperability
RFC: 0042-omnilock

Mercury is currently known to have built-in support for some Lock Scripts, such as Secp, Acp, Pw, etc. Omni Lock will definitely not be the last Lock to be supported, and there will be new Lock contracts that need to be supported in the future. Therefore, the integration of Omni Lock this time needs to consider the convenience of lock script extension, which requires abstraction of lock script based on the needs of Mercury RPC.