TEE-based Key Management System for Tendermint validators.
This repository contains tmkms-light
, a key management service intended to be deployed
in conjunction with Tendermint applications (ideally on separate physical hosts).
The code is based on the tmkms repository with the following differences:
- Smaller codebase with the following limitations:
- only one network can be configured in each
tmkms-light
process; - only the latest Tendermint protocol (v0.34) is supported;
- there is no support for HSMs;
- there is no support for transaction signing (
tx-signer
feature); - only
x86_64
Linux is supported.
- The code does not use conditional compilation (each signing provider has separate binaries) and dependencies on
abscissa
crates were replaced in order to fully supporttracing
for the application logging. - Signing in TEE (Trusted Execution Environments): currently, Intel(R) SGX and AWS Nitro Enclaves are supported.
Tendermint KMS Light is still in development (e.g. the SGX signing provider has not yet been signed, so that it could be launched in the production mode). In the future, the work developed in this repository may be upstreamed to the original tmkms repository.
The following signing backend providers are presently supported:
This is contained in the "providers/softsign" directory.
This is contained in the "providers/sgx" directory. There are two crates that need to be compiled separately:
tmkms-light-sgx-app
: this is the enclave application that needs to be compiled for thex86_64-fortanix-unknown-sgx
target, converted and signed as per EDP instructions;tmkms-light-sgx-runner
: this is the (host) runner application that is used to load the enclave application and interface with the host system.
First, Setup your sgx machine with EDP Installation guide
Build tmkms-light-sgx-runner
cargo build --target x86_64-unknown-linux-gnu -p tmkms-light-sgx-runner --release
Build tmkms-light-sgx-app
docker build -f Dockerfile.sgx --target export --output type=local,dest=./ .
The output enclave file is tmkms-light-sgx-app.sgxs
.
You can then follow EDP instructions for SGXS signing.
⚠️ For SGXS signing, the EDP instructions are shown for the "Debug" mode. For the production mode, remove the--debug
/-d
flags.
⚠️ For CPUs without the Flexible Launch Control feature (i.e. SGX v1), the enclave code needs to be signed with the RSA key previously approved by Intel in order to launch in the production mode.
Build tmkms-light-sgx-app
cargo build --target x86_64-fortanix-unknown-sgx -p tmkms-light-sgx-app --release
Follow EDP instructions for SGXS conversion and signing.
⚠️ For SGXS conversion, change--heap-size/--stack-size
value to0x40000
, and--threads 2
should be enough.
⚠️ For SGXS conversion and signing, the EDP instructions are shown for the "Debug" mode. For the production mode, remove the--debug
/-d
flags.
⚠️ For CPUs without the Flexible Launch Control feature (i.e. SGX v1), the enclave code needs to be signed with the RSA key previously approved by Intel in order to launch in the production mode.
- Place the generated
*.sgxs
and*.sig
files intotmkms/enclave
directory. - Place the
tmkms-light-sgx-runner
binary totmkms
directory.
tmkms init
Provide your chain bech32_prefix
$ tmkms-light-sgx-runner init -b bech32_prefix -p "bech32"
⚠️ For those who are running on Azure or other cloud environments, one may want to runinit
command with a cloud backup key. In cloud environments such as Azure,CPU-affinity
may not be guaranteed, so SGX “sealing” (a way to encrypt the validator key that only a particular CPU can decrypt the validator key) may not be fully relied on. Please followWith cloud backup key
if you intend to deploy in those settings.
With cloud backup key
TODO: these instructions are to be enhanced later on.
One may provide flag -e backup_key_path
which is to encrypt and decrypt consensus-key.backup
in directory specified in -k backup_data_path
. Before init
, you will need to run tmkms-light-sgx-runner cloud-wrap -s wrap_key_path -d
$ tmkms-light-sgx-runner recover -b bech32_prefix -p "bech32" -e backup_key_path -k backup_data_path -r
Or follow the example python script to run recover
Lastly, edit the generated tmkms.toml
to fit the target chain config, i.e chain_id and enclave_path
tmkms start
tmkms-light-sgx-runner start
This is contained in the "providers/nitro" directory. There are two crates that need to be compiled separately:
tmkms-nitro-enclave
: this is the enclave application that needs to be compiled for thex86_64-unknown-linux-musl
target if you are using the Alpine Linux (or equivalent) for the Docker file that gets converted to the enclave image file. The enclave application also needs to be linked against AWS Nitro Enclaves SDK etc., so make sure these libraries are present in your build environment -- see this AWS Dockerfile.tmkms-nitro-helper
: this is the application on the host that pushes the configuration to the enclave and provides extra vsock proxy connections to interface with the host system.
Follow the step 1 and step 4 here
$ cd tmkms-light
$ docker build -t aws-ne-build \
--build-arg USER=$(whoami) \
--build-arg USER_ID=$(id -u) \
--build-arg GROUP_ID=$(id -g) \
--build-arg RUST_TOOLCHAIN="1.56.1" \
--build-arg CTR_HOME="$CTR_HOME" \
-f Dockerfile.nitro .
#####2.2. build tmkms-nitro-helper
$ cd tmkms-light
$ cargo build --target x86_64-unknown-linux-gnu -p tmkms-nitro-helper --release
After build process finished, move the binary tmkms-nitro-helper
to one of your PATH
directories.
#####2.3. create eli file
Use the docker image aws-ne-build
which built in step 2.1 or just pull the docker image from dockerhub by using docker pull cryptocom/nitro-enclave-tmkms:latest
$ mkdir ~/.tmkms
$ nitro-cli build-enclave \
--docker-uri aws-ne-build \
--output-file ~/.tmkms/tmkms.eif
First, you need to start nitro enclave and generate the encrypted validator key and config files:
$ tmkms-nitro-helper enclave run --cpu-count 2
In another shell, you can check the enclave status:
$ tmkms-nitro-helper enclave info
the output should be:
[
{
"EnclaveID": "i-xxx-xxx",
"ProcessID": 31388,
"EnclaveCID": 1,
"NumberOfCPUs": 2,
"CPUIDs": [
1,
3
],
"MemoryMiB": 512,
"State": "RUNNING",
"Flags": "NONE"
}
]
create the encrypted validator key and config files:
$ mkdir ~/.tmkms && cd ~/.tmkms
$ tmkms-nitro-helper init \
-a $KMS_REGIN \
-k $KMS_KEY_ID \
-p bech32 \
-b crocnclconspub \
--cid $EnclaveCID
where:
KMS_REGIN
: where your AWS ec2 located likeap-southeast-1
, see more hereKMS_KEY_ID
: AWS Key Management Service key idEnclaveCID
: can get from commandtmkms-nitro-helper enclave info
It should encrypted validator signing key secrets/secret.key
, tmkms config file tmkms.toml
and enclave config file enclave.toml
, the output looks like:
public key: crocnclconspub1zcjduep..........tq48jchd
Nitro Enclave attestation:
hEShATgioFkRZKlpb.......................3L2Z28yKrDMTR
You need to start three components to make it work:
- start nitro enclave:
$ tmkms-nitro-helper enclave run --cpu-count 2 -v
- start vsock proxy:
$ tmkms-nitro-helper enclave vsock-proxy --remote-addr kms.ap-southeast-1.amazonaws.com
do remember to change the value of remote-addr
associated with your AWS region.
- start tmkms helper:
$ tmkms-nitro-helper start -c ./tmkms.toml -v
You can use -h
or --help
to see more options to start the three components
There is a handy command to start all the three components all in one:
$ cd ~/.tmkms
$ tmkms-nitro-helper launch-all -v