nilfoundation / zkllvm

Zero-Knowledge Proof Systems Circuit Compiler

zkLLVM Circuit Compiler

zkLLVM is a compiler from high-level programming languages into an input for provable computations protocols. It can be used to generate input for any arbitrary zero-knowledge proof system or protocol, which accepts input data in form of algebraic circuits It assumed to be used together with Placeholder proof system or any other arithmetization compatible with Placeholder proof system.

Every proof output from zkLLVM is an in-EVM verifiable one through the Proof Market. Use the Proof Market Toolchain repository (https://github.com/NilFoundation/proof-market-toolchain) to generate in-EVM verifiers.

Notice: zkLLVM is NOT a virtual machine and has nothing to do with it. It, moreover, with its existence proves the absence of necessity in zkVMs, posing them as redundant.

zkLLVM is designed as an extension to LLVM toolchain, thus supports any front end which compiles to LLVM IR. This enables developers to write code in native language instead of DSL's specific to other libraries.

zkLLVM extends:

1. clang/clang++ : Compiles the program into general intermediate representation byte-code from C/C++.
2. rustc: Compiles the program into general intermediate representation byte-code from Rust. (https://github.com/NilFoundation/zkllvm-rslang)
3. assigner Creates the circuit execution trace (a.k.a. assignment table) and produces data, needed by the prover to produce proof.

Languages currently supported are:

1. C/C++ (all the standards Clang 15 supports).
2. Rust (https://github.com/NilFoundation/zkllvm-rslang).
3. Your language suggestions are warmly welcomed in Telegram (https://t.me/nilfoundation) or on Discord (https://discord.gg/KmTAEjbmM3).

Building

Unix

Install Dependencies

• Boost >= 1.76.0
• CMake >= 3.5
• Clang >= 12.0
• Python >= 3.7
• Rust >= 1.68 (if you want to build Rust compiler)

On Debian systems, everything except Boost and Rust can be installed with the following command:

sudo apt install build-essential libssl-dev cmake clang-12 git curl pkg-config

1. Clone the repository

Clone the repository and all the submodules via:

git clone --recurse-submodules https://github.com/NilFoundation/zkLLVM.git
cd zkLLVM

2. Configure CMake

cmake -G "Unix Makefiles" -B ${ZKLLVM_BUILD:-build} -DCMAKE_BUILD_TYPE=Release .

Or use the below command instead, if you prefer Ninja build system (as we do, because Ninja works much faster).

cmake -G "Ninja" -B ${ZKLLVM_BUILD:-build} -DCMAKE_BUILD_TYPE=Release .

3. Build C++ compiler

If you are using Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} assigner clang -j$(nproc)

If you are using Ninja:

ninja -C ${ZKLLVM_BUILD:-build} assigner clang -j$(nproc)

4. Build Rust compiler

Make sure you have rustc with cargo installed first.

4.1 Reconfigure CMake, adding Rust tools

Unix makefiles:

cmake -G "Unix Makefiles" -B ${ZKLLVM_BUILD:-build} -DCMAKE_BUILD_TYPE=Release -DRSLANG_BUILD_EXTENDED=TRUE -DRSLANG_BUILD_TOOLS=cargo .

Ninja:

cmake -G "Ninja" -B ${ZKLLVM_BUILD:-build} -DCMAKE_BUILD_TYPE=Release -DRSLANG_BUILD_EXTENDED=TRUE -DRSLANG_BUILD_TOOLS=cargo .

4.2 Export path for loading LLVM libraries

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$(pwd)/build/libs/circifier/llvm/lib"

4.3 Build Rust compiler and Cargo

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} rslang -j$(nproc)

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} rslang -j$(nproc)

After that you will be able to call Cargo like that:

export RSLANG="$(pwd)/build/libs/rslang/build/host"
RUSTC=$RSLANG/stage1/bin/rustc $RSLANG/stage1-tools-bin/cargo --version

Note: if you want an advanced Rust compilation, you can build zkllvm first:

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} -j$(nproc)

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} -j$(nproc)

And then use Rust default build system x.py.

Pre-built binaries

Rust toolchain

You can install rslang as another toolchain in rustup or as a standalone application. Installation is done via rslang-installer.py script. It finds and downloads required release of rslang and installs it in the desired location.

Supported platforms

• x86-64 GNU/Linux

Prerequisites

• Python 3.7+

Installation with rustup

1. Install rustup as described on official page.

2. Install rslang:

Run this in your shell:

curl --proto '=https' --tlsv1.2 -sSf https://cdn.jsdelivr.net/gh/NilFoundation/zkllvm@master/rslang-installer.py | python - --channel nightly

You can also download the rslang-installer.py first and then run it:

curl --proto '=https' --tlsv1.2 -O https://cdn.jsdelivr.net/gh/NilFoundation/zkllvm@master/rslang-installer.py
python rslang-installer.py --channel nightly

Now you can use toolchain called zkllvm to compile with rslang:

rustc +zkllvm -V

Stanalone installation

To install rslang without rustup use --no-rustup argument. You will need to pass PATH to desired installation directory.

curl --proto '=https' --tlsv1.2 -O https://cdn.jsdelivr.net/gh/NilFoundation/zkllvm@master/rslang-installer.py
python rslang-installer.py --no-rustup --prefix PATH

Usage

zkLLVM's workflow is as follows:

1. Write Circuit : Users willing to prove some statement are supposed to implement an application in a language compatible with some frontend (C++ for now). This code will be compiled with a modified version of the clang compiler, which will output intermediate representation of the circuit.

   For the most performant cryptography circuits (e.g. hashes, signatures, VDFs, proof system verifications, etc.) we recommend using =nil; Foundation's Crypto3 library.

   The circuit developer will be generating the in-EVM applications for the circuits they have created. This will enable on-chain verification of the proof. The in-EVM logic consists of gate representations of the circuit. These contracts work in conjunction with the Placeholder proof validation in-EVM logic. The process to transpile the circuit into smart contracts is handled by the lorem-ipsum project.

2. Publish Circuit/Generate Proof: zkLLVM is tightly coupled with =nil; Foundation's Proof Market. Users willing to generate a proof for the circuit, will be matched with counter-parties based on price and other conditions. The circuit generated above needs to be published to proof market to enable this.

To generate a proof it is required to pass the following to the proof generator:

* Circuit : Arithmetization of the circuit.
* Inputs: Public (and private) inputs to circuit part of the proof request.

This generates the binary proof file. This flow is handled by the proof market toolchain repository & documented here.

Users can generate & inspect intermediate artifacts such as execution trace by running the assigner process. See examples below.

3. Verify Proof: Proof can be retrieved from the proof market and verified on chain. Users can verify proof in these modes :

   1. Offline : Tooling to support validation of off-chain proof will be added in the future.
   2. On-chain : This flow of generating smart contracts is handled by the lorem-ipsum project. A high level flow is described in the guides for circuit developer & proof verifier described above.

   Above we see how a dApp can use generated verifiers on-chain by simply including verification interfaces.

Examples

Linux

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(nproc)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(nproc)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas

macOS

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(sysctl -n hw.logicalcpu)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(sysctl -n hw.logicalcpu)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas

Validating the circuit

You can also run the assigner with --check flag to validate the satisfiability of the circuit. If the circuit is satisfiable, the assigner will output the satisfying assignment in the assignment.tbl file. If there is an error, the assigner will output the error message and throw an exception via std::abort.

Linux

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(nproc)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas --check

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(nproc)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas --check

macOS

Unix makefiles:

make -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(sysctl -n hw.logicalcpu)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas --check

Ninja:

ninja -C ${ZKLLVM_BUILD:-build} arithmetics_cpp_example -j$(sysctl -n hw.logicalcpu)
${ZKLLVM_BUILD:-build}/bin/assigner/assigner -b ${ZKLLVM_BUILD:-build}/examples/cpp/arithmetics_cpp_example.ll -i examples/inputs/arithmetics.inp -t assignment.tbl -c circuit.crct -e pallas --check

zkllvm's Issues

Add example with long constant

crypto3-zk compatibility

Fix inaccuracy in cmake workspace definition

Cmake workspace for this project is zkllvm. However, the crypto3 suite has crypto3 workspace inside and some cmake configs inside it throw warnings because of it.

Add zkml examples

Change dependency from crypto3

Make the repository dependant only on one repository.

Add example with global and dynamic memory

Merkle tree example compilation fails with code 139

Send curve_type into assigner as command line argument

Support building circuits with crypto3 include directories

Add list of crypto3 dependencies for circuits and pass corresponding include directories into clang.

Enable disabled examples

Command to compile own circuit

Guys, it's unclear how can I make my own .cpp compilation

make -C ${ZKLLVM_BUILD:-build} circuit_examples -j$(nproc)

what should I put instead of circuit_examples?

Add eddsa signature verification example

Add examples related to the kimchi verifier

Transpiler hardhat compatibility

Related to NilFoundation/zkllvm-transpiler#9 and NilFoundation/evm-placeholder-verification#48

Add deb package building

Make rslang a zkllvm submodule

rslang repo should be a submodule of zkllvm, not circifier, so move it to libs.

Use "assigner" target for clang to compile circuits

Add "-target assigner" to CircuitCompile.cmake

Github actions run tests CI

Use github Actions to automate tests execution (or manually trigger tests execution). Use self-hosted actions runners to run CI.

Github actions change patch version

Add file with library version. If necessary, the developer will change the major and minor versions of the library in it.

Make automatic change of library patch version in case of merge to master. (In case of successful closing of PR)

Extend examples set

Extend examples set to include:

• Merkle Tree example
• BLS12-381 example

Provide configuration for rslang build

Config file config.zkllvm.toml in rslang default configs will be removed. So in CMakeLists.txt we should provide configuration for building rslang using libs/rslang/configure script.

This also will provide more accurate paths to LLVM and allow to use absolute paths. As mentioned in this issue, absolute paths are required sometimes.

Verifier compatibility updates

Introduce verifier interface compatibility with NilFoundation/evm-placeholder-verification#17.

Add `rslang` installation script

The main issue is:

• NilFoundation/zkllvm-rslang/issues/20

As a result Python script and shell script invoking it should be added here.

Add static selectors amount assigner

Print stack trace on failure

Update transpiler to generate pure solidity files without yul(asm)

Warp transpiler does not support low-level Yul (asm) . We need to use warp to make our gates transpiled to cairo so they can be deployed on starknet.

We have two options here :
- Option 1: Output solidity without assembly -> Use warp (upcoming version) to generate cairo. We need to get a better estimate from nethermind team on when this version will be ready.
- Option 2: Output pure cairo 1.0

Add CI builds on macOS

Follow-up to #2: build zkllvm in CI on macOS, at least in one environment (os version × compiler × boost).

Support multiple source files for circuit

Add optimize option to transpiler app

--option allows to put multiple small gates in one .sol file.

It's compatible with:
NilFoundation/evm-placeholder-verification#37
NilFoundation/zkllvm-transpiler#3

Add example with memory operations

polynomial_example is broken

Refactor submodules definitions.

• ./.gitmodules

Build script for rslang

Build rslang from the root CMake file.

Linked branch should be merged only after merging rslang into circifier.

Fix uploading to S3

Enable constexpr compiling for field types

Use ccache in CI builds

Use ccache in CI builds on self-hosted Linux machines.

Options:

• Make a ccache server — (maybe) more effective, but more maintenance.
• Use local cache on the runner — less effective, requires little maintenance.
• https://github.com/hendrikmuhs/ccache-action for GitHub-hosted

Update submodules to enable more optimizations

Add BLS12-381 signature verification example

Update zkllvm examples

Support call instruction in assigner

Make a docker image with zkllvm and build dependencies

Devs who start working with zkllvm need a prepared environment. Besides installing zkllvm, they need to get other dependencies, and sometimes build a custom version of boost. It's not a good developer experience.

Implementation:

After building and releasing new packages build and publish a docker image.

https://hub.docker.com/r/nilfoundation/zkllvm

We can use tags for slim (just zkllvm) and fat/full builds (+cmake, boost, etc.).

Introduce optional Boost static library usage

Add curve types to clang

Add curve types to clang and IR, support basic operation for curves

Update guide to output ll files

Currently proof market does not take *bc files as input and needs *ll files.

Update hashes intrinsics

Sha2-256 and Poseidon intrinsics support has been recently added to circifier and assigner. We need to update the SDK and the submodules as well in order to show this functionality in the examples.

Add placeholder verifier example

https://github.com/NilFoundation/crypto3-zk/blob/master/include/nil/crypto3/zk/snark/systems/plonk/placeholder/verifier.hpp

Update submodules to enable building static libs

Fix curve arithmetic examples

Curve arithmetic examples on the old interface fail in the assigner with Core dump.

Support json format for public input

Deb packet for stdlib

Stdlib dev deb packet for installation