ethereum / c-kzg-4844 Goto Github PK
View Code? Open in Web Editor NEWA minimal implementation of the Polynomial Commitments API for EIP-4844 and EIP-7594, written in C.
License: Apache License 2.0
A minimal implementation of the Polynomial Commitments API for EIP-4844 and EIP-7594, written in C.
License: Apache License 2.0
In the spec for EIP-4844, the point evaluation precompile has the below line
assert verify_kzg_proof(commitment, z, y, kzg_proof)
but there is no method matching this interface exposed in the node-js bindings for this library. It looks like it does exist in the underlying C code for c-kzg
so could we have it added to the bindings as well so we can finish implementing the point evaluation precompile?
This may only pertain to the Java bindings for this function.
If a zero-length blob is passed to computeAggregateKzgProof
it will cause a segmentation fault. The problem is that this is allowed. I know that a blob should be 32 bytes, but the method allows arbitrary byte[]
types without length constraints.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000103059330, pid=53565, tid=5891
#
# JRE version: OpenJDK Runtime Environment Homebrew (11.0.16.1) (build 11.0.16.1+0)
# Java VM: OpenJDK 64-Bit Server VM Homebrew (11.0.16.1+0, mixed mode, tiered, compressed oops, g1 gc, bsd-aarch64)
# Problematic frame:
# C [libckzg4844jni.dylib+0x21330] blst_scalar_from_lendian+0x8
#
Excerpt of the stack trace:
C [libckzg4844jni.dylib+0x21330] blst_scalar_from_lendian+0x8
C [libckzg4844jni.dylib+0x39a8] Java_ethereum_ckzg4844_CKZG4844JNI_computeAggregateKzgProof+0x68
j ethereum.ckzg4844.CKZG4844JNI.computeAggregateKzgProof([BJ)[B+0
j tech.pegasys.teku.kzg.ckzg4844.CKZG4844.computeAggregateKzgProof(Ljava/util/List;)Ltech/pegasys/teku/kzg/KZGProof;+13
j tech.pegasys.teku.kzg.ckzg4844.CKZG4844PropertyTest.computeAggregateKzgProofThrowsExpected(Ljava/util/List;)V+11
v ~StubRoutines::call_stub
V [libjvm.dylib+0x32ffa8] JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x2e0
V [libjvm.dylib+0x5d60c0] invoke(InstanceKlass*, methodHandle const&, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*)+0x76c
V [libjvm.dylib+0x5d58f8] Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0x114
V [libjvm.dylib+0x3c2350] JVM_InvokeMethod+0x210
compute_aggregate_kzg_proof
calls poly_from_blob
:
Lines 1202 to 1205 in 130b629
poly_from_blob
calls bytes_to_bls_field
:
Lines 934 to 938 in 130b629
bytes_to_bls_field
calls blst_scalar_from_lendian
:
Lines 849 to 855 in 130b629
Associated with:
https://github.com/ethereum/c-kzg-4844/actions/runs/5813951682/job/15762618420?pr=330
It's a bit jumbled, but the benchmarks on Windows are overflowing the stack somehow:
Benchmarking verify_blob_kzg_proof
Benchmarking verify_blob_kzg_proof: Warming up for 3.0000 s
Benchmarking verify_blob_kzg_proof: Collecting 100 samples in estimated 5.0302 s (1300 iterations)
Benchmarking verify_blob_kzg_proof: Analyzing
Benchmarking verify_blob_kzg_proof_batch/1
Benchmarking verify_blob_kzg_proof_batch/1: Warming up for 3.0000 s
verify_blob_kzg_proof time: [3.8203 ms 3.8244 ms 3.8285 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
thread 'main' has overflowed its stack
error: bench failed, to rerun pass `--bench kzg_benches`
Caused by:
process didn't exit successfully: `D:\a\c-kzg-4844\c-kzg-4844\bindings\rust\target\release\deps\kzg_benches-887efed028bd7e9a.exe --bench` (exit code: 0xc00000fd, STATUS_STACK_OVERFLOW)
See #311 (comment)
┌─[tautvydas][kagamin][±][main ✓][~/Git/c-kzg-4844/bindings/rust]
└─▪ cargo bench
Compiling c-kzg v0.1.0 (/mnt/home/Git/c-kzg-4844/bindings/rust)
error[E0433]: failed to resolve: use of undeclared type `KZGSettings`
--> benches/kzg_benches.rs:31:33
|
31 | let kzg_settings = Arc::new(KZGSettings::load_trusted_setup_file(trusted_setup_file).unwrap());
| ^^^^^^^^^^^
| |
| use of undeclared type `KZGSettings`
| help: a struct with a similar name exists: `KzgSettings`
error[E0433]: failed to resolve: use of undeclared type `KZGCommitment`
--> benches/kzg_benches.rs:39:13
|
39 | KZGCommitment::blob_to_kzg_commitment(*blob, &kzg_settings)
| ^^^^^^^^^^^^^
| |
| use of undeclared type `KZGCommitment`
| help: a struct with a similar name exists: `KzgCommitment`
error[E0433]: failed to resolve: use of undeclared type `KZGProof`
--> benches/kzg_benches.rs:48:13
|
48 | KZGProof::compute_blob_kzg_proof(*blob, *commitment, &kzg_settings)
| ^^^^^^^^
| |
| use of undeclared type `KZGProof`
| help: a struct with a similar name exists: `KzgProof`
error[E0433]: failed to resolve: use of undeclared type `KZGCommitment`
--> benches/kzg_benches.rs:58:19
|
58 | b.iter(|| KZGCommitment::blob_to_kzg_commitment(*blobs.first().unwrap(), &kzg_settings))
| ^^^^^^^^^^^^^
| |
| use of undeclared type `KZGCommitment`
| help: a struct with a similar name exists: `KzgCommitment`
error[E0433]: failed to resolve: use of undeclared type `KZGProof`
--> benches/kzg_benches.rs:63:13
|
63 | KZGProof::compute_kzg_proof(
| ^^^^^^^^
| |
| use of undeclared type `KZGProof`
| help: a struct with a similar name exists: `KzgProof`
error[E0433]: failed to resolve: use of undeclared type `KZGProof`
--> benches/kzg_benches.rs:73:13
|
73 | KZGProof::compute_blob_kzg_proof(
| ^^^^^^^^
| |
| use of undeclared type `KZGProof`
| help: a struct with a similar name exists: `KzgProof`
error[E0433]: failed to resolve: use of undeclared type `KZGProof`
--> benches/kzg_benches.rs:83:13
|
83 | KZGProof::verify_blob_kzg_proof(
| ^^^^^^^^
| |
| use of undeclared type `KZGProof`
| help: a struct with a similar name exists: `KzgProof`
error[E0433]: failed to resolve: use of undeclared type `KZGProof`
--> benches/kzg_benches.rs:96:17
|
96 | KZGProof::verify_blob_kzg_proof_batch(
| ^^^^^^^^
| |
| use of undeclared type `KZGProof`
| help: a struct with a similar name exists: `KzgProof`
For more information about this error, try `rustc --explain E0433`.
error: could not compile `c-kzg` due to 8 previous errors
warning: build failed, waiting for other jobs to finish...
g1_mul()
is being too smart in an attempt to optimize for FFTs which we don't really use:
Lines 362 to 378 in ecc668b
I think we can probably make it look exactly like g2_mul()
without much loss in performance (maybe the current impl is faster when doing 1*P
which happens in batch verification, but IMO that's not sufficient gain to warrant this extra complexity).
So when looking at this again, I noticed a lack of length checks. I think there should be length checks for these types:
c-kzg-4844/bindings/node.js/kzg.ts
Lines 8 to 12 in 3ec2e9c
This should be updated after #92 is merged.
Some of the functions (mostly towards the bottom of the source file) are lacking documentation (doxygen) comments. I think it would be a good idea for someone to add these. Especially for the public/exported functions, but ideally all functions. I would do this myself, but I don't feel qualified to write these comments.
When publishing the C# package, nuget prints a warning about a missing readme:
Pushing Ckzg.Bindings.0.1.2.51.nupkg to 'https://www.nuget.org/api/v2/package'...
PUT https://www.nuget.org/api/v2/package/
warn : Readme missing. Go to https://aka.ms/nuget-include-readme learn How to include a readme file within the package.
Created https://www.nuget.org/api/v2/package/ 1174ms
Your package was pushed.
I think it would be good to add this, seems pretty simple:
Right now verify_kzg_proof()
(and verify_blob_kzg_proof()
and verify_kzg_proof_batch()
) will return C_KZG_RET
to indicate if the computation was finished properly, and then it will store the result of the verification in the argument ok
.
This means that their callers should check both the retval
and ok
to handle verification correctly. Given the crucial importance of getting verification correctly, it might be a good idea to simplify this two-layer checking. The worst case scenario is that a user of the library thinks that C_KZG_OK
means that verification went well.
My suggestion atm would be to merge ok
into retval
and make it such that verify_kzg_proof()
returning C_KZG_OK
denotes both a correct computation and also a correct verification. Then we could add another element to the enum called C_KZG_BAD_VERIFICATION
if the caller wants to handle that case in a special way. This way the verifier just needs to check retval
and I assume that for most clients it's gonna look like:
if retval == C_KZG_OK:
accept_tx
else:
reject_tx
but perhaps for others it could look like:
if retval == C_KZG_OK:
accept_tx
else if retval == C_KZG_MALLOC:
die()
else:
reject_tx
It still seems simpler than the status quo.
Current load_trusted_setup implementation takes a file as input
Line 773 in 9611bfd
To support browser light-clients we'll compile c-kzg to WASM, which would require no interaction with the file system. It would also make DX more flexible on how client implementations want to handle the trusted setup loading.
This was discussed offline in a chat and agreed to adopt a simple serialization format for the trusted setup. Just concat the point count and the serialized points in the following order.
G1Count | G2Count | [G1] | [G2]
just make makes .a lib file with includes. but those includes does not have
blob_to_kzg_commitment
compute_kzg_proof
compute_blob_kzg_proof
verify_kzg_proof
verify_blob_kzg_proof
verify_blob_kzg_proof_batch
Nice find. Maybe we suffer from a similar file descriptor leak in the nodejs bindings: https://github.com/ethereum/c-kzg-4844/blob/main/bindings/node.js/src/kzg.cxx#L142
Should we fix it as part of this PR or open a new one?
Originally posted by @asn-d6 in #241 (comment)
In this call:
Line 832 in 97c46a8
The 256 corresponds to the scalar bit width:
size_t blst_p1s_mult_pippenger_scratch_sizeof(size_t npoints);
void blst_p1s_mult_pippenger(blst_p1 *ret, const blst_p1_affine *const points[],
size_t npoints, const byte *const scalars[],
size_t nbits, limb_t *scratch);
void blst_p1s_tile_pippenger(blst_p1 *ret, const blst_p1_affine *const points[],
size_t npoints, const byte *const scalars[],
size_t nbits, limb_t *scratch,
size_t bit0, size_t window);
But the curve order is 255-bit hence 256 is doing extra work.
I don't think the cost is just 1/256 and even if it was, over thousands of points, that's a lot more computations to do, especially given that it's likely one of the biggest bottleneck.
I think we need a test for it and optionally a bench.
The function is used in g1_lincomb:
Lines 796 to 842 in c6fa137
which is used in the most "raw" manner in poly_to_kzg_commitment
Lines 946 to 957 in c6fa137
It seems that amd64 image of the C# bindings crash when being run on an M1 Mac via docker.
$ docker run --rm -it jtraglia/ckzg-csharp-tests
Unable to find image 'jtraglia/ckzg-csharp-tests:latest' locally
latest: Pulling from jtraglia/ckzg-csharp-tests
14726c8f7834: Pull complete
6c3981608c2b: Pull complete
a2e354eccee4: Pull complete
cbca4ad4689c: Pull complete
31e2a952779c: Pull complete
6ecbf1fb37e1: Pull complete
6d8c48eeb4a8: Pull complete
2e9754864261: Pull complete
3e544255b783: Pull complete
d48c03db7ca1: Pull complete
4f4fb700ef54: Pull complete
e48d511b25b5: Pull complete
Digest: sha256:3f3de9187639a237ad04a831444637a9491817402c72b19cd357bda8f42f44b2
Status: Downloaded newer image for jtraglia/ckzg-csharp-tests:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
cd ../../blst && ./build.sh -D__BLST_PORTABLE__
+ clang -O2 -fno-builtin -fPIC -Wall -Wextra -Werror -D__BLST_PORTABLE__ -c ./src/server.c
+ clang -O2 -fno-builtin -fPIC -Wall -Wextra -Werror -D__BLST_PORTABLE__ -c ./build/assembly.S
+ llvm-ar rc libblst.a assembly.o server.o
clang -fPIC -O2 -Wall -Wextra -shared -DFIELD_ELEMENTS_PER_BLOB=4096 -I../../src -I../../blst/bindings -o Ckzg.Bindings/runtimes/linux-x64/native/ckzg.so ckzg.c ../../src/c_kzg_4844.c ../../blst/libblst.a
dotnet build
MSBuild version 17.7.1+971bf70db for .NET
make: *** [Makefile:64: ckzg-dotnet] Segmentation fault
But the Go version of this does work, which implies it's C# specific:
$ docker run --rm -it jtraglia/ckzg-go-tests
Unable to find image 'jtraglia/ckzg-go-tests:latest' locally
latest: Pulling from jtraglia/ckzg-go-tests
785ef8b9b236: Already exists
5a6dad8f55ae: Already exists
bd36c7bfe5f4: Already exists
9e59e6b803ed: Already exists
9e40ac7456b5: Already exists
8e11a6c7a2e6: Already exists
98e1ff1d4522: Pull complete
0c6b6951a232: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:30f635d1dd9a0ee9cf8d401e13b89e9c98c0c2a2b06904b4345d178867bb105d
Status: Downloaded newer image for jtraglia/ckzg-go-tests:latest
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
go: downloading github.com/supranational/blst v0.3.11
go: downloading github.com/stretchr/testify v1.8.1
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/pmezard/go-difflib v1.0.0
go: downloading github.com/davecgh/go-spew v1.1.1
PASS
ok github.com/ethereum/c-kzg-4844/bindings/go 3.931s
Special thanks to @parithosh for pointing this out.
So I think I finally figured out how to provide Go bindings. I've made a proof of concept here:
You can import this package and use the exported C-KZG-4844 functions.
Is this something we want in this repository?
Benchmarks for other bindings are likely broken as well. It would be great if they were included in CI.
┌─[tautvydas][kagamin][±][main ✓][~/Git/c-kzg-4844/bindings/rust]
└─▪ cargo bench
Compiling c-kzg v0.1.0 (/mnt/home/Git/c-kzg-4844/bindings/rust)
error[E0308]: mismatched types
--> benches/kzg_benches.rs:56:61
|
56 | .verify_aggregate_kzg_proof(&blobs, &kzg_commitments, &kzg_settings)
| -------------------------- ^^^^^^^^^^^^^^^^ expected slice, found struct `Vec`
| |
| arguments to this function are incorrect
|
= note: expected reference `&[Bytes48]`
found reference `&Vec<c_kzg::KZGCommitment>`
note: associated function defined here
--> /mnt/home/Git/c-kzg-4844/bindings/rust/src/lib.rs:205:12
|
205 | pub fn verify_aggregate_kzg_proof(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
For more information about this error, try `rustc --explain E0308`.
error: could not compile `c-kzg` due to previous error
Currently there's a mix of byte arrays and internal types (like g1_t
) in the interface. It should all be byte arrays. The conversion functions (like bytes_to_g1
) should be removable from the interface.
Caught SIGILL in blst_cgo_init [...]
free(NULL)
is a no-op, so we can simplify code that currently looks like if (ptr != NULL) free(ptr);
to free(ptr);
See e.g. https://stackoverflow.com/questions/1938735/does-freeptr-where-ptr-is-null-corrupt-memory
This is a discussion from #196:
Just out of curiosity, what happened in f4c2749 ?
That was a finding from -fsantitize=memory
before it crashed with this error:
$ make sanitize_memory
Running memory sanitizer
FATAL: Code 0xaaab4cc9c1e0 is out of application range. Non-PIE build?
FATAL: MemorySanitizer can not mmap the shadow memory.
FATAL: Make sure to compile with -fPIE and to link with -pie.
FATAL: Disabling ASLR is known to cause this error.
FATAL: If running under GDB, try 'set disable-randomization off'.
I felt like because of the error, it was showing an incomplete list of findings. I plan to investigate why this is happening more later, but I suspect it's because blst
isn't built with -fsanitize=memory
. It seemed wrong to fix one instance when there are other instances of the same thing elsewhere, for example the original + one more:
Lines 228 to 232 in da83e45
Lines 243 to 248 in da83e45
I may make this change again later. I don't think they are crucial though.
Originally posted by @jtraglia in #196 (comment)
We should tweak our Makefile to run our unittests with ASan at some point, to detect various C memory issues that can happen.
Considering the conversation here ethereum/consensus-specs#3057 and ongoing discussion, just want to record that the C library may need to detect and fail on non-canonical inputs (i.e., greater than the modulus).
The current tests run against the (mainnet) trusted setup, which has a value of FIELD_ELEMENTS_PER_BLOB of 4096 and a correspondingly sized trusted setup.
However, the library actually takes FIELD_ELEMENTS_PER_BLOB as an external input (defined in the Makefile as a compile-flag). This suggest that the library should also work for a different value of FIELD_ELEMENTS_PER_BLOB. In particular, the way at least parts of the code are written suggest that the library should also work for FIELD_ELEMENTS_PER_BLOB not a power of 2.
With the test setup as is, it is not possible to check that (if it is even true in the first place).
A possible fix would be for the tests to create their own mock (un-)trusted setup of the correct size. (Such a mock trusted setup would basically choose a known tau rather than powers-of-tau ceremony-- note that this would not be secure in production).
I wrote an integration today for Geth for both the C and the Go libs. ethereum/go-ethereum#27155
Interestingly, blob to commitmets and proof generations with this lib seem to be 5-6x slower than the pure Go one, which seems be mean that something goes very wrong in compiling the C code: ethereum/go-ethereum#27155 (comment)
Any idea on how to investigate this?
This file is pretty out-of-date:
It says that it's a machine generated file though, not quite sure how to generate it.
c-kzg-4844/bindings/java/c_kzg_4844_jni.h
Lines 1 to 3 in edb1274
I'm surprised there are no complaints anywhere. Where is this used?
Is this repo the src for npm package?
Error log:
http://pastie.org/p/25xkd6j82tGM4x8gjQ4bMl
Cant get it installed on Ubuntu18, but I think this one flag it complains about -fPIC might solve the issue. on gcc11 seems to work fine.
The c-kzg public method verify_aggregate_kzg_proof()
expects that its blobs
and expected_kzg_commitments
arguments have length of size n
.
This is an undocumented requirement but still real because of the array accessing loop in
Lines 1290 to 1294 in c72ea8e
However, this is not respected by all bindings:
count
commitments presentcommitments[blob_index]
arrayI suggest we document this requirement in the core c-kzg library so that bindings know what to expect. We also need to fix the above two cases. We might also need some unittests that check this behavior.
Right now g1_lincomb()
will attempt to convert all points to affine in batch using blst_p1s_to_affine()
. However blst_p1s_to_affine()
might have undefined behavior if one of the input points is zero (and IIRC returns zero for the result).
This can cause issues in our batch verification routine by zeroing out the pairing, so we should make sure we are protected against it.
Perhaps one way to do it is to check if one of the inputs are zero, and if it is, pass the entire computation through the basic non-pippenger MSM path. Another approach would be to try to weed out the bad points from the list of points but this might be more troublesome to implement.
They currently live in https://github.com/henridf/nim-kzg-4844
This seems like a good idea so that we have everything in one place and they are also included in the auditing scope.
Some things that also need to happen:
We can perhaps add support for parsing the yaml after merging, just to get something in first.
/cc @henridf
Every so often, the build-ckzg-dotnet
CI action will fail with this error:
System.Exception: Unable to read beyond the end of the stream.
He's an instance of this error:
https://github.com/ethereum/c-kzg-4844/actions/runs/4485648662/jobs/7887434752?pr=238
Here's the full output:
cd bindings/csharp && dotnet test -c release --no-restore
shell: /usr/bin/bash -e {0}
env:
binding_build_number_based_version: 0.1.2.108
/home/runner/.nuget/packages/microsoft.build.tasks.git/1.1.1/build/Microsoft.Build.Tasks.Git.targets(25,5): warning : Could not find file '/home/runner/work/c-kzg-4844/c-kzg-4844/blst/.git'. The source code won't be available via Source Link. [/home/runner/work/c-kzg-4844/c-kzg-4844/bindings/csharp/Ckzg.Bindings/Ckzg.Bindings.csproj]
Ckzg.Bindings -> /home/runner/work/c-kzg-4844/c-kzg-4844/bindings/csharp/Ckzg.Bindings/bin/Release/net6.0/Ckzg.Bindings.dll
Ckzg.Test -> /home/runner/work/c-kzg-4844/c-kzg-4844/bindings/csharp/Ckzg.Test/bin/Release/net7.0/Ckzg.Test.dll
Test run for /home/runner/work/c-kzg-4844/c-kzg-4844/bindings/csharp/Ckzg.Test/bin/Release/net7.0/Ckzg.Test.dll (.NETCoreApp,Version=v7.0)
Microsoft (R) Test Execution Command Line Tool Version 17.5.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
The active test run was aborted. Reason: Test host process crashed
Test Run Aborted with error System.Exception: One or more errors occurred.
---> System.Exception: Unable to read beyond the end of the stream.
at System.IO.BinaryReader.Read7BitEncodedInt()
at System.IO.BinaryReader.ReadString()
at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.LengthPrefixCommunicationChannel.NotifyDataAvailable()
at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.TcpClientExtensions.MessageLoopAsync(TcpClient client, ICommunicationChannel channel, Action`1 errorHandler, CancellationToken cancellationToken)
--- End of inner exception stack trace ---.
The node bindings (https://github.com/ethereum/c-kzg-4844/tree/main/bindings/node.js) seem to contain quite a few files and it makes it hard to review the actual code. Can we tidy them up somehow?
/cc @dgcoffman
The current approach to making the sha256 functions visible is to patch the header file. This works fine for bindings built via some script/make process, where patching is just a step amongst others to be executed.
In the case of Nim however, c bindings are integrated with the regular compilation (basically, the c code is just "inlined"), so there's no good place to insert a patching step.
For the time being I have forked this repo and added a change which removes the patch and instead includes sha256.h. Can similarly remove the patch here?
We'd like to use the c-kzg
bindings in revm
. Currently, we would have to import them as a git
dependency on this repository, but then we will not be able to release the revm
crates on crates.io ourselves, since git
dependencies are not allowed: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-git-repositories.
The nodejs bindings readme is pretty out of date. It still references the old interface.
There is a problem in the rust bindings (or c-kzg) that is causing an intermittent segfault. The following is output from running the rust bindings tests in a loop. If you can find them, there are two segfault errors in there.
jtraglia at xanax in ~/Projects/ethereum/c-kzg-4844/bindings/rust (main)
$ git rev-parse HEAD
50bf358c6d7641328520c2cb4a17ca6566fd9ffb
jtraglia at xanax in ~/Projects/ethereum/c-kzg-4844/bindings/rust (main)
$ while true; do cargo test --all --release --tests; done
Compiling autocfg v1.1.0
Compiling cfg-if v1.0.0
Compiling libc v0.2.137
Compiling proc-macro2 v1.0.49
Compiling quote v1.0.23
Compiling unicode-ident v1.0.6
Compiling syn v1.0.107
Compiling crossbeam-utils v0.8.14
Compiling serde_derive v1.0.150
Compiling serde v1.0.150
Compiling scopeguard v1.1.0
Compiling serde_json v1.0.89
Compiling either v1.8.0
Compiling rayon-core v1.10.1
Compiling itoa v1.0.4
Compiling os_str_bytes v6.4.1
Compiling hashbrown v0.12.3
Compiling plotters-backend v0.3.4
Compiling half v1.8.2
Compiling ryu v1.0.11
Compiling ciborium-io v0.2.0
Compiling cast v0.3.0
Compiling bitflags v1.3.2
Compiling regex-syntax v0.6.28
Compiling itertools v0.10.5
Compiling textwrap v0.16.0
Compiling clap_lex v0.2.4
Compiling plotters-svg v0.3.3
Compiling memoffset v0.7.1
Compiling crossbeam-epoch v0.9.13
Compiling indexmap v1.9.2
Compiling num-traits v0.2.15
Compiling ciborium-ll v0.2.0
Compiling same-file v1.0.6
Compiling ppv-lite86 v0.2.17
Compiling oorandom v11.1.3
Compiling walkdir v2.3.2
Compiling c-kzg v0.1.0 (/Users/jtraglia/Projects/ethereum/c-kzg-4844/bindings/rust)
Compiling lazy_static v1.4.0
Compiling anes v0.1.6
Compiling hex v0.4.3
Compiling crossbeam-channel v0.5.6
Compiling regex v1.7.0
Compiling num_cpus v1.15.0
Compiling getrandom v0.2.8
Compiling atty v0.2.14
Compiling rand_core v0.6.4
Compiling clap v3.2.23
Compiling criterion-plot v0.5.0
Compiling rand_chacha v0.3.1
Compiling crossbeam-deque v0.8.2
Compiling rand v0.8.5
Compiling rayon v1.6.1
Compiling plotters v0.3.4
Compiling ciborium v0.2.0
Compiling tinytemplate v1.2.1
Compiling criterion v0.4.0
Finished release [optimized] target(s) in 9.45s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.03s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.42s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.03s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_end_to_end ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.41s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.39s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.45s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.40s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.95s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.40s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.78s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.69s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.53s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.50s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.02s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.01s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.85s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.39s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.40s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.53s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.69s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.37s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_end_to_end ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.39s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.62s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
error: test failed, to rerun pass `--lib`
Caused by:
process didn't exit successfully: `/Users/jtraglia/Projects/ethereum/c-kzg-4844/bindings/rust/target/release/deps/c_kzg-28ad36221918cd1f` (signal: 11, SIGSEGV: invalid memory reference)
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.77s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.53s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.94s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.19s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.11s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.86s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.52s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 3.27s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.40s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.62s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_end_to_end ... ok
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.41s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.86s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.61s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.44s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.94s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
error: test failed, to rerun pass `--lib`
Caused by:
process didn't exit successfully: `/Users/jtraglia/Projects/ethereum/c-kzg-4844/bindings/rust/target/release/deps/c_kzg-28ad36221918cd1f` (signal: 11, SIGSEGV: invalid memory reference)
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
running 3 tests
test tests::test_verify_kzg_proof ... ok
test tests::test_compute_agg_proof ... ok
test tests::test_end_to_end ... ok
test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.53s
Finished release [optimized] target(s) in 0.03s
Running unittests src/lib.rs (target/release/deps/c_kzg-28ad36221918cd1f)
Currently the c-kzg
library uses #define
constants for the mainnet/minimal paramaters like FIELD_ELEMENTS_PER_BLOB
:
Lines 39 to 43 in cfdd9e5
This makes it very hard (but not impossible*) to link two copies of the library to enable switching between mainnet and minimal configurations at runtime.
*My attempt at doing this is documented here: sigp/lighthouse#4276. I had to use objcopy
to rename c-kzg
's symbols so that they wouldn't conflict. Needless to say this is not a sensible approach to take.
To do this better I was thinking the library could accept another #define
constant which it prepends to all symbol names. Then the downstream consumer of the library could easily avoid collisions, with names like ckzg_blob_to_kzg_commitment
and ckzgmin_blob_to_kzg_commitment
instead of just blob_to_kzg_commitment
. The ckzg
prefix could be the default if no prefix is specified.
Copying an idea of @asn-d6's from chat:
you can avoid a needless non-batched bls_modular_inverse in evaluate_polynomial_in_evaluation_form() by computing and caching inverse_width at boot time. then you wouldn't need:
fr_from_uint64(&tmp, FIELD_ELEMENTS_PER_BLOB);
fr_div(out, out, &tmp);and you could replace it with a multiplication by inverse_width instead
fr_batch_inv()
is used in c-kzg to implement the bls_modular_inverse()
of polynomial-commitments.md.
The documentation of the latter says return 0 for x == 0
, but fr_batch_inv()
would not have this behavior. In particular, due to using the batch inversion algorithm, if an input is zero it would also zero the entire output array and not just that index.
This should not be exploitable with the current protocol (since we are only inverting known values or RO outputs) but we should figure out how to handle this in a less suprirsing manner either way IMO.
Two approaches:
C_KZG_BADARGS
from fr_batch_inv()
if an input is zero (not the spec's behavior, but at least it's not a surprising behavior if we document it)(while we are at it, we should also see if we can ditch the prod
array and only do it with a single field element like gnark is doing. or i suspect we might be doing an extra field mul in the second loop)
(while we are at it, we should also document that fr_batch_inv
does not handle len==0)
Checking the code here : https://github.com/ethereum/c-kzg-4844/blob/main/src/c_kzg_4844.c#L498 It seems that we are not checking whether points are in the subgroup and are just checking whether points are in the group.
We merged the nodejs bindings in #147 but there is still polishing work to be done.
Please look at #147 for unresolved issues.
Most importantly:
VerifyBlobKzgProofBatch()
function is quite messy. How can we simplify it? (See #147 for some ideas)/cc @dapplion @dgcoffman
Currently the python bindings do not compile on MacOS. It seems like the "-Bsymbolic" compiler option is not available, and without it it's reporting lots of missing python symbols.
Tagging @hwwhww
This ticket tracks the changes that need to happen to the C# bindings to make them more defensive and validate their inputs: #146 (comment)
/cc @flcl42
I suggest adding compute_kzg_proof
to the interface as it is required in order to compute valid inputs to the point evaluation precompile.
It will also be useful for testing purposes (easier to generate test cases for point evaluation precompile).
Using a char buffer would be IMO a more compatible version of the current api.
Chars have a standard representation across platforms, while FILEs are a linux api that windows supports to some degree. The windows version of it are handles (HFILE). I see other languages are lenient on dealing with files/os paths but in general these don't exactly map since windows paths are encoded using wide chars. Under the current api we need to deal with these nuances at the ffi boundary. Using a char buffer instead we would delegate opening the file to the language implementation, and simply pass the contents to the c code
Is there a reason we are not also freeing the pointer s
in free_trusted_setup()
and the bindings need to do it themselves?
Two things to check here:
free_trusted_setup()
and also freeing the pointer (see #172).A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.