Comments (10)
When implementing curve serialization, I have seen 3 different cases:
- 3 extra bits, like BLS12-381, the extra bits are used for compressed form, infinity point, sign, Zcash serialization is the defacto standard and is mentioned in the IETF: https://www.ietf.org/archive/id/draft-irtf-cfrg-pairing-friendly-curves-11.html#name-zcash-serialization-format-
- 2 extra bits, like BN254, 1 bit has to be used for the sign, for the rest I would assume arkworks follow libsnarks, as libsnarks was the original that started all, I think halo2curves should follow suit.
- 1 extra bit, like Banderwagon and Pasta curves. That extra bit as to be used for the sign as we can derive compressed form or not from the length, and inifnity by having all zeros content. The only serialization standard followed by multi repos I'm aware of is from @kevaundray for Banderwagon (note, Ristretto/Decaf like construction so serialization is somewhat special: https://hackmd.io/@6iQDuIePQjyYBqDChYw_jg/BJBNcv9fq#Serialisation)
I've stopped myself from ranting against BE. Just letting you know I hate this... It should've all been LE.
BigInts/Fields are always serialized in BE, it's also natural when hex debugging. See I2OSP and OS2IP from RSA spec: https://www.rfc-editor.org/rfc/rfc8017.html#section-4
Additionally, if needed, Fp2 and Fp12 serializations might need standardization as well.
For Fp2, Zcash serializes extension field first, i.e. the i element then the base field.
For Fp12, the serialization should be canonical instead of being tied to a specific towering Fp2 -> Fp6 -> Fp12 vs Fp2 -> Fp4 -> Fp12. See discussion in supranational/blst#101 (comment)
from halo2curves.
related discussion on arkworks arkworks-rs/algebra#708
from halo2curves.
Ohhh! That's really unfortunate!
@kilic was the original creator of the bn256 inclusion PR. So he might know better about this specific nuance.
I agree we should make it unified such that we don't bring extra trouble to the usears.
Do you have any specific formatting proposal BTW @nanne007 ?
from halo2curves.
Thank you for the reply!
Currently dont have any particular proposal in mind.
In fact, the serialization(compressed version) of bn254 in both projects are very similar except the bit sign.
I think if we can agree on this, maybe a common format of bn254 can be derived from the two projects.
The zcash serialization of bls12-381 sets a good standard here. https://www.ietf.org/archive/id/draft-irtf-cfrg-pairing-friendly-curves-11.html#name-zcash-serialization-format-
But for bn254, it can be harder, as there're already many different impls used on production.
apart from these two projects, ethereum uses big endian encoding for Field, see https://eips.ethereum.org/EIPS/eip-196#encoding .
This can be solved by add an option about le or be when calling serialization function like to_bytes
.
However, adding other option for the bit sign of a curve point's negative of y and point of infinity seems a little inconvenient.
from halo2curves.
But for bn254, it can be harder, as there're already many different impls used on production.
apart from these two projects, ethereum uses big endian encoding for Field, see https://eips.ethereum.org/EIPS/eip-196#encoding .
I've stopped myself from ranting against BE. Just letting you know I hate this... It should've all been LE.
On the rest, I agree, we can solve this on this way or another. I think we can flip the bit order in the compressed form such that the two ecosystems match together.
For the general formatting what we can do is offer a general serialization impl which has arbitrary endianness based on an option as you say and sign compression (it's the default IMO).
For the rest, we can define an SerializedExt
trait or whatever where we can implement extra serialization options which are more optimal for certain use cases each.
Any idea how could we formalize this a bit more? We'll be happy to jump into a call, clarify this and update it!
At the end, a smooth transition from one ecosystem to the other would be the most desired thing to have!
from halo2curves.
Unifying the formatting makes sense for the community. Add @huyuncong , maybe he knows someone from arkworks who can take a look.
from halo2curves.
Maybe we can try summoning @Pratyush and he can orient us. Also curious to know his thoughts.
from halo2curves.
There is an extra case, the curves that have no extra bits for flags, like secp256k1
. For this curve we are adding an extra byte to hold the flags in both the compressed an uncompressed format. Bitcoin's implementation does this as well:
https://github.com/bitcoin-core/secp256k1/blob/77af1da9f631fa622fb5b5895fd27be431432368/src/eckey_impl.h#L37
I found a couple more issues.
The infinity flag is the 7th bit in the compressed form:
halo2curves/src/derive/curve.rs
Line 109 in bdb27cc
But it is the 6th in the uncompressed form:
halo2curves/src/derive/curve.rs
Line 235 in bdb27cc
The way of determining the sign seems not to be agreed upon either. In fact, we are not getting the sign but the parity of the y-coordinate.
from halo2curves.
The way of determining the sign seems not to be agreed upon either. In fact, we are not getting the sign but the parity of the y-coordinate.
Note that "sign" is a convention that splits a cyclic group in 2 halves.
When standardizing hash-to-curve, the "sign" was changed to mean x mod 2:
- 2019, sgn0 with little endian and big-endian variants: cfrg/draft-irtf-cfrg-hash-to-curve#144 / cfrg/draft-irtf-cfrg-hash-to-curve#176
- 2021: breaking spec change to parity - cfrg/draft-irtf-cfrg-hash-to-curve#225
from halo2curves.
and should notice that sign of bn254 curve in halo2curves and arkworks have different meaning
in arkworks:
the negative sign always means the bigger one, see the code:
while in halo2curves:
sign is the smallest bit of Y
, as y
and -y
have different even-old parity, so use last bit can distinguish them.
see the code:
That means one canno distinguish current y is the smaller one or the bigger one based on the sign in halo2curves impl.
from halo2curves.
Related Issues (20)
- Different output of bn256::G1::hash_to_curve between vanilla & asm branches HOT 11
- Refactor `BN` curves. HOT 1
- MSRV broken again in CI by dependencies HOT 1
- Add function in new_curve_impl has a wrong result HOT 1
- Hitting import issues on x86_64-unknown-linux-gnu from maybe_rayon crate HOT 4
- Add more info in regards impossibility to compile the lib with `multicore` feature to `wasm` targets HOT 1
- Remove `multicore` feature from `msm` bench HOT 1
- Enabling serialization feature in `pasta_curves`
- subtle v2.4 causes compilation errors HOT 1
- Add typos tool to CI to automate typo detection
- Re-define & unify the serialization for all primitives here and in any downstream crates
- Support various hash functions at `expand_message` part of hashing to curve
- Bad subgroup check or cofactor clearing in G2 pluto
- Continue `multiexp_serial` skips doubling when all bits are zero.
- RFC: Move MSM and FFT in this repo and offer a standard interface HOT 2
- Fix error in WithSmallOrderMulGroup impl for Bn254::Fr HOT 2
- Duplicated `mul_by_nonresidue` function in `Bn256` field extensions HOT 1
- Simplify `frobenius_map` in `bn256` HOT 1
- New crate release HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from halo2curves.