Coder Social home page Coder Social logo

cyphrme / coze Goto Github PK

View Code? Open in Web Editor NEW
105.0 8.0 3.0 4.19 MB

Coze is a cryptographic JSON messaging specification.

Home Page: https://cyphr.me/coze

License: BSD 3-Clause "New" or "Revised" License

Go 100.00%
authentication cryptography json coze auth login jwt es256 es384 es512

coze's Introduction

pkg.go.dev

⚠️ Coze is in alpha. We appreciate feedback and contributions. Use at your own risk.

Coze

Coze

Coze is a cryptographic JSON messaging specification.

Try Coze out!

Presentation

Example Coze

{
	"pay": {
		"msg": "Coze Rocks",
		"alg": "ES256",
		"iat": 1623132000,
		"tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
		"typ": "cyphr.me/msg"
	},
	"sig": "Jl8Kt4nznAf0LGgO5yn_9HkGdY3ulvjg-NyRGzlmJzhncbTkFFn9jrwIwGoRAQYhjc88wmwFNH5u_rO56USo_w"
}

Coze Design Goals

  1. Idiomatic JSON
  2. Human readable
  3. Limited scope
  4. Providing defined cipher suites

Coze Fields

Coze defines standard fields for the objects pay, key, and coze. Applications may include additional fields as desired. While all fields are optional, omitting standard fields may limit compatibility. Binary values are encoded as RFC 4648 base 64 URI canonical with padding truncated (b64ut). JSON components are serialized into UTF-8 for signing, verification, and hashing. All JSON fields must be unique, and unmarshalling JSON with duplicate fields must result in an error.

All Coze Standard Fields

Coze Standard Fields

Pay

pay contains the fields alg, iat, tmb, and typ and optionally any additional application fields. In the first example msg is additional.

pay Standard Fields

  • alg - Specific cryptographic algorithm. E.g. "ES256"
  • iat - Unix time of message signature. E.g. 1623132000
  • tmb - Thumbprint of the signature's key. E.g. "cLj8vs..."
  • typ - Type of pay. E.g. "cyphr.me/msg"

typ's value may be used by applications as desired. The value is recommended to denote API information such as versioning, expected fields, and/or other application defined programmatic functions. In the first example, "typ":"cyphr.me/msg" denotes a pay with the fields ["msg","alg","iat","tmb","typ"] as defined by an application.

Coze Key

Example Public Coze Key

{
	"alg":"ES256",
	"iat":1623132000,
	"kid":"Zami's Majuscule Key.",
	"tmb":"cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
	"x":"2nTOaFVm2QLxmUO_SjgyscVHBtvHEfo2rq65MvgNRjORojq39Haq9rXNxvXxwba_Xj0F5vZibJR3isBdOWbo5g"
}

Example Private Coze Key

{
	"alg":"ES256",
	"iat":1623132000,
	"kid":"Zami's Majuscule Key.",
	"d":"bNstg4_H3m3SlROufwRSEgibLrBuRq9114OvdapcpVA",
	"tmb":"cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
	"x":"2nTOaFVm2QLxmUO_SjgyscVHBtvHEfo2rq65MvgNRjORojq39Haq9rXNxvXxwba_Xj0F5vZibJR3isBdOWbo5g"
}

key Standard Fields

  • key - Key object. E.g. "key":{"alg":"ES256", ...}
  • alg - Algorithm. E.g. "ES256"
  • d - Private component. E.g. "bNstg4..."
  • iat - "Issued at", Key creation Unix time. E.g. 1623132000
  • kid - "Key identifier", Non-programmatic label. E.g. "kid":"My Cyphr.me Key".
  • tmb - Thumbprint. E.g. "cLj8vs..."
  • x - Public component. E.g. "2nTOaF...".
  • typ - "Type", Application defined label. E.g. "cyphr.me/key"
  • rvk - "Revoke", Key revocation Unix time. E.g. 1623132000

Note that the private component d is not included in tmb generation. Also note that kid must not be used programmatically while typ may be used programmatically.

Coze object

The JSON name coze may be used to wrap a coze.

{
	"coze":{
		"pay": {
			"msg": "Coze Rocks",
			"alg": "ES256",
			"iat": 1623132000,
			"tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
			"typ": "cyphr.me/msg"
		},
		"sig": "Jl8Kt4nznAf0LGgO5yn_9HkGdY3ulvjg-NyRGzlmJzhncbTkFFn9jrwIwGoRAQYhjc88wmwFNH5u_rO56USo_w"
	}
}

coze Standard Fields

  • coze "Coze" Coze object. E.g. {"coze":{"pay":..., sig:...}}
  • can "Canon" Canon of pay. E.g. ["alg","iat","tmb","typ"]
  • cad "Canon digest" Digest of pay. E.g. "LSgWE4v..."
  • czd "Coze digest" Digest of ["cad","sig"]. E.g. d0ygwQ...
  • pay "Payload" Signed payload. E.g. "pay":{"alg":...}
  • sig "Signature" Signature over cad. E.g. "sig":"ywctP6..."

sig is the signature over the bytes of cad. cad is not rehashed before signing. czd's hashing algorithm must align with alg in pay. czd refers to a particular signed message just as cad refers to a particular payload. cad and czd are calculated from brace to brace, including the braces. cad and czd are recalculatable and are recommended to be omitted from cozies, although they may be useful for reference.

As an added technical constraint, because sig and czd are used as identifiers, sig must be non-malleable. Malleable schemes like ECDSA must perform signature canonicalization that constrains signatures to a non-malleable form.

Verbose coze

Including unnecessary labels is not recommended. For example, the JSON object {"pay":{...},"sig":...} doesn't need the label coze if implicitly known by applications. The following should generally be omitted: key may be looked up by applications by using tmb, the fields can, cad, and czd are recalculatable, and the label coze may be inferred.

A tautologic coze:

{
	"coze": {
		"pay": {
			"msg": "Coze Rocks",
			"alg": "ES256",
			"iat": 1623132000,
			"tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
			"typ": "cyphr.me/msg"
		},
		"key": {
			"alg":"ES256",
			"iat":1623132000,
			"kid":"Zami's Majuscule Key.",
			"tmb":"cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
			"x":"2nTOaFVm2QLxmUO_SjgyscVHBtvHEfo2rq65MvgNRjORojq39Haq9rXNxvXxwba_Xj0F5vZibJR3isBdOWbo5g"
		},
		"can": ["msg","alg","iat","tmb","typ"],
		"cad": "Ie3xL77AsiCcb4r0pbnZJqMcfSBqg5Lk0npNJyJ9BC4",
		"czd": "TnRe4DRuGJlw280u3pGhMDOIYM7ii7J8_PhNuSScsIU",
		"sig": "Jl8Kt4nznAf0LGgO5yn_9HkGdY3ulvjg-NyRGzlmJzhncbTkFFn9jrwIwGoRAQYhjc88wmwFNH5u_rO56USo_w"
	}
}

Simplified:

{
	"pay": {
		"msg": "Coze Rocks",
		"alg": "ES256",
		"iat": 1623132000,
		"tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
		"typ": "cyphr.me/msg"
	},
	"sig": "Jl8Kt4nznAf0LGgO5yn_9HkGdY3ulvjg-NyRGzlmJzhncbTkFFn9jrwIwGoRAQYhjc88wmwFNH5u_rO56USo_w"
}

Canon

A canon is a list of fields used for normalization, e.g. ["alg","x"]. Coze objects are canonicalized for creating digests, signing, and verification. The canon of pay is the currently present fields in order of appearance. The following Coze fields have predefined canons:

  • cad's canon is pay's canon.
  • tmb's canon is ["alg","x"].
  • czd's canon is ["cad","sig"].

Using a canon, the canonical form of an object is generated by removing fields not appearing in the canon, ordering remaining fields by appearance in the canon, and eliding unnecessary whitespace. The canonical form is serialized into UTF-8 for signing, verification, and hashing.

Canonical form generation steps:

  • Omit fields not present in canon.
  • Order fields by canon.
  • Omit insignificant whitespace.

A canonical digest is generated by hashing the UTF-8 serialized canonical form using the hashing algorithm specified by alg. For example,"ES256"'s hashing algorithm is "SHA-256".

The key thumbprint, tmb, is the canonical digest of key using the canon ["alg","x"] and hashing algorithm specified by key.alg. For example, a key alg of ES256 corresponds to the hashing algorithm SHA-256. The canonical form of the example key is:

{"alg":"ES256","x":"2nTOaFVm2QLxmUO_SjgyscVHBtvHEfo2rq65MvgNRjORojq39Haq9rXNxvXxwba_Xj0F5vZibJR3isBdOWbo5g"}

Hashing this canonical form results in the following digest, which is tmb: cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk.

czd is the canonical digest of coze with the canon ["cad","sig"], which results in the JSON {"cad":"...",sig:"..."}. czd's hash must align with alg in pay.

The canonical digest of

  • pay is cad,
  • ["alg","x"] is tmb,
  • ["cad","sig"] is czd.

Using the first example, the following canonical digests are calculated:

  • tmb is cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk
  • cad is LSgWE4vEfyxJZUTFaRaB2JdEclORdZcm4UVH9D8vVto.
  • czd is d0ygwQCGzuxqgUq1KsuAtJ8IBu0mkgAcKpUJzuX075M.

Signing and verification functions must not mutate pay. Any mutation of pay via can must occur by canon related functions. Note that's since pay's canon is the present fields, no fields are removed when canonicalizing pay.

Coze and Binaries

The canonical digest of a binary file may simply be the digest of the file. The hashing algorithm and any other metadata may be denoted by an accompanying coze. For example, an image ("coze_logo_icon_256.png") may be referred to by its digest.

{
	"alg":"SHA-256",
	"file_name":"coze_logo_icon_256.png",
	"id":"oDBDAg4xplHQby6iQ2lZMS1Jz4Op0bNoD5LK3KxEUZo"
}

For example, a file's digest, denoted by id, may represent the authorization to upload a file to a user's account.

{
 "pay": {
  "alg": "ES256",
  "file_name": "coze_logo_icon_256.png",
  "id": "oDBDAg4xplHQby6iQ2lZMS1Jz4Op0bNoD5LK3KxEUZo",
  "iat": 1623132000,
  "tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
  "typ": "cyphr.me/file/create"
 },
 "sig": "DgJb6Qb81uhC-ulZJlIIj8ahi0b5rAbtnkQhiEH1FB0HeNiACVh_Deo6a22OkK2tr0UcDOiIRY1X-BUriw03Mg"
}

Revoke

A Coze key may be revoked by signing a coze containing the field rvk with an integer value greater than 0. The integer value 1 is suitable to denote revocation and the current Unix timestamp is the suggested value.

Example Self Revoke

{
 "pay": {
  "alg": "ES256",
  "iat": 1623132000,
  "msg": "Posted my private key online",
  "rvk": 1623132000,
  "tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
  "typ": "cyphr.me/key/revoke"
 },
 "sig": "KVjPjMVHoL828WyAH5biqIOt-IOaQ5EBtN_7eQifP2w3agUHu6KfqO40_oqQ5GE_BShgXvhbK0O6Z2h5YPNAcw"
}
  • rvk - Unix timestamp of key expiry.

Coze explicitly defines a self-revoke method so that third parties may revoke leaked keys. Systems storing Coze keys should provide an interface permitting a given Coze key to be marked as expired by receiving a self-revoke message. Self-revokes with future times must immediately be considered as expired.

rvk and iat must be a positive integer less than 2^53 – 1 (9,007,199,254,740,991), which is the integer precision limit specified by IEEE754 minus one. Revoke checks must error if rvk is not an integer or larger than 2^53 - 1.

Key expiration policies, key rotation, backdating, and alternative revocation methods are outside the scope of Coze.

Alg

alg specifies a parameter set and is a single source of truth for Coze cryptographic operations.

Example - "alg":"ES256"

  • Name: ES256
  • Genus: ECDSA
  • Family: EC
  • Use: sig
  • Hash: SHA-256
  • HashSize: 32
  • HashSizeB64: 43
  • XSize: 64
  • XSizeB64: 86
  • DSize: 32
  • DSizeB64: 43
  • Curve: P-256
  • SigSize: 64
  • SigSizeB64: 86

Supported Algorithms

  • ES224
  • ES256
  • ES384
  • ES512
  • Ed25519
  • Ed25519ph (planned)

Coze Verifier

The Coze verifier is an in-browser tool for signing and verifying.

Coze Verifier

coze_verifier

There is also the Simple Coze Verifier that has the minimal amount of code needed for a basic Coze application. Its codebase is in the Cozejs repo and may be locally hosted.

Coze Implementations

See docs/development.md for the Go development guide.

Coze Core and Coze X

The sections above are defined as the main Coze specification, Coze core. There are no plans to increase Coze's scope or features in core other than additional algorithm support. This will be especially true after Coze is out of Alpha/Beta. (At the moment, we would like more time for feedback before casting the specification into stone.)

Coze x (Coze extended) includes additional documentation, extra features, drafts, proposals, early new algorithms support that's not yet adopted in Coze core, and extended algorithm support.

See Coze_go_x/normal for an example of a Coze x feature not included in Coze core.

Repository structure:

  • Coze Main specification (core) and the Go Coze reference implementation.
  • CozeX Coze extended. Additional documents, discussion, and new algorithms (Not a code repository).
  • CozeGoX Go implementation of extended features.
  • CozeJS Javascript implementation of Coze core.
  • CozeJSX Javascript implementation of extended.
  • etc...

FAQ

Pronunciation? What does "Coze" mean?

We say "Co-zee" like a comfy cozy couch. Jared suggested Coze because it's funny. The English word Coze is pronounced "kohz" and means "a friendly talk; a chat" which is the perfect name for a messaging standard.

"Coze" vs "coze"?

We use upper case "Coze" to refer to the specification, and "coze"/"cozies" to refer to messages.

What is Coze useful for?

Coze's applications are endless as Coze is useful for anything needing cryptographic signing. Coze is deployed in various applications such as user authentication (user login), authorization, product tracking, user comments, user votes, chain of custody, Internet of things (IoT), sessions, and cookies.

As a timely example the CEO of Reddit (reddit.com/u/spez) edited people's comments. Messages signed by Coze prevents tampering by third parties.

Binary? Why not support binary payloads?

JSON isn't well designed for large binary payloads. Instead, Coze suggests including the digest of a binary file in a coze message while transporting the binary separately. There's nothing stopping an application from base 64 encoding a binary for transport, although it's not recommended.

Why is Coze's scope so limited?

Coze is intentionally scope limited. It is easier to extend a limited standard than to fix a large standard. Coze can be extended and customized for individual applications.

Is Coze versioned?

alg refers to a specific set of parameters for all operations and Coze Core "versioning" is accomplished by noting specific algorithm support. If an operation needs a different parameter set, alg itself must denote the difference. alg permits Coze implementations to support a subset of features while remaining Coze compliant. The specification hopes to stay simple and stable enough to preclude versioning, however we suspect further tweaks are probably warranted, so a long alpha and beta time is planned. Extension to Coze are defined by CozeX so implementations avoid feature bloat. Implementation releases themselves are versioned.

Why does pay have cryptographic components?

Coze's pay includes all payload information, a design we've dubbed a "fat payload". We consider single pass hashing critical for Coze's simple design.

Alternative schemes require a larger canon, {"head":{...},"pay":{...}}, or concatenation like digest(head) || digest(pay). By hashing only pay, the "head" label and encapsulating braces are dropped, pay:{...}, and the label "pay" may then be inferred, {...}. {...} is better than {"head":{...},"pay":{...}}.

Verifying a coze already requires hashing pay. Parsing alg from pay is a small additional cost.

JSON APIs? Can my API do versioning?

Coze is well suited for JSON APIs. API versioning may be handled by applications however desired. A suggested way of incorporating API versioning in Coze is to use typ, e.g. "typ":"cyphr.me/v1/msg/create", where "v1" is the api version.

Can my application use Canon/Canonicalization?

Yes, canon is suitable for general purpose application. Applications may specify canon expectations in API documentation, if using Coze denoted by "typ" or explicitly specified by can, or implicitly known and pre-established. Coze Core contains simple canonicalization functions, or for more expressive capabilities see Normal.

pay.typ vs key.typ.

For applications, pay.typ may denote a canon. For example, a typ with value cyphr.me/msg/create has a canon, as defined by the service, of ["alg", "iat", "msg", "tmb", "typ"]. The service may reject a coze that's not canonicalized as expected. For example, the service might reject cozies missing iat.

Key.tmb ignores key.typ because a static canon, ["alg","x"] is always used when producing key's tmb. Like typ in pay, applications may use key.typ to specify custom fields, e.g. "first_seen" or "account_id" and field order.

ECDSA x and sig Bytes.

For ECDSA , (X and Y) and (R and S) are concatenated for x and sig respectively. For ES512, which unlike the other ECDSA algorithms uses the odd numbered P-521, X, Y, R, and S are padded before concatenation.

Why use tmb and not x for references in messages?

Coze places no limit on public key size, which can be very large. For example, GeMSS128 public keys are 352,188 bytes, compared to Ed25519's 32 bytes. Using tmb instead of x generalizes Coze for present and future algorithm use. Additionally, x may be cryptographically significant for key security while tmb is not.

Required Coze Fields, Contextual Cozies, and the Empty Coze.

The standard fields provide Coze and applications fields with known types since JSON has limited type identifiers. Coze has no required fields, however omitting standard fields limits interoperability among applications, so it is suggested to include standard fields appropriately.

Cozies that are missing the fields pay.alg and/or pay.tmb are contextual cozies, denoting that additional information is needed for verification. Caution is urged when deploying contextual cozies as including the standard fields pay.alg and pay.tmb is preferred.

An empty coze, which has an empty pay and populated sig, is legitimate. It may be verified if key is known. The following empty coze was signed with the example key "cLj8vs".

{
	"pay":{},
	"sig":"9iesKUSV7L1-xz5yd3A94vCkKLmdOAnrcPXTU3_qeKSuk4RMG7Qz0KyubpATy0XA_fXrcdaxJTvXg6saaQQcVQ"
}

UTF-8 and b64ut (RFC base 64 URI canonical truncated) Encoding

Canonical base 64 (sometimes called "strict") encoding is required and non-strict encoding of both b64ut and UTF-8 must error. For the initial reason for why Coze uses b64ut see base64.md.

Why not PGP/OpenSSL/LibreSSL/SSHSIG/libsodium/JOSE(JWT)/COSE/etc...? How does Coze compare with prior arts?

We respect the various projects in the space. Other projects have noble goals and we're thankful they exist. Coze is influenced by ideas from many others. However existing solutions were not meeting our particular needs so we created Coze.

See coze_vs.md and the introduction presentation for more.

Does Coze have checksums?

x, tmb,cad, czd, and sig may be used for integrity checking.

Systems may use sig as an integrity check via cryptographic verification. If cad and/or czd are included they may be recalculated and error on mismatch.

For keys, x and/or tmb may be recalculated and error on mismatch.Coze keys cannot be integrity checked when d, x, or tmb are presented alone. In situations needing integrity checking, we recommend including at least two components. See checksums.md for more.

Performance hacks?

Coze is not optimized for long messages, but if early knowledge of Coze standard fields is critical for application performance, put the Coze standard fields first, e.g. {"alg", "tmb", ...}

I need to keep my JSON separate but inside a coze.

If appending custom fields after the standard Coze fields isn't sufficient, we suggest encapsulating custom JSON in "~", the last ASCII character. We've dubbed this a "tilde encapsulated payload". For example:

{
	"alg": "ES256",
	"iat": 1623132000,
	"tmb": "cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk",
	"typ": "cyphr.me/msg/create",
	"~": {
		"msg": "tilde encapsulated payload"
	}
}

ASCII/Unicode/UTF-8/UTF-16 and Ordering?

Even though Javascript uses UTF-16 and JSON was designed in a Javascript context, JSON implementations rejected the problematic UTF-16, which has some code points out of order, in favor of UTF-8. Requiring JSON UTF-8 encoding was formalized by the JSON RFC 8259 section 8.1. Unicode, ASCII, and UTF-8 all share sorting order.

Although JSON arrays are defined as ordered, JSON objects are defined as unordered. How is pay, an unordered JSON object, signed when signing requires a static representation? UTF-8 is the explicitly defined serialization for JSON. Coze's signing and verification operations are not over abstract JSON, but rather the concrete UTF-8. Coze marshals JSON into UTF-8 before signing, and Coze verifies UTF-8 before unmarshalling into JSON.

Additionally, object field order may be denoted by can, chaining normals, or communicate via other means.

Where does the cryptography come from?

Much of this comes from NIST FIPS.

For example, FIPS PUB 186-3 defines P-224, P-256, P-384, and P-521.

To learn more see this walkthrough of ECDSA.

Unsupported Things?

The following are out of scope or redundant.

  • ES192, P-192 - Not implemented anywhere and dropped from later FIPS.
  • SHA1, MD5 - Not considered secure for a long time.
  • kty - "Key type". Redundant by alg.
  • iss - tmb fulfills this role. Systems that need something like an issuer, associating messages with people/systems, can look up "issuer" based on thumbprint. Associating thumbprints to issuers is the design we recommend.
  • exp - "Expiration". Outside the scope of Coze.
  • nbf - "Not before". Outside the scope of Coze.
  • aud - "Audience". Outside the scope of Coze, but consider denoting this with typ.
  • sub - "Subject". Outside the scope of Coze, but consider denoting this with typ.
  • jti - "Token ID/JWT ID". Redundant by czd, cad, or an application specified field.

Encryption?

Coze does not currently support encryption. If or when it ever does it would be similar to or simply complement age.

Why define algorithms?

It’s not enough to implement a single standard; it’s vital that our systems be able to easily swap in new algorithms when required. We’ve learned the hard way how algorithms can get so entrenched in systems that it can take many years to update them: in the transition from DES to AES, and the transition from MD4 and MD5 to SHA, SHA-1, and then SHA-3.

-- Bruce Schneier

Coze's design is generalized and not overly coupled to any single primitive. Because of this, applications that use Coze can easy upgrade cryptographic primitives. Using a single primitive is perfectly fine, but tightly coupling systems to a single primitive is not. Simultaneous support for multiple primitives is a secondary, and optional, perk.

JSON "Name", "Key", "Field Name", "Member Name"?

They're all synonyms. A JSON name is a JSON key is a JSON field name is a JSON member name. In this document we use "field name" to avoid confusion with Coze key.

Why are duplicate field names prohibited?

Coze explicitly requires that implementations disallow duplicate field names in coze, pay, and key. Existing JSON implementations have varying behavior. Douglas Crockford, JSON's inventor, tried to fix this but it was decided it was too late.

Although Douglas Crockford couldn't change the spec forcing all implementations to error on duplicate, his Java JSON implementation errors on duplicate names. Others use last-value-wins, support duplicate keys, or other non-standard behavior. The JSON RFC states that implementations should not allow duplicate keys, notes the varying behavior of existing implementations, and states that when names are not unique, "the behavior of software that receives such an object is unpredictable." Also note that Javascript objects (ES6) and Go structs already require unique names.

Duplicate fields are a security issue, a source of bugs, and a surprising behavior to users. See the article, "An Exploration of JSON Interoperability Vulnerabilities"

Disallowing duplicates conforms to the small I-JSON RFC. The author of I-JSON, Tim Bray, is also the author of current JSON specification (RFC 8259). See also json5/json5-spec#38.

Why is human readability a goal?

Although humans cannot verify a signature without the assistance of tools, readability allows humans to visually verify what a message does.

We saw the need for JSON-centric cryptography and idiomatic JSON is human readable. JSON is not a binary format; it is a human readable format and any framwork built on JSON should embrace its human readability. If human readability is unneeded, JSON is entirely the wrong message format to employ. All else being equal, human readability is better than non-human readability.

JSON?

See also I-JSON and JSON5

HTTP? HTTP Cookies? HTTP Headers?

When using Coze with HTTP cookies, Coze messages should be JSON minified. For example, we've encountered no issues using the first example as a cookie:

token={"pay":{"msg":"Coze Rocks","alg":"ES256","iat":1623132000,"tmb":"cLj8vsYtMBwYkzoFVZHBZo6SNL8wSdCIjCKAwXNuhOk","typ":"cyphr.me/msg"},"sig":"Jl8Kt4nznAf0LGgO5yn_9HkGdY3ulvjg-NyRGzlmJzhncbTkFFn9jrwIwGoRAQYhjc88wmwFNH5u_rO56USo_w"}; Path=/;  Secure; Max-Age=999999999; SameSite=None

For more considerations see http_headers.md

Why release pre-alpha on 2021/06/08?

Coze was released on 2021/06/08 (1623132000) since it's 30 years and one day after the initial release of PGP 1.0. We wrote a blog with more details of Coze's genesis.

Signature Malleability?

Coze prohibits signature malleability. See malleability_low_s.md.

Who created Coze?

Coze was created by Cyphr.me.

Discussion? Social Media?

Other Resources

Keywords

Coze JSON alg iat tmb typ rvk kid d x coze pay key can cad czd sig cryptography crypto authentication auth login hash digest signature Cypherpunk Cyphrme Ed25519 Ed25519ph ES224 ES256 ES384 ES512 SHA-224 SHA-256 SHA-384 SHA512 JOSE JWS JWE JWK JWT PASETO PASERK signify ssh SSHSIG PGP Bitcoin Ethereum base64 b64ut SQRL


Attribution, Trademark Notice, and License

Coze is released under The 3-Clause BSD License.

"Cyphr.me" is a trademark of Cypherpunk, LLC. The Cyphr.me logo is all rights reserved Cypherpunk, LLC and may not be used without permission.

coze's People

Contributors

horvski avatar zamicol avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

coze's Issues

Active

Is this project still active?

Use JSONv2 when production ready

Instead of making various issues for various JSON concerns. I'm going to use this issue to track any concerns with JSON.

Coze needs strictly defined JSON capabilities. Some of these capabilities are not provided by the standard
Go library, or misbehaves under certain circumstances. Currently, there are no known 3rd party libraries that we consider suitable for Coze. Our hope is that we can use JSONv2 when production ready and that will solve any JSON concerns.

A new JSON library should minimally include these characteristics in addition to
the existing behavior of the standard library:

  • Duplicate fields should error.
  • Preserve order of JSON fields.
  • Invalid UTF-8 should error.
  • JSON should not be HTML escaped. (For example when containing the characters &, <,>. and while not adding an additional new line as the standard library currently does.)

Three of the larger obstacles:

To resolve these obstacles, Coze implemented orderedmap, its own custom JSON unmarshaler, and a "checkDuplicate" helper function.

A search of other Go JSON issues that may be relevant to Coze:
https://github.com/golang/go/issues?q=is%3Aissue+is%3Aopen+encoding%2Fjson+in%3Atitle

JSONv2

See https://pkg.go.dev/github.com/go-json-experiment/json
There are other best practices in JSONv2.

cmd

A cli client would be fantastic.

  • sign
  • signpay
  • verify
  • newkey
  • tmb
  • meta
  • revoke

Make new repositories for the specification and implementations

Edit:

Coze repository organization

- Coze          ("Core"/main specification and the Go Coze reference implementation)
- Coze_x        (Coze extended)
- Coze_go_x     (Go implementation of extended features)
- Coze_js       (Javascript implementation)
- Coze_js_x     (Javascript implementation of extended)
- etc...
  • Discussion on the main spec or the Go reference implementation should go into
    Coze, aka "core".
  • Discussion on "x" design goes into Coze_x. Future discussion on
    implementing/supporting new algorithms also goes into x, as implementations of
    new algorithms will live in x first before being adopted by core. Only
    established and widely adopted algorithms are eventually included into core.
  • Every language that implements Coze should be in it's own respective language
    specific directory (except the Go reference implementation).
  • It is suggested that implementation of "x" features should go into the
    appropriate language "x" repository.

Old

This repository should be split into a few different repositories. It is usually good practice to split out documents from code so that developers aren't burdened with reviewing changes to the repository.

I suggest the following repositories:

  • coze
  • coze_go
  • coze_go_experimental
  • coze_js
  • coze_js_experimental
    etc...

This unfortunately would probably require renaming "Coze" to "coze". "coze" would be for the main spec (README.md) which defines Coze Core, and also documents, discussion, proposals, best practices, and FAQ. Github appears to be a good place for discussion and document modification, but we don't want a large number of developers worrying about the changes happening in the repository due to simple document modification.

Experimental is for new algorithms and useful Coze related libraries not in the core spec (normal would be one such library). New algorithms are allowed time to mature first in experimental before getting accepted into "Core".

The plan for Coze Core

The only planned expansion of "Coze Core", the implementation and specification that currently lives in this repo, is adding new algorithms. No changes are currently planned to the spec other than per algorithm adjustments. However, we wanted to give this more time and receive more feedback before we made a more final decision.

There will be minor tweaks on a per algorithm basis for Coze Core. For example, @LoupVaillant suggested doing the following when handling Ed25519:

My choice for Monocypher was to do the same as Zebra:

  • Reject any S that equals or exceeds the order of the curve.
  • Accept low-order A and R.
  • Accept non-canonical A and R.
  • Use the batch verification equation (it's the forgiving one).

This needs to be specified in Coze Core so that all implementations align.

Alternative Repository Structure

We could do a less dramatic division. However, this is less normalized so I'd advocate for the aforementioned naming.

  • Coze - The Go (and reference) implementation of Coze. (This repo.)
  • Cozejs (Already exists)
  • coze_spec - "The main spec, documents, discussion, proposals, best practices, and FAQ.
  • coze_experimental - For useful libraries "Standard" and

JSON round-trip fails for MapSlice

It's possible to successfully unmarshal valid JSON into a MapSlice, but then fail to marshal that same MapSlice back into JSON.

I'm assuming this isn't desired behavior, but I could be wrong.

POC: https://go.dev/play/p/MlNV74p2cCY

package main

import (
	"encoding/json"
	"log"

	"github.com/cyphrme/coze"
)

func main() {
	b := []byte("{\"\x7f\":{}}") // discovered via fuzzing
	if !json.Valid(b) {
		log.Fatal("invalid JSON")
	}

	var ms coze.MapSlice
	if err := json.Unmarshal(b, &ms); err != nil {
		log.Fatalf("json.Unmarshal: %v", err)
	}

	if _, err := json.Marshal(ms); err != nil {
		log.Fatalf("json.Marshal: %v", err) // fails here
	}
}

Base64 encoding can only elide padding when the size of encoded data is known

https://github.com/Cyphrme/Coze/blob/01c154e4024b4e876b8d152166ce85cf2a945e22/README.md#coze-fields

Binary values are encoded as RFC 4648 base64 URI with padding truncated (b64ut).

https://www.rfc-editor.org/rfc/rfc4648#section-3.2

when assumptions about the size of transported data cannot be made, padding is required to yield correct decoded data.

As far as I can tell, the size of binary values is not communicated to recipients, and therefore padding should not be truncated. (The URI encoding is also non-standard.)

Further constraints on Ed25519

@LoupVaillant suggested doing the following when handling Ed25519:

In my opinion standardizing signatures and public keys is much more
important than worrying about anything related to the private key. And
just at that level you have to grapple with much more fundamental issues
than how to define your private key:

https://hdevalence.ca/blog/2020-10-04-its-25519am

So you have a public key A, and a signature R || S.
A and R are points on the curve, and S is just a number.
Thankfully, the main issues were dealt with from the beginning:

  • Points on the curve are compressed as a field element and a sign bit.
  • All numbers are encoded in little-endian.
  • A, R, and S are all serialised with 32 bytes.

But there's still room for variation in the verifier:

  • Do we accept S when it exceeds the order of the curve?
  • Do we accept A and R when they have low order?
  • Do we accept non-canonical encodings of A and R?
  • What verification equation do we use exactly?

When two verifiers disagree on any of the above, this can cause problems
when maliciously crafted signatures end up being accepted by some and
rejected by others, leading to problems like network partitions. Worse,
the RFC didn't clearly answer all of those questions, and allowed users
to chose which verification equation they would use. And it's difficult
in practice to find two implementations that behave identically. It's a
freaking nightmare.

My choice for Monocypher was to do the same as Zebra:

  • Reject any S that equals or exceeds the order of the curve.
  • Accept low-order A and R.
  • Accept non-canonical A and R.
  • Use the batch verification equation (it's the forgiving one).

The reason I reject high S is because (i) everyone else does, and (ii)
accepting it would enable malleability attacks. For everything else I
chose to be as permissive as possible. This has the advantage of being
backwards compatible with any other implementation: no signature that
was previously accepted will be rejected.

The RFC on the other hand made the following choices:

  • Reject any S that equals or exceeds the order of the curve.
  • Accept low-order A and R.
  • Reject non-canonical A and R.
  • Leave equation choice to the implementer.

I personally disagree with the last two items. Interoperability with
batch verification (which is twice as fast as regular verification)
should be mandatory, and rejecting non-canonical points makes the code
more complex for no benefit at all.

You'll have to make your own choice too if you want a complete
specification. I personally would recommend you imitate Zebra and
Monocypher, because many implementations can be made compatible with a
bit of pre-processing:

  1. Reject the signature if S is too big. Almost all implementations
    already do this however, so you can generally skip this step.
  2. If both A and R have low order, and S == 0, accept the signature.
    In total, low order points have 14 different encodings, so you can
    just use a table and compare buffers to do that check.
  3. Run your implementation of choice. It must use the batch equation.
    If it accepts the signature, accept it.
    If it rejects the signature, reject it.

Ran into this great presentation on normalizing/standardizing Ed25519, "Taming the many EdDSAs"
https://csrc.nist.gov/csrc/media/Presentations/2023/crclub-2023-03-08/images-media/20230308-crypto-club-slides--taming-the-many-EdDSAs.pdf

Also consider the advice in:

Duplicate JSON keys create misleading verification results in the web UI

During a conversation, I was sent this example message: https://cyphr.me/coze#?input={%22pay%22:{%22msg%22:%22Hello%20Retr0id!%22,%22alg%22:%22ES256%22,%22iat%22:1701603747,%22tmb%22:%221KGZzsqiFAYE5uDO3CCh3PMl9pqwlG1RrI8i5gZY94c%22,%22typ%22:%22cyphr.me/msg/create%22},%22sig%22:%22vZEALUQShRlLYyhJoGmkxpQhhEeUPlzbIZqyTsUyPBMlJMIqGClgSs3uXTDiwumsMivFQKU8K4z4Ec7WlxYIew%22}&dontSignRevoke&updateIat&selectedAlg=ES256&verify

By introducing a duplicate "msg" key, I was able to forge a new message that also passes signature verification according to the web UI: https://cyphr.me/coze#?input={%22pay%22:{%22msg%22:%22Hello,%20Zamicol.%20I%20believe%20you%20will%20find%20that%20this%20Coze%20message%20is%20also%20(supposedly)%20signed%20by%20your%20key!%20Coze%20could%20fix%20this%20issue%20with%20better%20UI,%20but%20I%20think%20this%20illustrates%20just%20how%20hard%20it%20is%20to%20canonicalize%20JSON.%20This%20wouldn't%20be%20possible%20in%20the%20first%20place%20if%20you%20were%20signing%20base64'd%20bytes.%22,%22msg%22:%22Hello%20Retr0id!%22,%22alg%22:%22ES256%22,%22iat%22:1701603747,%22tmb%22:%221KGZzsqiFAYE5uDO3CCh3PMl9pqwlG1RrI8i5gZY94c%22,%22typ%22:%22cyphr.me/msg/create%22},%22sig%22:%22vZEALUQShRlLYyhJoGmkxpQhhEeUPlzbIZqyTsUyPBMlJMIqGClgSs3uXTDiwumsMivFQKU8K4z4Ec7WlxYIew%22}&dontSignRevoke&updateIat&selectedAlg=ES256&verify

The web UI gives no indication that there's anything awry here, and proclaims that the message was verified.

image

This would be a non-issue in many use-cases, because any code reading the msg parameter will likely see only the real one - but an implementation in some other language may see only the first, causing breakage. But, for the purposes of the web UI, it's definitely misleading.

As an aside, I think Coze's general design/approach is fine, but dealing with JSON like this is error-prone, and here is one such error - one which is hopefully an easy fix.

Expunge "cryptographic agility" from Coze vocabulary

@LoupVaillant suggested to avoid semantical confusion by avoiding entirely the term "cryptographic agility". In its place we could say "provide loose primitive coupling."

The design goals would then become:

  • Valid and idiomatic JSON.
  • Human readable and writable.
  • Small in scope.
  • Provide loose primitive coupling.

Edit: Thinking more on this, perhaps the fourth design goal is dropped altogether since Coze implementations inherently provide loose primitive coupling, and that phrase itself would need to be rigidly defined. It also doesn't capture what I was trying to convey, that Coze provides "versioning" via "alg". Perhaps even something along the lines of "provide defined cipher suites". Edit2: or "Specify cipher suite expectations"

MapItem is unsafe, and MapSlice does not have a well-defined order

indexCounter is accessed without synchronization, which produces data races that violate the memory model. As a proof of concept, build and run the following program with the -race flag.

package main

import (
	"encoding/json"
	"sync"

	"github.com/cyphrme/coze"
)

func main() {
	n := 10
	var wg sync.WaitGroup
	for i := 0; i < n; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			var item coze.MapItem
			json.Unmarshal([]byte(`{}`), &item)
		}()
	}
	wg.Wait()
}

You'll see output like the following.

==================
WARNING: DATA RACE
Read at 0x00010041e838 by goroutine 13:
  github.com/cyphrme/coze.nextIndex()
      .../pkg/mod/github.com/cyphrme/[email protected]/mapslice.go:36 +0xf0
  github.com/cyphrme/coze.(*MapItem).UnmarshalJSON()
      .../pkg/mod/github.com/cyphrme/[email protected]/mapslice.go:118 +0x11c
  encoding/json.(*decodeState).object()
      ...

Previous write at 0x00010041e838 by goroutine 12:
  github.com/cyphrme/coze.nextIndex()
      .../pkg/mod/github.com/cyphrme/[email protected]/mapslice.go:36 +0x108
  github.com/cyphrme/coze.(*MapItem).UnmarshalJSON()
      .../pkg/mod/github.com/cyphrme/[email protected]/mapslice.go:118 +0x11c
  encoding/json.(*decodeState).object()
      ...

Enforce Canonical Base 64 encoding.

Playground demonstrating the issue:

There's an apparent problem with RFC 4648. There are three places base 64 representation may contain string variation:

  1. Padding
  2. Alphabet (URI unsafe or URI safe)
  3. Canonical encoding (various characters can encode to the same byte string, but there is only one canonical decoding)

What is "canonical encoding"? From the last three characters of the example tmb, "cLj8vs...XNuhOk", the values hOk and hOl may both decode to the same byte value (in Hex, 84E9) even though they are different UTF-8 values. (Example decoding hOk and hOl.) The canonical encoding is hOk

The RFC specifically addresses 1 and 2, but not really 3.

RFC 4648 advises to reject non-alphabet characters, which can include padding. I agree with this advice:

Implementations MUST reject the encoded data if it contains
characters outside the base alphabet when interpreting base-encoded
data, unless the specification referring to this document explicitly
states otherwise. [...] Furthermore, such specifications MAY ignore the pad
character, "=", treating it as non-alphabet data[.]

I don't see the RFC really address the to the third concern.

Behavior

Obviously non-"strict"/non-canonical base 64 encoding is incorrect, and any encoder producing non-strict encoding should be fixed. However the question is what should Coze specify regarding non-strict encoding/decoding? Both Go and Javascript are permissive when decoding and do not throw errors.

Ultimately, the concern is different base 64 encoders/decoders may have different behavior. Ideally, Coze should specify the appropriate behavior for Coze. Section 3.5 mentions non-canonical encoding in the context of unpadded data and this issues is unrelated to padding (hOk= and hOl=, both padded, have the same issue as unpadded strings).

The concern is that if a Coze implementation used string comparison instead of byte comparison, this could result implementations disagreeing about valid messages. For example, with a non-strict tmb encoded string, if a Coze implementation checks tmb before cryptographic verification, it may check this based on the string value or the byte value, and comparing the string value or the byte value will result in different behavior.

Another note for any Coze restriction on encoding: JSON is base 64 unaware, any sort of Coze specified enforcement of base 64 encoding can only be applied to Coze known fields with type b64ut, and cannot be applied generally to any b64ut value.

Solutions

There appears to be only two options to handle this:

  1. Be permissive on inbound encoding, force strict outbound encoding.
  2. Force strict encoding and decoding. (This can only be done when type is known to be b64ut.)

2 is more conservative, but may require unnecessary checks that don't really add value. 1 has the potential to be more compatible if assuming that systems can decode permissively (other programming language's base 64 libraries decode permissively), which may be a bad assumption.

Regardless, I believe that 1 is the correct behavior here. Even if languages/system do no error on non-canonical encoding, implementing an encoding error can be implemented by re-encoding the decoded data and comparing strings.

Security Considerations

This base 64 decoding bug doesn't appear to be a structural/architectural/security concern since Coze uses the UTF-8 encoding of the string for signing and verification, however it is a interesting problem that should be known when working with RFC base 64. Concerning specifically replay attacks, signatures are still not malleable as payloads are UTF-8 encoded and the signing operation is not base 64 aware.

If Coze used the base 64 representation directly, this would be a security concern and could result in reply attacks.

Notes

It should be obvious, but this situation also applies to the URI unsafe alphabet and messages with base 64 padding, which all are interpreted as the same bytes. (My conversion tool only has "base64 as an input and not the various permutations since all variations can be known (or is irrelevant) and results in the sames decoded binary payload.

RFC 4648

I currently have errata open on one of the relevant sections.

I'm going to implement a non-canonical encoding check on Go and JS Coze.

See also the Go base64 package.

Go's base64 ignores carriage return and new line, so it is malleable, but JSON unmarshal does not, making Go Coze non-malleable. https://go.dev/play/p/X0J74F0zWVf See also the new line test in base64_test.go

Consider documenting that `rvk` denotes expiry for Coze keys.

The word revoke may connotate that a key was never trusted/authorized.

The specification should better explain that rvk is expected to be an expiry time.

Cozies signed before the rvk time should be considered valid and action signed after should be ignored.

As is currently noted by the specification, actionable events based on future expiration times are outside the scope of Coze, and revoke messages with future times should result in the signing key considered immediately expired.

Thanks @qbit for bringing up this concern.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.