google / der-ascii

A reversible DER and BER pretty-printer.

DER ASCII

DER ASCII is a small human-editable language to emit DER (Distinguished Encoding Rules) or BER (Basic Encoding Rules) encodings of ASN.1 structures and malformed variants of them.

It provides two tools, ascii2der and der2ascii, to convert DER ASCII to a byte string and vice versa. To install them, run:

go install github.com/google/der-ascii/cmd/...@latest

These tools may be used to create test inputs by taking an existing DER or BER structure, disassembling it with der2ascii into DER ASCII, making adjustments, and assembling back into binary with ascii2der. This avoids having to manually fix up all the length prefixes. As a bonus, it acts as a human-readable view for DER structures.

For the language specification and basic examples, see language.txt. The samples directory includes more complex examples from real inputs.

Backwards compatibility

The DER ASCII language itself may be extended over time, but the intention is for extensions to be backwards-compatible. Specifically:

• The command-line interface to ascii2der and der2ascii will remain compatible, though new options may be added in the future.

• Previously valid inputs to ascii2der will remain valid and produce the same output. In particular, checking in test data as ascii2der inputs should be future-proof, though it is recommended to check in the generated version as well in case of mistakes.

• Previously invalid inputs to ascii2der may become valid in the future if the language is extended.

• der2ascii is necessarily a heuristic, so its output may change over time. For example, later revisions may recognize new OIDs, tweak the formatting, or disassemble a malformed DER input in a (hopefully) more useful form.

Disclaimer

This is not an official Google project.

der-ascii's Issues

Versioning?

Hi, I'd like to package der2ascii and ascii2der for my package manager of choice. However, this package manager requires some kind of versioning scheme. Would it be possible to introduce some form of versioning, such as Git tags for releases?

Thank you for writing such a useful tool!

Add a flag to der2ascii to have it pem decode before treating the input as DER

Sometimes you have a pile of PEM, it'd be great if der-ascii could simply handle it natively.

OCTWRAP, SEQWRAP, SETWRAP, BITWRAP

EDIT: the cause of this is ambiguity over an old feature in OpenSSL's version of ascii2der, the BITWRAP type, which, much like a void* in C, breaks out of the ASN.1 type system for times that you need that.

---

I made myself a secp256r1 key with openssl and converted it to a public key (btw OpenSSL's prime256v1 == NIST's P-256 == secp256r1):

$ openssl ecparam -name prime256v1 -genkey -noout -out icantbelieveitsnotyubikey.pem
$ openssl ec -in icantbelieveitsnotyubikey.pem -pubout -outform DER -out icantbelieveitsnotyubikey.pub.der

And I got these files (mind, they aren't actually .txts, github just sensibly has a file extension filter on):

icantbelieveitsnotyubikey.pem

icantbelieveitsnotyubikey.pub.der

OpenSSL parses each like this:

$ openssl ec -in icantbelieveitsnotyubikey.pem -text
read EC key
Private-Key: (256 bit)
priv:
    00:82:b3:19:91:10:37:a3:ea:21:76:99:66:c4:28:
    de:ba:ec:be:e7:9a:c4:9f:87:29:d0:c9:1c:c7:5d:
    e1:e6:4e
pub: 
    04:3f:d3:8b:c5:df:75:d6:ca:e0:55:cf:81:60:b2:
    32:77:d2:95:f5:9a:8e:34:a2:b0:dc:69:b8:1a:f9:
    0a:1c:04:2c:02:c9:55:80:e6:9e:88:37:e7:52:ec:
    a1:39:5e:30:a9:20:22:83:ec:1a:85:f8:da:a7:8a:
    de:83:3c:75:2b
ASN1 OID: prime256v1
NIST CURVE: P-256
writing EC key
-----BEGIN EC PRIVATE KEY-----
MHcCAQEEIIKzGZEQN6PqIXaZZsQo3rrsvueaxJ+HKdDJHMdd4eZOoAoGCCqGSM49
AwEHoUQDQgAEP9OLxd911srgVc+BYLIyd9KV9ZqONKKw3Gm4GvkKHAQsAslVgOae
iDfnUuyhOV4wqSAig+wahfjap4regzx1Kw==
-----END EC PRIVATE KEY-----

$ openssl asn1parse -inform DER -in icantbelieveitsnotyubikey.pub.der -dump
    0:d=0  hl=2 l=  89 cons: SEQUENCE          
    2:d=1  hl=2 l=  19 cons: SEQUENCE          
    4:d=2  hl=2 l=   7 prim: OBJECT            :id-ecPublicKey
   13:d=2  hl=2 l=   8 prim: OBJECT            :prime256v1
   23:d=1  hl=2 l=  66 prim: BIT STRING        
      0000 - 00 04 3f d3 8b c5 df 75-d6 ca e0 55 cf 81 60 b2   ..?....u...U..`.
      0010 - 32 77 d2 95 f5 9a 8e 34-a2 b0 dc 69 b8 1a f9 0a   2w.....4...i....
      0020 - 1c 04 2c 02 c9 55 80 e6-9e 88 37 e7 52 ec a1 39   ..,..U....7.R..9
      0030 - 5e 30 a9 20 22 83 ec 1a-85 f8 da a7 8a de 83 3c   ^0. "..........<
      0040 - 75 2b                                             u+                   ..

That all makes sense to me: openssl ec -text's "pub:" output matches the BITSTRING object in the DER (ignoring the \x00 header), and that object has 65 bytes starting with a \x04 so it's a secg uncompressed elliptic curve point.

But der2ascii crops the first two bytes off the pubkey:

$ der2ascii -i icantbelieveitsnotyubikey.pub.der.txt 
SEQUENCE {
  SEQUENCE {
    # ecPublicKey
    OBJECT_IDENTIFIER { 1.2.840.10045.2.1 }
    # secp256r1
    OBJECT_IDENTIFIER { 1.2.840.10045.3.1.7 }
  }
  BIT_STRING {
    `00`
    OCTET_STRING { `d38bc5df75d6cae055cf8160b23277d295f59a8e34a2b0dc69b81af90a1c042c02c95580e69e8837e752eca1395e30a9202283ec1a85f8daa78ade833c752b` }
  }
}

Notice that the tail is correct, (de 83 3c 75 2b), and if you check the rest it's all there, and the leading \x00 is too, all except for the first \x04 secg format header, and the first byte of the first coordinate.

---

I have macOS 10.14.4,

$ openssl version  # installed from brew
OpenSSL 1.0.2r  26 Feb 2019

$ go version
go version go1.12.4 darwin/amd64

and the latest der2ascii from go get github.com/google/der-ascii/cmd/....

Handle non-minimally-encoded tags and lengths

The disassembler currently doesn't decode non-minimally-encoded tags and lengths. This is illegal in DER but is legal BER. The disassembler's goal is to "reasonably" process all valid BER. This sort of thing is also (sadly) accepted by many parsers, so it's valuable to be able to pretty-print and sometimes produce such inputs.

For tags, hex literals should suffice. 3f 10 02 05 00 could disassemble to something like

# Malformed SEQUENCE
`3f10` {
  NULL {}
}

As a start, the comment trick also work for lengths, and maybe that's good enough:

# Malformed length = 2
SEQUENCE `8102`
  NULL {}

# Malformed SEQUENCE
# Malformed length = 2
`3f10` `8102`
   NULL {}

But this is not very malleable. Lengths, in particular, are a nuisance to count. Encoding tags by hand isn't that difficult, but still not ideal. The tag syntax is easy extensible to, say, [SEQUENCE long-form:2]. Lengths are a bit more ugly as they're attached to the curly braces, not the tag. (But if we have this syntax, we can use it for indefinite-length encoding too. 80 and 0000 are easy enough, but I guess it saves finding the closing brace when editing a large structure.

Add round-trip fuzzers

It should be the case that, for all byte strings b, ascii2der(der2ascii(b)) = b. Write a fuzzer for this. Also write a fuzzer to check that ascii2der doesn't panic.

One nuisance this'll run into is that I'm sure both functions stack overflow or go quadratic when given the wrong inputs. For the project's use cases, I hadn't considered this important, but the fuzzers will likely get stuck in that rabbithole.

Making certificates is a little tedious

The instructions here work, but they are a little involved. It would be nice to make this easier. I'm not sure exactly how, or even whether it should be a tool on top of ascii2der or built into ascii2der itself.

Currently I'm hovering around some way for an input file to assign names to chunks and then to pass those chunks into either built-in hooks for signing stuff (Go crypto libraries) or arbitrary program (OpenSSL and BoringSSL command-line tools). (Obviously, if the latter, it needs to either be separate tool or gated on some config or other command-line flag!)

Chromium has some templating system on top using PEM-like BEGIN and END markers, though they don't emit signatures.

Pretty-print UniversalString contents

Like BMPString, we should pretty-print UniversalString contents (UTF-32). After UTF-16 literals, the natural syntax would be U"hello world" with the same set of escapes and analogous semantics.

Decode BMPString contents more usefully.

BMPString contents currently decode into a hex literal, which isn't very helpful. They're used enough that they probably should have a syntax for it.

One proposal was u16"hello world" which seems reasonable. Maybe u"hello world" to match C++11. It does give u to UTF-16 instead of UTF-8, which is not the One True Encoding, but UTF-8 already gets to use unadorned quotes.

Installing executables with ‘go get’ in module mode is deprecated in Go 1.7

Documentation for the tool suggests the usage of go get for installing the command. However, this command is deprecated in Go 1.7. It takes some time for ones who are not experienced with Go eco-system to install the tool. I personally came up with the solution below but I'm sure there is a more Go-ish way to do it. Please update the documentation with the corresponding command(s).

git clone https://github.com/google/der-ascii.git
cd der-ascii
go install ./cmd/...