Feature idea: support ASN.1 (DER/BER) about fq

[wader]

Hey, some update. I have a basic BER decoder working now, also working on adding support for arbitrary large integer to the decoder which will be useful for ASN.1. and make it more useful for certificates and crypto related things i think. Hope i can push something in the next few days.

[wader]

Hi, yes that is a good idea. It would also require figuring out a nice way of passing in a grammar file to the decoder.
If you want to work on this i would recommend having a look at the protobutf decoder https://github.com/wader/fq/blob/master/format/protobuf/protobuf.go. It does not support passing in schemas but other decoders can pass it something schema-like internally to improve the decode output.

[gjvnq]

Thanks. I will take a look if I have time in January.

[wader]

👍 There are some other issues #25 #24 that will share som issues with this like how to provide an external description file and how to model the decoded value tree.

[wader]

Hey, now there is basic ASN1 BER support in master and also some kind of torepr support.

To make it a bit easer to work with PEM there is also frompem/topem now.

Any feedback what could be done differently or bugs are much appreciated.

Example reading PEM file, decode it and display it as decode tree, JSON and torepr JSON:

$ fq -o line_bytes=10 -d raw '(. | tobytes | tostring | println), (frompem | asn1_ber | d, tovalue, torepr)' format/asn1/testdata/test.pem
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDMYfnvWtC8Id5bPKae5yXSxQTt
+Zpul6AnnZWfI2TtIarvjHBFUtXRo96y7hoL4VWOPKGCsRqMFDkrbeUjRrx8iL91
4/srnyf6sh9c8Zk04xEOpK1ypvBz+Ks4uZObtjnnitf0NBGdjMKxveTq+VE7BWUI
yQjtQ8mbDOsiLLvh7wIDAQAB
-----END PUBLIC KEY-----
    │00 01 02 03 04 05 06 07 08 09│0123456789│.{}: (asn1_ber)
0x00│30                           │0         │  class: "universal" (0)
0x00│30                           │0         │  form: "constructed" (1)
0x00│30                           │0         │  tag: "sequence" (0x10)
0x00│   81 9f                     │ ..       │  length: 159
    │                             │          │  constructed[0:2]:
    │                             │          │    [0]{}:
0x00│         30                  │   0      │      class: "universal" (0)
0x00│         30                  │   0      │      form: "constructed" (1)
0x00│         30                  │   0      │      tag: "sequence" (0x10)
0x00│            0d               │    .     │      length: 13
    │                             │          │      constructed[0:2]:
    │                             │          │        [0]{}:
0x00│               06            │     .    │          class: "universal" (0)
0x00│               06            │     .    │          form: "primitive" (0)
0x00│               06            │     .    │          tag: "object_identifier" (0x6)
0x00│                  09         │      .   │          length: 9
    │                             │          │          value[0:7]:
0x00│                     2a      │       *  │            [0]: 1
0x00│                     2a      │       *  │            [1]: 2
0x00│                        86 48│        .H│            [2]: 840
0x0a│86 f7 0d                     │...       │            [3]: 113549
0x0a│         01                  │   .      │            [4]: 1
0x0a│            01               │    .     │            [5]: 1
0x0a│               01            │     .    │            [6]: 1
    │                             │          │        [1]{}:
0x0a│                  05         │      .   │          class: "universal" (0)
0x0a│                  05         │      .   │          form: "primitive" (0)
0x0a│                  05         │      .   │          tag: "null" (0x5)
0x0a│                     00      │       .  │          length: "indefinite" (0)
    │                             │          │          value: null
    │                             │          │    [1]{}:
0x0a│                        03   │        . │      class: "universal" (0)
0x0a│                        03   │        . │      form: "primitive" (0)
0x0a│                        03   │        . │      tag: "bit_string" (0x3)
0x0a│                           81│         .│      length: 141
0x14│8d                           │.         │
0x14│   00                        │ .        │      unused_bits_count: 0
0x14│      30 81 89 02 81 81 00 cc│  0.......│      value: raw bits
0x1e│61 f9 ef 5a d0 bc 21 de 5b 3c│a..Z..!.[<│
0x28│a6 9e e7 25 d2 c5 04 ed f9 9a│...%......│
*   │until 0xa1.7 (end) (140)     │          │
{
  "class": "universal",
  "constructed": [
    {
      "class": "universal",
      "constructed": [
        {
          "class": "universal",
          "form": "primitive",
          "length": 9,
          "tag": "object_identifier",
          "value": [
            1,
            2,
            840,
            113549,
            1,
            1,
            1
          ]
        },
        {
          "class": "universal",
          "form": "primitive",
          "length": "indefinite",
          "tag": "null",
          "value": null
        }
      ],
      "form": "constructed",
      "length": 13,
      "tag": "sequence"
    },
    {
      "class": "universal",
      "form": "primitive",
      "length": 141,
      "tag": "bit_string",
      "unused_bits_count": 0,
      "value": "<140>MIGJAoGBAMxh+e9a0Lwh3ls8pp7nJdLFBO35mm6XoCedlZ8jZO0hqu+McEVS1dGj3rLuGgvhVY48oYKxGowUOStt5SNGvHyIv3Xj+yufJ/qyH1zxmTTjEQ6krXKm8HP4qzi5k5u2OeeK1/Q0EZ2MwrG95Or5UTsFZQjJCO1DyZsM6yIsu+HvAgMBAAE="
    }
  ],
  "form": "constructed",
  "length": 159,
  "tag": "sequence"
}
[
  [
    [
      1,
      2,
      840,
      113549,
      1,
      1,
      1
    ],
    null
  ],
  "<0b10001100>MIGJAoGBAMxh+e9a0Lwh3ls8pp7nJdLFBO35mm6XoCedlZ8jZO0hqu+McEVS1dGj3rLuGgvhVY48oYKxGowUOStt5SNGvHyIv3Xj+yufJ/qyH1zxmTTjEQ6krXKm8HP4qzi5k5u2OeeK1/Q0EZ2MwrG95Or5UTsFZQjJCO1DyZsM6yIsu+HvAgMBAAE="
]