certtools / ieps Goto Github PK

As a follow-up to the discussion in the hackathon, we need to define the format of the "format" field.

We have several related fields:

• The format name
• The format version
• The type in that version

One suggestion in the hackathon was that the type should be a child of the format field. So I start the discussion with two proposals:

A:

{
    "format": "intelmq",
    "version": 1,
    "type": "event",
    ...
}

B:

{
    "format": {
        "name": "intelmq"
        "version": 1,
        "type": "event"
    },
    ...
}

Valid values for the type field are event and report. Also replaces the __type field which we currently have in the payload.

cc @aaronkaplan @adulau @certbe-trey

https://github.com/certtools/intelmq/tree/schacht/http-mixin does not work anymore

Not sure I fully understand the document and proposed format.

The UUID mentioned is the UUID of the IntelMQ instance or a unique UUID per message?

I would recommend to have a unique UUID per IntelMQ instance.

@aaronkaplan please write a new IEP on the protocol extension mechanism.

I think the IEP03 is very well written, thank you a lot for this! Thinking this through was important and I think Sebastian Waldbauer did a great job.

Reading it, I realised that my initial proposal of having multiple values is really breaking the KISS principle of IntelMQ in a bad way. Worse than I had thought. So, I am thinking of retracting the proposal.

However, .... https://github.com/certtools/ieps/tree/main/003#alternatives has a good core in it.

If we have multiple values, instead of doing the n x m complexity explosion, we link different events (JSON rows) together via UUIDs this gives us what we need:

• UUIDs help with deduplication! That's important when linking IntelMQ instances!
• lower complexity / keep the KISS principle
• consumers can ignore the UUID-linking if it's not relevant for them (f.ex enrichment processes/bots)
• we can still represent linked events.

I would like to add one little but important thing for the UUID linking idea: add a "link-type".

Examples for link-types:

• parent-child event
• grouping types (all of these events belong to the same report)
  etc.

With this triplet information , we are close to RDF (left-side, type, right-side) and thus we can (future-proof) represent any type of relation.

A list of valid types needs to be documented in the IDF format page of course.

So, I think with that, we can go ahead.

Thanks,
a.