Need granular validation info about tls-observatory

[0xdiba]

I'm wondering about how we should change the schema to support that.

An idea would be to update the validationInfo struct with a Valid and an Invalid array of strings where we can store the domains according to the validation process.
Following that I don't know how we can save the errors about domains with invalid certificates in a Kibana friendly way...

[jvehent]

Another approach would be to limit the json of a certificate to one certificate for one domain. This way, a SAN certificate with 50 domains would be stored 50 times, and the validation info would be specific to the domain being stored.
It would make querying in Kibana a lot easier than storing the validation in an array.

[0xdiba]

That would me a good approach. How do you propose we change the IDs of the certificates to match that? In ES you can save only one document to each ID so the IDs must be a little different.
Should we append the domain name to the ID? I think this would make searching the domains for a single cert really easy.

Any other idea welcome :)

[jvehent]

I agree with appending the domain to the idea. Does ES have a max length for the ID?

[0xdiba]

No it doesn't have any limit.
Some people mention that it very long ids may slow ES down but I do not
think that our ids are going to be so long.
On 12 Feb 2015 17:07, "Julien Vehent" [email protected] wrote:

I agree with appending the domain to the idea. Does ES have a max length
for the ID?

—
Reply to this email directly or view it on GitHub
#18 (comment).

[0xdiba]

Should I change the id format for all the SAN certificates or append the domain only to the certificates that appear more than once ?

[jvehent]

I think it's more than just changing the ID. If we go with this format, a cert record should have only one domain entry, thus replacing Domains []string with Domain string.
The IPs []string behaviour should also be changed to only contain IPs where Domain was seen.
Thus it would make sense to change the ID format for everyone.

[0xdiba]

Of course. That is the implementation I went with i just asked to be sure.
I'm testing the code right now and I'm going to push in a while.

[0xdiba]

10a2893

[jvehent]

Before I push to prod, could you update the index schema as well?
https://github.com/mozilla/TLS-Observer/blob/master/certificates_schema.json

[0xdiba]

Done.

The only problem we have with the schema is how to store the validation info fields that are built dynamically ( with the name of every truststore ).