Parsing of large records about joy

[fliphil]

Hello! Just to make sure that I understand your problem correctly, you are saying that the TLS messages are being segmented into multiple smaller packets. This is resulting in cases where the TLS message headers have a length greater than the actual packet size (i.e. 3000 > 1500), and this is causing the sanity check at tls.c:1787 to fail.

[simon-struk]

Exactly. It happens when there are multiple messages inside one TLS handshake record - the length of the handshake records is the sum of the lengths of messages, like server hello and certificates. Admittedly, it might be a rare case - I found this behavior in a pcap containing Noris botnet traffic

[fliphil]

Thanks for looking at this in detail. I've also come to the same conclusions upon inspection in GDB. I think ultimately the solution will be to keep track of the segmentation state of the messages in order to piece the whole TLS message together when it is spread over multiple packets. Implementing the segmentation state tracking for TLS is on our work board, we hope to have that completed sometime in the near future :)

Is this something that you are interested in to implement? It sounds like you know your way around the code, and it would be great to have your help in getting this done sooner!

[fliphil]

I made an attempt at fixing this. Please take a look at this branch to see if it helps:
https://github.com/cisco/joy/tree/tls-continuity

[simon-struk]

The ServerHello message is now parsed, but the certificate part is not. I think the problem is not looking at the lengths of the messages and not handling 3-packet-long messages correctly. Maybe a Wireshark screenshot will help better explain what I have in mind. Here, the r->segmented becomes something over 3000 in the first packet and then it makes a mess when parsing the second packet (start points somewhere beyond the end of the packet, rem_len underflows).

This might require a more thorough revision of the code... on the other hand, the very fact that the messages are constructed in this way may be a good indicator that something is amiss. (this is botnet traffic)

[fliphil]

I added a new commit to address multiple handshake messages when looking for the certificates. Could you verify that solves the cert problem? I think we still need to add a logic clause within the r->segmented block (right before line 1772) to check if the segmentation amount is greater than the packet size. Something like:

if (r->segmented >= rem_len) { r->segmented -= rem_len; return; }

Could you see if making that modification works with your PCAP? Also could you please send that PCAP to me via [email protected]