quic-tracker / dissector

A packet dissector

A test suite for QUIC

The test suite comprises a minimal Go implementation of QUIC which is currently draft-29 and TLS-1.3 compatible, as well as several test scenarii built upon this implementation. The test suite outputs its result as JSON files, which contains the result, the decrypted packets exchanged, as well as a pcap file and exporter secrets.

Installation

You should have Go 1.9, tcpdump, libpcap libraries and headers as well as openssl headers installed before starting.

go get -u github.com/QUIC-Tracker/quic-tracker  # This will fail because of the missing dependencies that should be build using the 4 lines below
cd $GOPATH/src/github.com/mpiraux/pigotls
make
cd $GOPATH/src/github.com/mpiraux/ls-qpack-go
make

The test suite is run using the scripts in bin/test_suite/. For help about their usage see:

go run bin/test_suite/scenario_runner.go -h
go run bin/test_suite/test_suite.go -h

Docker

Docker builds exist on Docker Hub.

docker run --network="host" mpiraux/quictracker /http_get -h
docker run --network="host" mpiraux/quictracker /scenario_runner -h
docker run --network="host" mpiraux/quictracker /test_suite -h
# Running the web app and mounting traces files from the current directory
docker run -v "$PWD:/traces" -p 5000:5000 mpiraux/quictracker flask run -h 0.0.0.0

dissector's Issues

Dissector unable to parse trace.

Hey!

I got this trace from a connection to a local MsQuic server. It should show the client sending a ClientHello, and the server replying with a Retry, however when I pass this to QUIC-Tracker I get an exception about the dissection:

127.0.0.1 - - [29/Jul/2020 15:22:09] "GET /traces/20200729/0 HTTP/1.1" 500 -
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2450, in wsgi_app
    response = self.handle_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1867, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/tiferrei/Developer/web-app/quic_tracker/app.py", line 101, in dissector
    trace = parse_trace(traces[trace_idx])
  File "/Users/tiferrei/Developer/web-app/quic_tracker/traces.py", line 34, in parse_trace
    packet['dissection'] = parse_packet(packet['data'], context[packet['direction']])
  File "/usr/local/lib/python3.8/site-packages/quic_tracker/dissector/__init__.py", line 47, in parse_packet
    raise last_e
  File "/usr/local/lib/python3.8/site-packages/quic_tracker/dissector/__init__.py", line 43, in parse_packet
    return p, parse_packet_with(buffer[:], protocol=yaml.load(f), context=context)
  File "/usr/local/lib/python3.8/site-packages/quic_tracker/dissector/__init__.py", line 65, in parse_packet_with
    raise last_e
  File "/usr/local/lib/python3.8/site-packages/quic_tracker/dissector/__init__.py", line 56, in parse_packet_with
    ret, inc, _ = parse_structure_type(buffer[:], top_struct, protocol, 0, context)
  File "/usr/local/lib/python3.8/site-packages/quic_tracker/dissector/__init__.py", line 101, in parse_structure_type
    raise ParseError('No structure could be parsed for type {}, first byte was {}'.format(type_name, buffer[0]))
quic_tracker.dissector.ParseError: No structure could be parsed for type Version Negotiation Packet, first byte was 246

This is the source trace:

[{"commit":"","scenario":"Adapter","scenario_version":1,"host":"implementation:4433","ip":"172.29.0.2","results":{},"started_at":1596032468,"duration":0,"error_code":0,"stream":[{"direction":"to_server","timestamp":1596032472788,"data":"wP8AAB0IDTjjQRNLYEUIyVOG2U7UxSgARMoABgBBAwEAAP8DA0AT7RtdLnCIx/VGMjjkjDw9MMwHy3VMSEa9nI0/naBKAAAGEwITARMDAQAA0AAzAEcARQAXAEEEIvsKxBB6EWXgHW/ycz9SVS3V2AnkPbAitJiE9AqXb1GCZ6q3yYH5fn0sK3wYTFt9TcakscQFfjxsZgd2S4qBuQAAABYAFAAAEXF1aWMudGlmZXJyZWkuY29tABAACAAGBWhxLTI5ACsACQgDBH8cfxt/GgANAAoACAgEBAMEAQIBAAoABgAEABcAHf+lAC8FBIAAQAAHBIAAQAAEBIAAgAAIAQEJAQMBBIAJJ8AOAQQDAkXADwjJU4bZTtTFKAAtAAMCAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA","is_of_interest":false},{"direction":"to_client","timestamp":1596032472791,"data":"9v8AAB0IyVOG2U7UxSgIj4rN1deU1yHU1vKacwEAAJ06MtRvL3HYWyoxwk0bgFP+gVIr7rENqfCWQ4qKjXiV+wTkbSxM8sgPY9db+HRj4O5RcWachNtSXNYZaWMlh094RDXwAAAAAN9BybZ/YYIKImODrcRP6hs=","is_of_interest":false}],"pcap":null,"qlog":{"qlog_version":"draft-01","title":"QUIC Adapter Trace","description":"QUIC-Tracker","summary":null,"traces":[{"vantage_point":{"name":"QUIC-Tracker","type":"client"},"title":"","description":"Connection to quic.tiferrei.com (172.29.0.2:4433), using version ff00001d and alpn hq-29","configuration":{"time_offset":"0","time_units":"us"},"common_fields":{"ODCID":"0d38e341134b6045","group_id":"0d38e341134b6045","reference_time":1596032468231306},"event_fields":["relative_time","category","event","data"],"events":[[4557145,"transport","packet_sent",{"packet_type":"initial","header":{"packet_number":"0","packet_size":1252,"payload_length":1226,"scil":"8","dcil":"8","scid":"c95386d94ed4c528","dcid":"0d38e341134b6045"},"frames":[{"frame_type":"crypto","offset":"0","length":"259"}]}],[4558556,"transport","packet_received",{"packet_type":"retry","header":{"packet_number":"0","packet_size":119,"scil":"8","dcil":"8","scid":"8f8acdd5d794d721","dcid":"c95386d94ed4c528"}}]]}]},"client_random":"Ya53AjQuKFvHV/4/gRWCqwbVdge3p5VZYiqRpI8yiaE=","secrets":{}}]

Any idea what might be happening here? I've been using QUIC-Tracker to debug some traces for a while with no problems. This is the first time I'm trying to visualise a retry, don't know if that's related or more of a red herring.

Thank you!
Tiago

__init__.py incompatible with the new pyyaml

init.py has the same problem as mentioned in the issue in web-app repo issue i created today