microsoft / etl2pcapng

Utility that converts an .etl file containing a Windows network packet capture into .pcapng format.

About

This tool enables you to view ndiscap packet captures with Wireshark.

Windows ships with an inbox packet capture component called "ndiscap," which is implemented as an ETW trace provider. Due to performance problems with the other popular packet capture method (WinPcap, which was included with older versions of Wireshark), ndiscap should be preferred. A capture can be collected with:

netsh trace start capture=yes report=disabled
<repro>
netsh trace stop

The file generated by ndiscap is an etl file, which can be opened by ETW-centric tools like Microsoft Message Analyzer, but cannot be opened by Wireshark, which is the preferred tool for many engineers. Etl2pcapng.exe can convert the etl file to a pcapng file for opening with Wireshark.

Usage

Prebuilt binaries are available in the Releases section: https://github.com/microsoft/etl2pcapng/releases

Run the tool with:

etl2pcapng.exe in.etl out.pcapng

After converting the file, the tool prints a table which shows mappings between Windows interface indices and pcapng interface IDs.

The output pcapng file will have a comment on each packet indicating the PID of the current process when the packet was logged. WARNING: this is frequently not the same as the actual PID of the process which caused the packet to be sent or to which the packet was delivered, since the packet capture provider often runs in a DPC (which runs in an arbitrary process). The user should keep this in mind when using the PID information.

Building

In a command prompt in the src directory run the following. Cmake 3.15 at minimum is required.

mkdir build
cd build
cmake ..
cmake --build . --config Release

The binary will be at build/Release/etl2pcapng.exe

History

1.11.0 - Allows output filename to be omitted (e.g. "etl2pcapng.exe foo.etl". The output filename becomes "foo.pcapng").

1.10.0 - 10x faster.

1.9.0 - Adding support of Microsoft-Windows-Ras-NdisWanPacketCapture (VPN/RemoteAccess) events decoding + adding ThreadID (TID) to packet comments

1.8.0 - Adding RSS Hash value to packet comments for VMSwitch packets.

1.7.0 - Include VMSwitch packet info in packet comments.

1.6.0 - Enable ControlFlowGuard.

1.5.0 - Write iftype and ifindex into interface description blocks and statically link C runtime so vcredist doesn't need to be installed.

1.4.1 - Fix a bug leading to writing corrupt packets.

1.4.0 - Automatically infer original fragment length if captured fragments were truncated.

1.3.0 - Add a comment to each packet containing the process id (PID).

1.2.0 - Write direction info of each packet (epb_flags).

1.1.0 - Added support for multi-event packets found in traces from Win8 and older.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

etl2pcapng's Issues

No prerequisites listed

The tool crashes if vc_redist is not installed, only telling the user that vcredist140.dll is missing.

• This is not a friendly error message.
• The installation notes should list any dependencies.

Windows Defender detected: Trojan:Script/Woreflint.A!cl in the etl2pcapng.zip

Windows Defender detected: Trojan:Script/Woreflint.A!cl in the etl2pcapng.zip

Downloaded from Address: https://github-releases.githubusercontent.com/208918651/731e2400-75ce-11eb-89c4-2b372266f542?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210301%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210301T100459Z&X-Amz-Expires=300&X-Amz-Signature=17ee083f963039b5eff1d995d97a1fc3e1a5e69a384777b6a1a1519023bf0a66&X-Amz-SignedHeaders=host&actor_id=4356481&key_id=0&repo_id=208918651&response-content-disposition=attachment%3B%20filename%3Detl2pcapng.zip&response-content-type=application%2Foctet-stream

Referer: https://github.com/microsoft/etl2pcapng/releases/tag/1.50

Fails To Convert Packets

This version of etl2pcapng is failing to convert the packets at all that is captured using netsh. Have tried from multiple machines as well. Do we know what could be the issue.

Support for ETL files captured by Windows Performance Recorder

Windows Performance Recorder (WPR) can capture a lot of system information including network activities. Directly using this tool on the ETL file produced by WPR incurs a does not contain packet capture error. WPR and this etl2pcapng are both Microsoft offical tools, so I think it may not be so complicated to support the ETL files by WPR, any plans on this?

Input ETL file does not contain a packet capture.

I recently was asked to view an etl capture file for a friend of mine.

It has been a couple of years since we had done this, but the directions seemed to be the same.

1. Open command prompt on scanner
2. Start capture with command line: netsh trace start persistent=yes capture=yes tracefile=d:\log\networkTrace.etl
3. Recreate issue/exchange on network
4. Stop capture with command line: netsh trace stop
5. Copy networkTrace.etl file to USB stick

I have tried these same instructions and perform a ping to my router to generate some traffic.
When I attempt to convert it with etl2pcapng, it gives me "Input ETL file does not contain a packet capture."
I can open it in an old version of MS Message Analyzer and it does show data, but nothing that looks like a capture.

Have you heard of MS changing the netsh command or parameters to make this not work?

Thanks,
Mark

Support for packets captured by Microsoft-Windows-Ras-NdisWanPacketCapture

Situation: Capturing packets traversing F5 VPN tunnel using DTLS Over PPP connection. Captured using Microsoft-Windows-Ra-NdisWanPacketCapture provider

Command line used to generate traces:
netsh trace start tracefile=c:\working\netsh_tracev1.etl capture=yes report=disabled overwrite=yes provider=Microsoft-Windows-Ras-NdisWanPacketCapture

Output from ETL2PCAPNG:
PS C:\working> etl2pcapng netsh_tracev1.etl netsh_F5_trace_etl2pcapng.pcap
IF: medium=eth ID=0 IfIndex=17
IF: medium=eth ID=1 IfIndex=74
Converted 5113 frames
Only DTLS encrypted packets going over ppp tunnel are in PCAP file - PCAP file 4068KB

Output from Windows Message Analyzer Save-As->Export-All Messages>save
~10,078 frames+some none-network entries
Both DTLS encrypted packets as well as cleartext packets are in PCAP file

Arp packets to large after conversion

ls,

At the moment of writing the following seems to occur when I convert an elt trace file to an pcapng file there appears to be a problem converting the arp packets. There are represented to be over 2G in size. If i do a live capture on the same server with wireshark or netmon the arp packets are normal size.

Support for Filter Flags such as IP Address, Protocol, etc

netsh comes with quite a few filter flags when capturing the ETL file. I have noticed when using these filters that the resultant pcapng file has packets missing when comparing directly to wireshark capturing on the same interface (with the same filter).

Example filter:
netsh trace start capture=yes report=disabled tracefile=".\NetTrace.etl" persistent=no Protocol=17 Ethernet.Type=IPv4

This should capture all IPv4 UDP traffic, however, once I convert the file using etl2pcapng there are many packets missing. I have found this also using IP Address filters using the IPv4.Address flag. When using IPv4.SourceAddress or IPv4.DestinationAddress I don't seem to get any output in the pcapng file.

Are these filters supposed to work with etl2pcapng?

Make output file name an optional command-line parameter

It would be convenient for me if there was a default output filename that could be used instead of manually providing one.

For example, I would like to be able to run the command:
etl2pcapng.exe trace.etl

and get an output file called something like trace.etl.pcapng.

I'd be glad to help with the implementation if this sounds like a useful feature for others as well.

Wrong enhanced packet size when it has a comment

Hi, based on what I see in PcapNgWriteEnhancedPacket function, sizeof(EndOption) is added twice if a comment is provided.

Regards.

Crash on windows 7

The tool crashes on Windows 7 and Server 2008 R2, no matter if the input file exists or not.

Vanilla Win 7 SP1, vc_redist.x64 installed.

Log Name:      Application
Source:        Application Error
Date:          2020-02-26 08:44:38
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     [redacted]
Description:
Faulting application name: etl2pcapng.exe, version: 0.0.0.0, time stamp: 0x5e123992
Faulting module name: KERNELBASE.dll, version: 6.1.7601.17514, time stamp: 0x4ce7c78c
Exception code: 0xc0000005
Fault offset: 0x0000000000001ba4
Faulting process id: 0x694
Faulting application start time: 0x01d5ec7899325f47
Faulting application path: C:\temp\etl2pcapng.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: d6dee9fe-586b-11ea-85c8-0050568ea07c

Input ETL file does not contain an ndiscap packet capture.

Hi,

as the title suggests, analogue to #40 I have a very similar issue.
I captured about 3MB of Packet-Data in an ETL-file via:
netsh trace start capture=yes tracefile=D:\temp\Netcaps\foobar-2024-02-02-11-40.etl maxsize=4 filemode=single
and
netsh trace stop
about 20minutes later. (yeah, I know, not much traffic :P)

When I try to:
etl2pcapng.exe D:\temp\Netcaps\foobar-2024-02-02-11-40.etl
or
etl2pcapng.exe D:\temp\Netcaps\foobar-2024-02-02-11-40.etl D:\temp\Netcaps\foobar-2024-02-02-11-40.pcapng
I get the error message
"Input ETL file does not contain an ndiscap packet capture."

etl2pcapng.exe worked with other captures I did.

Can anyone explain the issue to me? Or has any other suggestions?

Thanks for the tool. Normally it works wonderfully.

Greetings
JNE

released version should be dropped in zip format Etl2Pcapng.zip

The Explorer plugin from James Kehr at https://github.com/microsoft/Convert-Etl2Pcapng expects to see a .zip file on the release server (https://github.com/microsoft/etl2pcapng/releases)

Can you please post the compressed Etl2Pcapng.zip there again? Thanks.

Reason: all engineers having installed the Explorer plugin are failing to convert ETL files to Wireshark pcap format after version 1.9.0 had been released.

Unable to download file

Hi,
Downloading the .zip file from the below gives virus/malware error in both Chrome and Firefox. Is this expected?
https://github.com/microsoft/etl2pcapng/releases/tag/1.50

Need support for multi-event packets

See code comment in EventCallback:

    // Supposedly, some packets may be logged across multiple events with the
    // use of these keywords. In that case we'll have to accumulate packet
    // fragments across multiple EventCallback calls. Add that feature if
    // anyone actually turns out to need it.

I've heard reports that people are seeing the warning event on some captures, so that work should be done.

TCP PID always 4

Please see attached screenshot. TCP protocol always has PID 4 , expected is 3492

support for etl file captured using pktmon?

Is there a difference in the format of the etl created using netsh trace and pktmon? When trying to convert an etl generated using pktmon the result is an empty file. Well not completely empty, but the pcapng coverted file only has this data:

\n\r\r\n\x1c\x00\x00\x00M<+\x1a\x01\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x1c\x00\x00\x00

Malware warnings

Several browsers block the newest zip and detect it as being malware. The x86 binary is detected by 21 AV engines (virustotal.com) and x64 version by one. Not cool.

Shouldn't CMake be used as a build system?

To make it easier building without VisualStudio.

Support winget Installation

We should support installation of this via winget, for example winget install etl2pcapng. This would allow for easy installation and update of the tool. The following tasks need to be accomplished:

1. Create an installer for etl2pcapng.exe and publish it on the GitHub release.
2. Add an entry to the winget repo to point to the installer

Show version number

running etl2pcapng.exe /? or just etl2pcapng.exe or any invocation for converting an Etl should tell the current version number 1.10.0

C:\>etl2pcapng.exe
etl2pcapng <infile> <outfile>
Converts a packet capture from etl to pcapng format.

Reason: user should be informed, if his local version is up-to-date (compared to info given on https://github.com/microsoft/etl2pcapng/releases)

• other tools like SysInternals tools always show the current version :-)

Is it importnat that etl2pcapng is run on the same machine where the etl was captured?

We're creating a document outlining network packet capture steps, and we'd like to know if etl2pcapng should be run on the same box where netsh trace start was run or whether it's okay for us to run it on a different diagnostic box after the fact? (Totally unrelated, but do you happen to know how to filter netsh trace to some port? Some internet posts suggest a TCP.AnyPort=X filter but that doesn't seem to work.) Thanks

Converted 0 frames

Using version 1.4.
Server 2019, always says converted 0 frames.

Crash

I had a ETL that I had already transferred to a Linux box after a lot of security control finagling...soon to find out Wireshark didnt support ithat format. Found etl2pcapng but of course it's for Windows when Im on LInux now lol. Decided to see if WINE would work since it looked small and self-contained.. Got the errors below. This may be a WINE thing but just in case it's not gonna post about it here.

wine64 etl2pcapng/x64/etl2pcapng.exe NetTrace.etl WindowsPcap.pcapng 87 ⨯

0009:fixme:advapi:OpenTraceW 0x22fa10: stub
OpenTrace failed with 5
0009:fixme:kernelbase:AppPolicyGetProcessTerminationMethod FFFFFFFFFFFFFFFA, 000000000022FB90
    

_On a sidenote I was surprised there was no Linux tool for this. Maybe I'll get of my rump, spend a week trying to understand this code and see if I can port it to .NET CORE _

Using GitHub Actions to automate releases

Hey we could use GitHub Actions to automate building and releasing the project itself.

Given that git tags are already being used, I would recommend something in which the maintainers will have to create a tag after which a GitHub Action will execute and perform the deployment

Is that something which the maintainers would interested in?

Additional data in comment. Add process name with PID in comment

not recognized event

Hi
i recoded by this commnd (Win 10 Home last update)
pktmon start --capture --comp {} --pkt-size 0 -f {}
and save the ETL
when i use the tool here i go error "Input ETL file does not contain a packet capture"

but when i use "PktMon.exe etl2pcap PktMon.etl -o dd.pcap" [inside the system]
Processing...

Packets total: 8
Packet drop count: 0
Packets formatted: 8
Formatted file: dd.pcap

all good

what the difference ?

Possible Linux/ macOS port?

I have some analysis scripts for pcap (should also work on pcapng) files but they are tested only on Linux. Would it be feasible to port etl2pcapng to be able to run on Linux (and maybe macOS) as well?
There seems to be some Windows dependencies in the source code. I am not sure how hard it would be to abstract those away.

Seg Fault on 2008 r2 sp1

Tried running v1.3.0 from 2008 r2 sp1 and it crashes. See screenshots.

Conversion is not working. 0 packets converted.

I am taking authlogs for Azure AD and I get .etl file and want to convert it to pcapng.
But it's not working.
user to work about a month ago.