Introduction

This document takes a look at Event Tracing for Windows (ETW), it’s purpose and the resulting ecosystem that is available to interact with it.

What Is it

Event Tracing for Windows (ETW) is an efficient kernel-level tracing facility that lets you log kernel or application-defined events to a log file. You can consume the events in real time or from a log file

ETW

As seen in the above diagram, ETW is broken into four main components:

Controllers

Controllers provide the ability to start and stop event tracing sessions and enable or disable the providers that will log events into a given session.

Providers

Providers are what log events to sessions. They can either be enabled (logging events) or disabled (not logging events) by a controller. There are four main types of providers:

MOF (classic) providers
WPP providers
Manifest-based providers
TraceLogging providers

For a given provider, each type of event that can be logged has a corresponding schema, referred to as a manifest.

Sessions

Event sessions are what store the events coming from a set of providers, this can either go into an in memory buffer owned by the session or be routed directly into a log file (in the .etl format).

Consumers

Consumers are any application/kernel that reads events from either as a live stream from a session or directly from a .etl file.

Things to be Aware of

Structure & Schema of Events

The structure of a .etl file and the events that make it up is not something that is published. It is intended that to access these events while in the .etl format that one uses a Microsoft tool, such as tracerpt, or one of Microsoft’s APIs. There’s two ways I’ve found so far to access ETW traces:

Microsoft provided way to access Event Traces (live or from a logfile) via Trace Processing
ETWDeserializer
TraceEvent

ETW is strongly-typed in that each event has a specific schema (known as a manifest) for it’s event data. Event providers define that schema and make it available to consumers such that they can decode the event’s data. For a given .etl file there is a corresponding manifest collection that contain the relevant manifests for the events. This manifest collection can either be embedded in the .etl or sit to the side in an XML format. There’s no guarantee of the manifest being available. If not directly available, the decoder looks to see if the provider GUID’s in the event has a corresponding provider on the system that knows how to decode it.

Decoded event in XML (with known manifest):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
        <System>
                <Provider Guid="{9e814aad-3204-11d2-9a82-006008a86939}" />
                <EventID>0</EventID>
                <Version>3</Version>
                <Level>0</Level>
                <Task>0</Task>
                <Opcode>11</Opcode>
                <Keywords>0x0</Keywords>
                <TimeCreated SystemTime="2019-09-20T03:05:59.355838900-07:00" />
                <Correlation ActivityID="{00000000-0000-0000-0000-000000000000}" />
                <Execution ProcessID="4294967295" ThreadID="4294967295" ProcessorID="8" KernelTime="0" UserTime="0" />
                <Channel />
                <Computer />
        </System>
        <EventData>
                <Data Name="DiskNumber">      21</Data>
                <Data Name="IrpFlags">0x62A01</Data>
                <Data Name="TransferSize">    4096</Data>
                <Data Name="Reserved">       0</Data>
                <Data Name="ByteOffset">61037965312</Data>
                <Data Name="FileObject">0xFFFF870800209150</Data>
                <Data Name="Irp">0xFFFF9989E7B487F0</Data>
                <Data Name="HighResResponseTime">73</Data>
                <Data Name="IssuingThreadId">   10940</Data>
        </EventData>
        <RenderingInfo Culture="en-US">
                <Opcode>Write</Opcode>
                <Provider>MSNT_SystemTrace</Provider>
                <EventName xmlns="http://schemas.microsoft.com/win/2004/08/events/trace">DiskIo</EventName>
        </RenderingInfo>
        <ExtendedTracingInfo xmlns="http://schemas.microsoft.com/win/2004/08/events/trace">
                <EventGuid>{3d6fa8d4-fe05-11d0-9dda-00c04fd7ba7c}</EventGuid>
        </ExtendedTracingInfo>
</Event>

Decoded event in XML (without known manifest):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
        <System>
                <Provider Guid="{9e814aad-3204-11d2-9a82-006008a86939}" />
                <EventID>0</EventID>
                <Version>2</Version>
                <Level>0</Level>
                <Task>0</Task>
                <Opcode>32</Opcode>
                <Keywords>0x0</Keywords>
                <TimeCreated SystemTime="2019-09-20T03:05:59.355916600-07:00" />
                <Correlation ActivityID="{00000000-0000-0000-0000-000000000000}" />
                <Execution ProcessID="4294967295" ThreadID="4294967295" ProcessorID="8" KernelTime="0" UserTime="0" />
                <Channel />
                <Computer />
        </System>
        <ProcessingErrorData>
                <ErrorCode>111</ErrorCode>
                <DataItemName />
                <EventPayload>DAA88A009800000064180000BC2A0000145CBC81FA7F00006B4D3B7EFA7F0000CAC8460000000000CE1341000000000037DF450000000000C4B4470000000000CAB27B7F
        </ProcessingErrorData>
        <RenderingInfo Culture="en-US">
                <Opcode>Stack</Opcode>
                <Provider>MSNT_SystemTrace</Provider>
                <EventName xmlns="http://schemas.microsoft.com/win/2004/08/events/trace">StackWalk</EventName>
        </RenderingInfo>
        <ExtendedTracingInfo xmlns="http://schemas.microsoft.com/win/2004/08/events/trace">
                <EventGuid>{def2fe46-7bd6-4b80-bd94-f57fe20d0ce3}</EventGuid>
        </ExtendedTracingInfo>
</Event>

You can see with the above event that in the case we don’t have a manifest, the raw event payload is shown. Were we to have a manifest describing how to interpret that EventPayload, we would be able to properly decode it.