-<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/xhtml1-strict.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+
<head>
- <title>The new LTT trace format</title>
+ <title>The LTTng trace format</title>
</head>
- <body>
-<h1>The new LTT trace format</h1>
+<body>
+
+<h1>The LTTng trace format</h1>
+
+<p>
+<em>Last update: 2008/06/02</em>
+</p>
+
+<p>
+This document describes the LTTng trace format. It should be useful mainly to
+developers who code the LTTng tracer or the traceread LTTV library, as this
+library offers all the necessary abstractions on top of the raw trace data.
+</p>
-<P>
-A trace is contained in a directory tree. To send a trace remotely,
-the directory tree may be tar-gzipped. Trace foo, placed in the home
-directory of user john, /home/john, would have the following content:
+<p>
+A trace is contained in a directory tree. To send a trace remotely, the
+directory tree may be tar-gzipped. The trace <tt>foo</tt>, placed in the home
+directory of user john, /home/john, would have the following contents:
+</p>
-<PRE><TT>
+<pre><tt>
$ cd /home/john
$ tree foo
foo/
-|-- eventdefs
-| |-- core.xml
-| |-- net.xml
-| |-- ipv4.xml
-| `-- ide.xml
-|-- info
-| |-- bookmarks.xml
-| `-- system.xml
|-- control
-| |-- facilities
-| |-- interrupts
-| `-- processes
-`-- cpu
- |-- 0
- |-- 1
- |-- 2
- `-- 3
-</TT></PRE>
-
-<P>
-The eventdefs directory contains the events descriptions for all the
-facilities used. The syntax is a simple subset of XML; XML is widely
-known and easily parsed or hand edited. Each file contains one or more
-<FACILITY NAME=name>...</FACILITY> elements. Indeed, several
-facilities may have the same name but different content (and thus will
-generate a different checksum), typically when the event descriptions
-for a given facility change from one version to the next, if a module
-is recompiled and reloaded during a trace.
-
-<P>
-A small number of events are predefined, part of the "builtin" facility,
-and are not present there. These "builtin" events include "facility_load",
-"block_start", "block_end" and "time_heartbeat".
-
-<P>
-The cpu directory contains a tracefile for each cpu, numbered from 0,
-in .trace format. A uniprocessor thus only contains the file cpu/0.
+| |-- facilities_0
+| |-- facilities_1
+| |-- facilities_...
+| |-- interrupts_0
+| |-- interrupts_1
+| |-- interrupts_...
+| |-- modules_0
+| |-- modules_1
+| |-- modules_...
+| |-- network_0
+| |-- network_1
+| |-- network_...
+| |-- processes_0
+| |-- processes_1
+| `-- processes_...
+|-- cpu_0
+|-- cpu_1
+`-- cpu_...
+
+</tt></pre>
+
+<p>
+The root directory contains a tracefile for each cpu, numbered from 0,
+in .trace format. A uniprocessor thus only contains the file cpu_0.
A multi-processor with some unused (possibly hotplug) CPU slots may have some
-unused CPU numbers. For instance a 8 way SMP board with 6 CPUs randomly
+unused CPU numbers. For instance an 8 way SMP board with 6 CPUs randomly
installed may produce tracefiles named 0, 1, 2, 4, 6, 7.
-
-<P>
-The files in the control directory also follow the .trace format.
-The "facilities" file only contains "builtin" facility_load events
-and is used to determine the facilities used and the code range assigned
-to each facility. The other control files contain the initial system
-state and various subsequent important events, for example process
-creations and exit. The interest of placing such subsequent events
-in control trace files instead of (or in addition to) in the per cpu
-trace files is that they may be accessed more quickly/conveniently
-and that they may be kept even when the per cpu files are overwritten
-in "flight recorder mode".
-
-<P>
-The info directory contains in system.xml a description of the system on which
-the trace was created as well as different user annotations in bookmark.xml.
-This directory may also contain various information about the trace, generated
-during trace analysis (statistics, index...).
-
-
-<H2>Trace format</H2>
-
-<P>
-Each tracefile is divided into equal size blocks with an uint32 at the block
-end giving the offset to the last event in the block. Events are packed
-sequentially in the block starting at offset 0 with a "block_start" event
-and ending, at the offset stored in the last 4 bytes of the block, with a
-block_end event. Both the block_start and block_end events
-contain the kernel timestamp (timespec binary structure,
-uint32 seconds, uint32 nanoseconds), the cycle counter (uint64 cycles),
-and the buffer id (uint64).
-
-<P>
-Each event consists in an event type id (uint16 which is the event type id
-within the facility + the facility base id), a time delta (uint32 in cycles
-or nanoseconds, depending on configuration, since the last time value, in the
-block header or in a "time_heartbeat" event) and the event type specific data.
-All values are packed in native byte order binary format.
-
-
-<H2>System description</H2>
-
-<P>
+</p>
+
+<p>
+The files in the control directory also follow the .trace format and are
+also per cpu. The "facilities" files only contain "core" marker_id,
+marker_format and time_heartbeat events. The first two are used to describe the
+events that are in the trace. The other control files contain the initial
+system state and various subsequent important events, for example process
+creations and exit. The interest of placing such subsequent events in control
+trace files instead of (or in addition to) in the per cpu trace files is that
+they may be accessed more quickly/conveniently and that they may be kept even
+when the per cpu files are overwritten in "flight recorder mode".
+</p>
+
+<h2>Trace format</h2>
+
+<p>
+Each tracefile is divided into equal size blocks with a header at the beginning
+of the block. Events are packed sequentially in the block starting right after
+the block header.
+</p>
+
+<p>
+Each block consists of :
+</p>
+
+<pre><tt>
+block start/end header
+trace header
+event 1 header
+event 1 variable length data
+event 2 header
+event 2 variable length data
+....
+padding
+</tt></pre>
+
+<h3>The block start/end header</h3>
+
+<pre><tt>
+begin
+ * the beginning of buffer information
+ uint64 cycle_count
+ * TSC at the beginning of the buffer
+ uint64 freq
+ * frequency of the CPUs at the beginning of the buffer.
+end
+ * the end of buffer information
+ uint64 cycle_count
+ * TSC at the end of the buffer
+ uint64 freq
+ * frequency of the CPUs at the end of the buffer.
+uint32 lost_size
+ * number of bytes of padding at the end of the buffer.
+uint32 buf_size
+ * size of the sub-buffer.
+</tt></pre>
+
+
+
+<h3>The trace header</h3>
+
+<pre><tt>
+uint32 magic_number
+ * 0x00D6B7ED, used to check the trace byte order vs host byte order.
+uint32 arch_type
+ * Architecture type of the traced machine.
+uint32 arch_variant
+ * Architecture variant of the traced machine. May be unused on some arch.
+uint32 float_word_order
+ * Byte order of floats and doubles, sometimes different from integer byte
+ order. Useful only for user space traces.
+uint8 arch_size
+ * Size (in bytes) of the void * on the traced machine.
+uint8 major_version
+ * major version of the trace.
+uint8 minor_version
+ * minor version of the trace.
+uint8 flight_recorder
+ * Is flight recorder mode activated ? If yes, data might be missing
+ (overwritten) in the trace.
+uint8 has_heartbeat
+ * Does this trace have heartbeat timer event activated ?
+ Yes (1) -> Event header has 32 bits TSC
+ No (0) -> Event header has 64 bits TSC
+uint8 alignment
+ * Are event headers in this trace aligned ?
+ Yes -> the value indicates the alignment
+ No (0) -> data is packed.
+uint8 tsc_lsb_truncate
+ * Used for compact channels
+uint8 tscbits
+ * Used for compact channels
+uint8 compact_data_shift
+ * Used for compact channels
+uint32 freq_scale
+ event time is always calculated from :
+ trace_start_time + ((event_tsc - trace_start_tsc) * (freq / freq_scale))
+uint64 start_freq
+ * CPUs clock frequency at the beginnig of the trace.
+uint64 start_tsc
+ * TSC at the beginning of the trace.
+uint64 start_monotonic
+ * monotonically increasing time at the beginning of the trace.
+ (currently not supported)
+start_time
+ * Real time at the beginning of the trace (as given by date, adjusted by NTP)
+ This is the only time reference with the real world : the rest of the trace
+ has monotonically increasing time from this point (with TSC difference and
+ clock frequency).
+ uint32 seconds
+ uint32 nanoseconds
+</tt></pre>
+
+
+<h3>Event header</h3>
+
+<p>
+Event headers differ according to the following conditions : does the
+traced system have a heartbeat timer? Is tracing alignment activated?
+</p>
+
+<p>
+Event header :
+</p>
+<pre><tt>
+{ uint32 timestamp
+ or
+ uint64 timestamp }
+ * if has_heartbeat : 32 LSB of the cycle counter at the event record time.
+ * else : 64 bits complete cycle counter.
+uint8 facility_id
+ * Numerical ID of the facility corresponding to the event. See the facility
+ tracefile to know which facility ID matches which facility name and
+ description.
+uint8 event_id
+ * Numerical ID of the event inside the facility.
+uint16 event_size
+ * Size of the variable length data that follows this header.
+</tt></pre>
+
+<p>
+Event header alignment
+</p>
+
+<p>
+If trace alignment is activated (<tt>alignment</tt>), the event header is
+aligned. In addition, padding is automatically added after the event header so
+the variable length data is automatically aligned on the architecture size.
+</p>
+
+<!--
+<h2>System description</h2>
+
+<p>
The system type description, in system.xml, looks like:
+</p>
-<PRE><TT>
+<pre><tt>
<system
node_name="vaucluse"
domainname="polymtl.ca"
>
Some comments about the system
</system>
-</TT></PRE>
+</tt></pre>
-<P>
+<p>
The system attributes kernel_name, node_name, kernel_release,
kernel_version, machine, processor, hardware_platform and operating_system
come from the uname(1) program. The domainname attribute is obtained from
-the "hostname --domain" command. The arch_size attribute is one of
+the "hostname --domain" command. The arch_size attribute is one of
LP32, ILP32, LP64 or ILP64 and specifies the length in bits of integers (I),
long (L) and pointers (P). The endian attribute is "little" or "big".
While the arch_size and endian attributes could be deduced from the platform
platforms. The cpu attribute specifies the maximum number of processors in
the system; only tracefiles 0 to this maximum - 1 may exist in the cpu
directory.
+</p>
-<P>
+<p>
Within the system element, the text enclosed may describe further the
system traced.
+</p>
-<H2>Event type descriptions</H2>
-
-<P>
-A facility contains the descriptions of several event types. When a structure
-is reused in several event types, a named type is defined and may be referenced
-by several other event types or named types.
-
-<PRE><TT>
-<facility name=facility_name>
- <description>Some text</description>
- <event name=eventtype_name>
- <description>Some text</description>
- --type structure--
- </event>
- ...
- <type name=type_name>
- --type structure--
- </type>
-</facility>
-</TT></PRE>
-
-<P>
-The type structure may be one of the following primitive type elements.
-Whenever the keyword isize is used, the allowed values are
-short, medium, long, 1, 2, 4, 8, indicating the size in bytes.
-The fsize keyword represents one of medium, long, 4 and 8 bytes.
-
-<PRE><TT>
-<int size=isize format="printf format"/>
-
-<uint size=isize format="printf format"/>
-
-<float size=fsize format="printf format"/>
-
-<string format="printf format"/>
+<h2>Bookmarks</h2>
-<enum size=isize format="printf format">label1 label2 ...</enum>
-</TT></PRE>
-
-<P>
-The string is null terminated. For the enumeration, the size of the integer
-used for its representation is specified.
-
-<P>
-The type structure may also be a compound type.
-
-<PRE><TT>
-<array size=n> --type structure-- </array>
-
-<sequence lengthsize=isize> --type structure-- </sequence>
-
-<struct>
- <field name=field_name>
- <description>Some text</description>
- --type structure--
- </field>
- ...
-</struct>
-
-<union typecodesize=isize>
- <field name=field_name>
- <description>Some text</description>
- --type structure--
- </field>
- ...
-</union>
-</TT></PRE>
-
-<P>
-Array is a fixed size array of length size. Sequence is a variable size
-array with its length stored as a prepended uint of length lengthsize.
-A structure is simply an aggregation of fields. An union is one of its n
-fields (variant record), as indicated by a preceeding code (0 to n - 1)
-of the specified size typecodesize.
-
-<P>
-Finally the type structure may be defined by referencing a named type.
-
-<PRE><TT>
-<typeref name=type_name/>
-</PRE></TT>
-
-<H2>Builtin events</H2>
-
-<P>
-The facility named "builtin" is always present and contains at least the
-following event types.
-
-<PRE><TT>
-<event name=facility_load>
- <description>Facility used in the trace</description>
- <struct>
- <field name="name"><string/></field>
- <field name="checksum"><uint size=4/></field>
- <field name="base_code"><uint size=4/></field>
- </struct>
-</event>
-
-<event name=block_start>
- <description>Block start timestamp</description>
- <typeref name=block_timestamp/>
-</event>
-
-<event name=block_end>
- <description>Block end timestamp</description>
- <typeref name=block_timestamp/>
-</event>
-
-<event name=time_heartbeat>
- <description>System time values sent periodically to minimize cycle counter
- drift with respect to real time clock and to detect cycle counter
- rollovers
- </description>
- <typeref name=timestamp/>
-</event>
-
-<type name=block_timestamp>
- <struct>
- <field name=timestamp><typeref name=timestamp></field>
- <field name=block_id><uint size=4/></field>
- </struct>
-</type>
-
-<type name=timestamp>
- <struct>
- <field name=time><typeref name=timespec/></event>
- <field name="cycle_count"><uint size=8/></field>
- </struct>
-</event>
-
-<type name=timespec>
- <struct>
- <field name="seconds"><uint size=4/></field>
- <field name="nanoseconds"><uint size=4/></field>
- </struct>
-</type>
-</TT></PRE>
-
-<H2>Control files</H2>
-
-<P>
-The interrupts file reflects the content of the /proc/interrupts system file.
-It contains one event describing each interrupt. At trace start, events are
-generated describing all the current interrupts. If the assignment of
-interrupts changes later, due to devices or device drivers being activated or
-deactivated, additional events may be added to the file. Each interrupt
-event has the following structure.
-
-<PRE><TT>
-<event name=interrupt>
- <description>Interrupt request number assignment<description>
- <struct>
- <field name="number"><uint size=4/></field>
- <field name="count"><uint size=4/></field>
- <field name="controller"><string/></field>
- <field name="name"><string/></field>
- </struct>
-</event>
-</TT></PRE>
-
-<P>
-The processes file contains the list of processes already created when the
-trace starts. Each process describing event is modeled after the
-/proc/self/status system file. The number of fields in this event is
-expected to be expanded in the future to include groups, signal masks,
-opened file descriptors and address maps.
-
-<PRE><TT>
-<event name=process>
- <description>Existing process<description>
- <struct>
- <field name="name"><string/></field>
- <field name="pid"><uint size=4/></field>
- <field name="ppid"><uint size=4/></field>
- <field name="tracer_pid"><uint size=4/></field>
- <field name="uid"><uint size=4/></field>
- <field name="euid"><uint size=4/></field>
- <field name="suid"><uint size=4/></field>
- <field name="fsuid"><uint size=4/></field>
- <field name="gid"><uint size=4/></field>
- <field name="egid"><uint size=4/></field>
- <field name="sgid"><uint size=4/></field>
- <field name="fsgid"><uint size=4/></field>
- <field name="state"><enum size=4>
- Running WaitInterruptible WaitUninterruptible Zombie Traced Paging
- </enum></field>
- </struct>
-</event>
-</TT></PRE>
-
-<H2>Facilities</H2>
-
-<P>
-Facilities define a granularity of events grouping for filtering, activation
-and compilation. Each facility does cost a table entry in the kernel (name,
-checksum, event type code range), or somewhere between 20 and 30 bytes. Having
-one facility per tracing statement in the kernel would be too much (assuming
-that they eventually are routinely inserted in the kernel code and replace
-the 80000+ printk statements in some proportion). However, having a few
-facilities, up to a few tens, would make sense.
-
-<P>
-The "builtin" facility contains a small number of predefined events which must
-always exist. The "core" facility contains a small subset of OS events which
-are almost always of interest (scheduling, interrupts, faults, system calls).
-Then, specialized facilities may exist for each subsystem (network, disks,
-USB, SCSI...).
-
-
-<H2>Bookmarks</H2>
-
-<P>
+<p>
Bookmarks are user supplied information added to a trace. They contain user
annotations attached to a time interval.
+</p>
+
-<PRE><TT>
+<pre><tt>
<bookmarks>
<location name=name cpu=n start_time=t end_time=t>Some text</location>
...
</bookmarks>
-</TT></PRE>
+</tt></pre>
-<P>
+<p>
The interval is defined using either "time=" or "start_time=" and
"end_time=", or "cycle=" or "start_cycle=" and "end_cycle=".
The time is in seconds with decimals up to nanoseconds and cycle counts
are unsigned integers with a 64 bits range. The cpu attribute is optional.
+</p>
-</BODY>
-</HTML>
-
-
-
-
+-->
+</body>
+</html>
projects / lttv.git / blobdiff