ltt/branches/poly/doc/developer/format.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/2002/REC-xhtml1-20020801/DTD/xhtml1-strict.dtd">
   2 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   3
   4 <head>
   5   <title>The LTTng trace format</title>
   6 </head>
   7
   8 <body>
   9
  10 <h1>The LTTng trace format</h1>
  11
  12 <p>
  13 <em>Last update: 2008/06/02</em>
  14 </p>
  15
  16 <p>
  17 This document describes the LTTng trace format. It should be useful mainly to
  18 developers who code the LTTng tracer or the traceread LTTV library, as this
  19 library offers all the necessary abstractions on top of the raw trace data.
  20 </p>
  21
  22 <p>
  23 A trace is contained in a directory tree. To send a trace remotely, the
  24 directory tree may be tar-gzipped. The trace <tt>foo</tt>, placed in the home
  25 directory of user john, /home/john, would have the following contents:
  26 </p>
  27
  28 <pre><tt>
  29 $ cd /home/john
  30 $ tree foo
  31 foo/
  32 |-- control
  33 |   |-- facilities_0
  34 |   |-- facilities_1
  35 |   |-- facilities_...
  36 |   |-- interrupts_0
  37 |   |-- interrupts_1
  38 |   |-- interrupts_...
  39 |   |-- modules_0
  40 |   |-- modules_1
  41 |   |-- modules_...
  42 |   `-- processes_0
  43 |   `-- processes_1
  44 |   `-- processes_...
  45 |-- cpu_0
  46 |-- cpu_1
  47 `-- cpu_...
  48
  49 </tt></pre>
  50
  51 <p>
  52 The root directory contains a tracefile for each cpu, numbered from 0,
  53 in .trace format. A uniprocessor thus only contains the file cpu_0.
  54 A multi-processor with some unused (possibly hotplug) CPU slots may have some
  55 unused CPU numbers. For instance an 8 way SMP board with 6 CPUs randomly
  56 installed may produce tracefiles named 0, 1, 2, 4, 6, 7.
  57 </p>
  58
  59 <p>
  60 The files in the control directory also follow the .trace format and are
  61 also per cpu.  The "facilities" files only contain "core" marker_id,
  62 marker_format and time_heartbeat events. The first two are used to describe the
  63 events that are in the trace. The other control files contain the initial
  64 system state and various subsequent important events, for example process
  65 creations and exit. The interest of placing such subsequent events in control
  66 trace files instead of (or in addition to) in the per cpu trace files is that
  67 they may be accessed more quickly/conveniently and that they may be kept even
  68 when the per cpu files are overwritten in "flight recorder mode".
  69 </p>
  70
  71 <h2>Trace format</h2>
  72
  73 <p>
  74 Each tracefile is divided into equal size blocks with a header at the beginning
  75 of the block. Events are packed sequentially in the block starting right after
  76 the block header.
  77 </p>
  78
  79 <p>
  80 Each block consists of :
  81 </p>
  82
  83 <pre><tt>
  84 block start/end header
  85 trace header
  86 event 1 header
  87 event 1 variable length data
  88 event 2 header
  89 event 2 variable length data
  90 ....
  91 padding
  92 </tt></pre>
  93
  94 <h3>The block start/end header</h3>
  95
  96 <pre><tt>
  97 begin
  98         * the beginning of buffer information
  99         uint64 cycle_count
 100                 * TSC at the beginning of the buffer
 101         uint64 freq
 102                 * frequency of the CPUs at the beginning of the buffer.
 103 end
 104         * the end of buffer information
 105         uint64 cycle_count
 106                 * TSC at the end of the buffer
 107         uint64 freq
 108                 * frequency of the CPUs at the end of the buffer.
 109 uint32 lost_size
 110         * number of bytes of padding at the end of the buffer.
 111 uint32 buf_size
 112         * size of the sub-buffer.
 113 </tt></pre>
 114
 115
 116
 117 <h3>The trace header</h3>
 118
 119 <pre><tt>
 120 uint32 magic_number
 121         * 0x00D6B7ED, used to check the trace byte order vs host byte order.
 122 uint32 arch_type
 123         * Architecture type of the traced machine.
 124 uint32 arch_variant
 125         * Architecture variant of the traced machine. May be unused on some arch.
 126 uint32 float_word_order
 127         * Byte order of floats and doubles, sometimes different from integer byte
 128           order. Useful only for user space traces.
 129 uint8 arch_size
 130         * Size (in bytes) of the void * on the traced machine.
 131 uint8 major_version
 132         * major version of the trace.
 133 uint8 minor_version
 134         * minor version of the trace.
 135 uint8 flight_recorder
 136         * Is flight recorder mode activated ? If yes, data might be missing
 137           (overwritten) in the trace.
 138 uint8   has_heartbeat
 139         * Does this trace have heartbeat timer event activated ?
 140                 Yes (1) -> Event header has 32 bits TSC
 141                 No (0) -> Event header has 64 bits TSC
 142 uint8 alignment
 143         * Are event headers in this trace aligned ?
 144                 Yes -> the value indicates the alignment
 145                 No (0) -> data is packed.
 146 uint8 tsc_lsb_truncate
 147 uint8 tscbits
 148 uint8 compact_data_shift
 149 uint32 freq_scale
 150                 event time is always calculated from :
 151                         trace_start_time + ((event_tsc - trace_start_tsc) * (freq / freq_scale))
 152 uint64 start_freq
 153         * CPUs clock frequency at the beginnig of the trace.
 154 uint64 start_tsc
 155         * TSC at the beginning of the trace.
 156 uint64 start_monotonic
 157         * monotonically increasing time at the beginning of the trace.
 158                 (currently not supported)
 159 start_time
 160         * Real time at the beginning of the trace (as given by date, adjusted by NTP)
 161                 This is the only time reference with the real world : the rest of the trace
 162                 has monotonically increasing time from this point (with TSC difference and
 163                 clock frequency).
 164         uint32 seconds
 165         uint32 nanoseconds
 166 </tt></pre>
 167
 168
 169 <h3>Event header</h3>
 170
 171 <p>
 172 Event headers differ according to the following conditions : does the
 173 traced system have a heartbeat timer? Is tracing alignment activated?
 174 </p>
 175
 176 <p>
 177 Event header :
 178 </p>
 179 <pre><tt>
 180 { uint32 timestamp
 181         or
 182         uint64 timestamp }
 183         * if has_heartbeat : 32 LSB of the cycle counter at the event record time.
 184         * else : 64 bits complete cycle counter.
 185 uint8 facility_id
 186         * Numerical ID of the facility corresponding to the event. See the facility
 187           tracefile to know which facility ID matches which facility name and
 188                 description.
 189 uint8 event_id
 190         * Numerical ID of the event inside the facility.
 191 uint16 event_size
 192         * Size of the variable length data that follows this header.
 193 </tt></pre>
 194
 195 <p>
 196 Event header alignment
 197 </p>
 198
 199 <p>
 200 If trace alignment is activated (<tt>alignment</tt>), the event header is
 201 aligned. In addition, padding is automatically added after the event header so
 202 the variable length data is automatically aligned on the architecture size.
 203 </p>
 204
 205 <!--
 206 <h2>System description</h2>
 207
 208 <p>
 209 The system type description, in system.xml, looks like:
 210 </p>
 211
 212 <pre><tt>
 213 &lt;system
 214  node_name="vaucluse"
 215  domainname="polymtl.ca"
 216  cpu=4
 217  arch_size="ILP32"
 218  endian="little"
 219  kernel_name="Linux"
 220  kernel_release="2.4.18-686-smp"
 221  kernel_version="#1 SMP Sun Apr 14 12:07:19 EST 2002"
 222  machine="i686"
 223  processor="unknown"
 224  hardware_platform="unknown"
 225  operating_system="Linux"
 226  ltt_major_version="2"
 227  ltt_minor_version="0"
 228  ltt_block_size="100000"
 229 &gt;
 230 Some comments about the system
 231 &lt;/system&gt;
 232 </tt></pre>
 233
 234 <p>
 235 The system attributes kernel_name, node_name, kernel_release,
 236  kernel_version, machine, processor, hardware_platform and operating_system
 237 come from the uname(1) program. The domainname attribute is obtained from
 238 the "hostname &#045;&#045;domain" command. The arch_size attribute is one of
 239 LP32, ILP32, LP64 or ILP64 and specifies the length in bits of integers (I),
 240 long (L) and pointers (P). The endian attribute is "little" or "big".
 241 While the arch_size and endian attributes could be deduced from the platform
 242 type, having these explicit allows analysing traces from yet unknown
 243 platforms. The cpu attribute specifies the maximum number of processors in
 244 the system; only tracefiles 0 to this maximum - 1 may exist in the cpu
 245 directory.
 246 </p>
 247
 248 <p>
 249 Within the system element, the text enclosed may describe further the
 250 system traced.
 251 </p>
 252
 253
 254 <h2>Bookmarks</h2>
 255
 256 <p>
 257 Bookmarks are user supplied information added to a trace. They contain user
 258 annotations attached to a time interval.
 259 </p>
 260
 261
 262 <pre><tt>
 263 &lt;bookmarks&gt;
 264   &lt;location name=name cpu=n start_time=t end_time=t&gt;Some text&lt;/location&gt;
 265   ...
 266 &lt;/bookmarks&gt;
 267 </tt></pre>
 268
 269 <p>
 270 The interval is defined using either "time=" or "start_time=" and
 271 "end_time=", or "cycle=" or "start_cycle=" and "end_cycle=".
 272 The time is in seconds with decimals up to nanoseconds and cycle counts
 273 are unsigned integers with a 64 bits range. The cpu attribute is optional.
 274 </p>
 275
 276 -->
 277 </body>
 278 </html>