Commit | Line | Data |
---|---|---|
5cd1099a PMF |
1 | \input texinfo @c -*-texinfo-*- |
2 | @c %**start of header | |
3 | @setfilename ust.info | |
4 | @settitle LTTng Userspace Tracer (UST) Manual | |
5 | @c %**end of header | |
6 | ||
7 | @copying | |
8 | This manual is for program, version version. | |
9 | ||
10 | Copyright @copyright{} copyright-owner. | |
11 | ||
12 | @quotation | |
13 | Permission is granted to ... | |
14 | @end quotation | |
15 | @end copying | |
16 | ||
17 | @titlepage | |
18 | @title LTTng Userspace Tracer (UST) Manual | |
19 | @c @subtitle subtitle-if-any | |
20 | @c @subtitle second-subtitle | |
21 | @c @author author | |
22 | ||
23 | @c The following two commands | |
24 | @c start the copyright page. | |
25 | @c @page | |
26 | @c @vskip 0pt plus 1filll | |
27 | @c @insertcopying | |
28 | ||
29 | @c Published by ... | |
30 | @end titlepage | |
31 | ||
32 | @c So the toc is printed at the start. | |
33 | @contents | |
34 | ||
35 | @ifnottex | |
36 | @node Top | |
37 | @top LTTng Userspace Tracer | |
38 | ||
39 | This manual is for UST 0.1. | |
40 | @end ifnottex | |
41 | ||
42 | @menu | |
43 | * Overview:: | |
44 | * Installation:: | |
45 | * Quick start:: | |
46 | * Instrumenting an application:: | |
47 | * Recording a trace:: | |
48 | * Viewing traces:: | |
49 | * Performance:: | |
50 | @c * Copying:: Your rights and freedoms. | |
51 | @end menu | |
52 | ||
53 | @node Overview | |
54 | @chapter Overview | |
55 | ||
56 | @menu | |
79f9fac7 | 57 | * License:: |
5cd1099a PMF |
58 | * Supported platforms:: |
59 | @end menu | |
60 | ||
79f9fac7 PMF |
61 | @node License |
62 | @section License | |
63 | The LTTng Userspace Tracer is intended to be linkable to open source software | |
64 | as well as to proprietary applications. This was accomplished by licensing | |
65 | the code that needs to be linked to the traced program as @acronym{LGPL}. | |
66 | ||
675ac4ab PMF |
67 | Components licensed as LGPL v2.1: |
68 | @itemize @bullet | |
69 | @item libust | |
70 | @item libinterfork | |
71 | @item libustcomm | |
72 | @end itemize | |
73 | ||
74 | Components licensed as GPL v2: | |
75 | @itemize @bullet | |
76 | @item ustctl | |
77 | @item libustcmd | |
78 | @item ustd | |
79 | @end itemize | |
79f9fac7 | 80 | |
5cd1099a PMF |
81 | @node Supported platforms |
82 | @section Supported platforms | |
83 | ||
84 | UST can currently trace applications running on Linux, on the x86-32 and x86-64 architectures. | |
85 | ||
86 | @node Installation | |
87 | @chapter Installation | |
88 | ||
89 | The LTTng userspace tracer is a library and a set of userspace tools. | |
90 | ||
91 | The following packages are required: | |
92 | ||
93 | @itemize @bullet | |
94 | @item | |
95 | ust | |
96 | ||
97 | This contains the tracing library, the ustd daemon, trace control tools | |
98 | and other helper tools. | |
99 | ||
100 | Repository: http://git.dorsal.polymtl.ca | |
101 | ||
102 | @item | |
103 | libkcompat | |
104 | ||
105 | This is a library that contains a userspace port of some kernel APIs. | |
106 | ||
107 | Repository: http://git.dorsal.polymtl.ca | |
108 | ||
109 | @item | |
110 | liburcu | |
111 | ||
112 | This is the userspace read-copy update library by Mathieu Desnoyers. | |
113 | ||
114 | Available in Debian as package liburcu-dev. | |
115 | ||
116 | Home page: http://lttng.org/?q=node/18 | |
117 | ||
118 | @item | |
119 | LTTV | |
120 | ||
121 | LTTV is a graphical (and text) viewer for LTTng traces. | |
122 | ||
123 | Home page: http://lttng.org | |
124 | ||
125 | @end itemize | |
126 | ||
127 | Libkcompat and liburcu should be installed first. UST may then be compiled | |
128 | and installed. LTTV has no dependency on the other packages; it may therefore | |
129 | be installed on a system which does not have UST installed. | |
130 | ||
131 | Refer to the README in each of these packages for installation instructions. | |
132 | ||
133 | @c @menu | |
134 | @c @end menu | |
135 | ||
136 | @node Quick start | |
137 | @chapter Quick start | |
138 | ||
139 | First, instrument a program with a marker. | |
140 | ||
141 | @example | |
142 | @verbatim | |
143 | ||
144 | #include <ust/marker.h> | |
145 | ||
146 | int main(int argc, char **argv) | |
147 | { | |
148 | int v; | |
149 | char *st; | |
150 | ||
151 | /* ... set values of v and st ... */ | |
152 | ||
153 | /* a marker: */ | |
154 | trace_mark(ust, myevent, "firstarg %d secondarg %s", v, st); | |
155 | ||
156 | /* a marker without arguments: */ | |
157 | trace_mark(ust, myotherevent, MARK_NOARGS); | |
158 | ||
159 | return 0; | |
160 | } | |
161 | ||
162 | @end verbatim | |
163 | @end example | |
164 | ||
165 | Then compile it in the regular way, linking it with libust. For example: | |
166 | ||
167 | @example | |
168 | gcc -o foo -lust foo.c | |
169 | @end example | |
170 | ||
171 | Run the program with @command{usttrace}. The @command{usttrace} output says where the trace | |
172 | was written. | |
173 | ||
174 | @example | |
175 | usttrace ./foo | |
176 | @end example | |
177 | ||
178 | Finally, open the trace in LTTV. | |
179 | ||
180 | @example | |
181 | lttv-gui -t /path/to/trace | |
182 | @end example | |
183 | ||
184 | The trace can also be dumped as text in the console: | |
185 | ||
186 | @example | |
187 | lttv -m textDump -t /path/to/trace | |
188 | @end example | |
189 | ||
190 | @node Instrumenting an application | |
191 | @chapter Instrumenting an application | |
192 | ||
193 | In order to record a trace of events occurring in a application, the | |
194 | application must be instrumented. Instrumentation points resemble function | |
195 | calls. When the program reaches an instrumentation point, an event is | |
196 | generated. | |
197 | ||
198 | There are no limitations on the type of code that may be instrumented. | |
199 | Multi-threaded programs may be instrumented without problem. Signal handlers | |
200 | may be instrumented as well. | |
201 | ||
202 | There are two APIs to instrument programs: markers and tracepoints. Markers are | |
203 | quick to add and are usually used for temporary instrumentation. Tracepoints | |
204 | provide a way to instrument code more cleanly and are suited for permanent | |
205 | instrumentation. | |
206 | ||
207 | In addition to executable programs, shared libraries may also be instrumented | |
208 | with the methods described in this chapter. | |
209 | ||
210 | @menu | |
211 | * Markers:: | |
212 | * Tracepoints:: | |
213 | @end menu | |
214 | ||
215 | @node Markers | |
216 | @section Markers | |
217 | ||
218 | Adding a marker is simply a matter of insert one line in the program. | |
219 | ||
220 | @example | |
221 | @verbatim | |
222 | #include <ust/marker.h> | |
223 | ||
224 | int main(int argc, char **argv) | |
225 | { | |
226 | int v; | |
227 | char *st; | |
228 | ||
229 | /* ... set values of v and st ... */ | |
230 | ||
231 | /* a marker: */ | |
232 | trace_mark(main, myevent, "firstarg %d secondarg %s", v, st); | |
233 | ||
234 | /* a marker without arguments: */ | |
235 | trace_mark(main, myotherevent, MARK_NOARGS); | |
236 | ||
237 | return 0; | |
238 | } | |
239 | @end verbatim | |
240 | @end example | |
241 | ||
242 | The invocation of the trace_mark() macro requires at least 3 arguments. The | |
243 | first, here "main", is the name of the event category. It is also the name of | |
244 | the channel the event will go in. The second, here "myevent" is the name of the | |
245 | event. The third is a format string that announces the names and the types of | |
246 | the event arguments. Its format resembles that of a printf() format string; it | |
247 | is described thoroughly in Appendix x. | |
248 | ||
249 | A given Marker may appear more than once in the same program. Other Markers may | |
250 | have the same name and a different format string, although this might induce | |
251 | some confusion at analysis time. | |
252 | ||
253 | @node Tracepoints | |
254 | @section Tracepoints | |
255 | ||
256 | The Tracepoints API uses the Markers, but provides a higher-level abstraction. | |
257 | Whereas the markers API provides limited type checking, the Tracepoints API | |
258 | provides more thorough type checking and discharges from the need to insert | |
259 | format strings directly in the code and to have format strings appear more than | |
260 | once if a given marker is reused. | |
261 | ||
262 | @quotation Note Although this example uses @emph{mychannel} as the channel, the | |
263 | only channel name currently supported with early tracing is @strong{ust}. The | |
264 | @command{usttrace} tool always uses the early tracing mode. When using manual | |
265 | mode without early tracing, any channel name may be used. @end quotation | |
266 | ||
267 | A function instrumented with a tracepoint looks like this: | |
268 | ||
269 | @example | |
270 | @verbatim | |
271 | #include "tp.h" | |
272 | ||
273 | void function() | |
274 | { | |
275 | int v; | |
276 | char *st; | |
277 | ||
278 | /* ... set values of v and st ... */ | |
279 | ||
280 | /* a tracepoint: */ | |
281 | trace_mychannel_myevent(v, st); | |
282 | } | |
283 | @end verbatim | |
284 | @end example | |
285 | ||
286 | Another file, here tp.h, contains declarations for the tracepoint. | |
287 | ||
288 | @example | |
289 | @verbatim | |
290 | #include <ust/tracepoint.h> | |
291 | ||
292 | DECLARE_TRACE(mychannel_myevent, TPPROTO(int v, char *st), | |
293 | TPARGS(v, st)); | |
294 | @end verbatim | |
295 | @end example | |
296 | ||
297 | A third file, here tp.c, contains definitions for the tracepoint. | |
298 | ||
299 | @example | |
300 | @verbatim | |
301 | #include <ust/marker.h> | |
302 | #include "tp.h" | |
303 | ||
304 | DEFINE_TRACE(mychannel_myevent); | |
305 | ||
306 | void mychannel_myevent_probe(int v, char *st) | |
307 | { | |
308 | trace_mark(mychannel, myevent, "v %d st %s", v, st); | |
309 | } | |
310 | ||
311 | static void __attribute__((constructor)) init() | |
312 | { | |
313 | register_trace_mychannel_myevent(mychannel_myevent_probe); | |
314 | } | |
315 | @end verbatim | |
316 | @end example | |
317 | ||
318 | Here, tp.h and tp.c could contain declarations and definitions for other | |
319 | tracepoints. The constructor would contain other register_* calls. | |
320 | ||
321 | @node Recording a trace | |
322 | @chapter Recording a trace | |
323 | ||
324 | @menu | |
325 | * Using @command{usttrace}:: | |
326 | * Setting up the recording manually:: | |
327 | * Using early tracing:: | |
328 | * Crash recovery:: | |
329 | * Tracing across @code{fork()} and @code{clone()}:: | |
330 | * Tracing programs and libraries that were not linked to libust:: | |
331 | @end menu | |
332 | ||
333 | @node Using @command{usttrace} | |
334 | @section Using @command{usttrace} | |
335 | ||
336 | The simplest way to record a trace is to use the @command{usttrace} script. An | |
337 | example is given in the quickstart above. | |
338 | ||
339 | The @command{usttrace} script automatically: | |
340 | @itemize @bullet | |
341 | @item creates a daemon | |
342 | @item enables all markers | |
343 | @item runs the command specified on the command line | |
344 | @item after the command ends, prints the location where the trace was saved | |
345 | @end itemize | |
346 | ||
347 | Each subdirectory of the save location contains the trace of one process that | |
348 | was generated by the command. The name of a subdirectory consists in the the PID | |
349 | of the process, followed by the timestamp of its creation. | |
350 | ||
351 | The save location also contains logs of the tracing. | |
352 | ||
353 | When using @command{usttrace}, the early tracing is always active, which means | |
354 | that the tracing is guaranteed to be started by the time the process enters its | |
355 | main() function. | |
356 | ||
357 | Several @command{usttrace}'s may be run simultaneously without risk of | |
358 | conflict. This facilitates the use of the tracer by idependent users on a | |
359 | system. Each instance of @command{usttrace} starts its own daemon which | |
360 | collects the events of the processes it creates. | |
361 | ||
362 | @node Setting up the recording manually | |
363 | @section Setting up the recording manually | |
364 | ||
365 | Instead of using @command{usttrace}, a trace may be recorded on an already | |
366 | running process. | |
367 | ||
368 | First the daemon must be started. | |
369 | ||
370 | @example | |
371 | @verbatim | |
372 | # Make sure the directory for the communication sockets exists. | |
373 | $ mkdir /tmp/ustsocks | |
374 | ||
375 | # Make sure the directory where ustd will write the trace exists. | |
376 | $ mkdir /tmp/trace | |
377 | ||
378 | # Start the daemon | |
379 | $ ustd | |
380 | ||
381 | # We assume the program we want to trace is already running and that | |
382 | # it has pid 1234. | |
383 | ||
384 | # List the available markers | |
385 | $ ustctl --list-markers 1234 | |
386 | # A column indicates 0 for an inactive marker and 1 for an active marker. | |
387 | ||
388 | # Enable a marker | |
e61c1b29 | 389 | $ ustctl --enable-marker ust/mymark 1234 |
5cd1099a PMF |
390 | |
391 | # Create a trace | |
392 | $ ustctl --create-trace 1234 | |
393 | ||
394 | # Start tracing | |
395 | $ ustctl --start-trace 1234 | |
396 | ||
397 | # Do things... | |
398 | ||
399 | # Stop tracing | |
400 | $ ustctl --stop-trace 1234 | |
1af1f1db PMF |
401 | |
402 | # Destroy the trace | |
403 | $ ustctl --destroy-trace 1234 | |
5cd1099a PMF |
404 | @end verbatim |
405 | @end example | |
406 | ||
407 | @node Using early tracing | |
408 | @section Using early tracing | |
409 | ||
410 | Early tracing consists in starting the tracing as early as possible in the | |
411 | program, so no events are lost between program start and the point where the | |
412 | command to start the tracing is given. When using early tracing, it is | |
413 | guaranteed that by the time the traced program enters its @code{main()} | |
414 | function, the tracing will be started. | |
415 | ||
416 | When using @command{usttrace}, the early tracing is always active. | |
417 | ||
418 | When using the manual mode (@command{ustctl}), early tracing is enabled using | |
419 | environment variables. Setting @env{UST_TRACE} to @code{1}, enables early | |
420 | tracing, while setting @env{UST_AUTOPROBE} to @code{1} enables all markers | |
421 | automatically. | |
422 | ||
423 | ||
424 | @node Crash recovery | |
425 | @section Crash recovery | |
426 | ||
427 | When a process being traced crashes, the daemon is able to recover all the | |
428 | events in its buffers that were successfully commited. This is possible because | |
429 | the buffers are in a shared memory segment which remains available to the | |
430 | daemon even after the termination of the traced process. | |
431 | ||
432 | @node Tracing across @code{fork()} and @code{clone()} | |
433 | @section Tracing across @code{fork()} and @code{clone()} | |
434 | ||
2669948d PMF |
435 | Tracing across @code{clone()} when the @code{CLONE_VM} flag is specified is |
436 | supported without any particular action. | |
5cd1099a PMF |
437 | |
438 | When @code{clone()} is called without @code{CLONE_VM} or @code{fork()} is | |
439 | called, a new address space is created and the tracer must be notified to | |
2669948d PMF |
440 | create new buffers for it. |
441 | ||
442 | This can be done automatically, by @env{LD_PRELOAD}'ing @file{libinterfork.so}. | |
443 | This library intercepts calls to @code{fork()} and informs the tracer it is | |
444 | being called. When using @command{usttrace}, this is accomplied by specifying | |
445 | the @option{-f} command line argument. | |
446 | ||
447 | Alternatively, the program can call @code{ust_before_fork()} before calling | |
448 | @code{fork()} or @code{clone()} with @code{CLONE_VM}. After the call, | |
449 | @code{ust_after_fork_parent()} must be called in the parent process and | |
450 | @code{ust_after_fork_child()} must be called in the child process. | |
5cd1099a | 451 | |
5cd1099a PMF |
452 | |
453 | @node Tracing programs and libraries that were not linked to libust | |
454 | @section Tracing programs and libraries that were not linked to libust | |
455 | ||
2669948d PMF |
456 | Some programs need to be traced even though they were not linked to libust |
457 | either because they were not instrumented or because it was not practical. | |
458 | ||
459 | An executable that is not instrumented can still yield interesting traces when | |
460 | at least one of its dynamic libraries is instrumented. It is also possible to | |
461 | trace certain function calls by intercepting them with a specially crafted | |
462 | library that is linked with @env{LD_PRELOAD} at program start. | |
463 | ||
464 | In any case, a program that was not linked to libust at compile time must be | |
465 | linked to it at run time with @env{LD_PRELOAD}. This can be accomplished with | |
466 | @command{usttrace}'s @option{-l} option. It can also be done by setting the | |
467 | @env{LD_PRELOAD} environment variable on the command line. For example: | |
468 | ||
469 | @example | |
470 | @verbatim | |
471 | # Run ls with usttrace, LD_PRELOAD'ing libust | |
472 | # (assuming one of the libraries used by ls is instrumented). | |
473 | $ usttrace -l ls | |
474 | ||
475 | # Run ls, manually adding the LD_PRELOAD. | |
476 | $ LD_PRELOAD=/usr/local/lib/libust.so.0 ls | |
477 | @end verbatim | |
478 | @end example | |
479 | ||
5cd1099a PMF |
480 | |
481 | @node Performance | |
482 | @chapter Performance | |
483 | ||
484 | Todo. | |
485 | ||
486 | @node Viewing traces | |
487 | @chapter Viewing traces | |
488 | ||
2669948d PMF |
489 | Traces may be viewed with LTTV. An example of command for launching LTTV is |
490 | given in the quickstart. | |
491 | ||
492 | @menu | |
493 | * Viewing multiple traces:: | |
494 | * Combined kernel-userspace tracing:: | |
495 | @end menu | |
496 | ||
497 | @node Viewing multiple traces | |
498 | @section Viewing multiple traces | |
499 | ||
500 | When tracing multi-process applications or several applications simultaneously, | |
501 | more than one trace will be obtained. LTTV can open and display all these | |
502 | traces simultaneously. | |
503 | ||
504 | @node Combined kernel-userspace tracing | |
505 | @section Combined kernel-userspace tracing | |
506 | ||
507 | In addition to multiple userspace traces, LTTV can open a kernel trace recorded | |
508 | with the LTTng kernel tracer. This provides events that enable the rendering of | |
509 | the Control Flow View and the Resource View. | |
510 | ||
511 | When doing so, it is necessary to use the same time source for the kernel | |
512 | tracer as well as the userspace tracer. Currently, the recommended method is to | |
513 | use the timestamp counter for both. The TSC can however only be used on architectures | |
514 | where it is synchronized across cores. | |
515 | ||
5cd1099a | 516 | @bye |