Commit | Line | Data |
---|---|---|
5cd1099a PMF |
1 | \input texinfo @c -*-texinfo-*- |
2 | @c %**start of header | |
3 | @setfilename ust.info | |
4 | @settitle LTTng Userspace Tracer (UST) Manual | |
5 | @c %**end of header | |
6 | ||
7 | @copying | |
8 | This manual is for program, version version. | |
9 | ||
10 | Copyright @copyright{} copyright-owner. | |
11 | ||
12 | @quotation | |
13 | Permission is granted to ... | |
14 | @end quotation | |
15 | @end copying | |
16 | ||
17 | @titlepage | |
18 | @title LTTng Userspace Tracer (UST) Manual | |
19 | @c @subtitle subtitle-if-any | |
20 | @c @subtitle second-subtitle | |
21 | @c @author author | |
22 | ||
23 | @c The following two commands | |
24 | @c start the copyright page. | |
25 | @c @page | |
26 | @c @vskip 0pt plus 1filll | |
27 | @c @insertcopying | |
28 | ||
29 | @c Published by ... | |
30 | @end titlepage | |
31 | ||
32 | @c So the toc is printed at the start. | |
33 | @contents | |
34 | ||
35 | @ifnottex | |
36 | @node Top | |
37 | @top LTTng Userspace Tracer | |
38 | ||
38ab1298 | 39 | This manual is for UST 0.14. |
5cd1099a PMF |
40 | @end ifnottex |
41 | ||
42 | @menu | |
43 | * Overview:: | |
44 | * Installation:: | |
45 | * Quick start:: | |
46 | * Instrumenting an application:: | |
47 | * Recording a trace:: | |
48 | * Viewing traces:: | |
49 | * Performance:: | |
bb0496b2 | 50 | * Resource Usage:: |
4bea9b39 PMF |
51 | * List of environment variables detected by libust:: |
52 | * GDB integration:: | |
5cd1099a PMF |
53 | @c * Copying:: Your rights and freedoms. |
54 | @end menu | |
55 | ||
56 | @node Overview | |
57 | @chapter Overview | |
58 | ||
59 | @menu | |
70e55dbe | 60 | * What is UST?:: |
79f9fac7 | 61 | * License:: |
5cd1099a PMF |
62 | * Supported platforms:: |
63 | @end menu | |
64 | ||
70e55dbe PMF |
65 | @node What is UST? |
66 | @section What is UST? | |
67 | ||
68 | The LTTng Userspace Tracer (UST) is a library accompanied by a set of tools to | |
69 | trace userspace code. | |
70 | ||
71 | Code may be instrumented with either markers or tracepoints. A highly efficient | |
72 | lockless tracer records these events to a trace buffers. These buffers are reaped | |
73 | by a deamon which writes trace data to disk. | |
74 | ||
75 | High performance is achieved by the use of lockless buffering algorithms, RCU and | |
76 | per-cpu buffers. In addition, special care is taken to minize cache impact. | |
77 | ||
79f9fac7 PMF |
78 | @node License |
79 | @section License | |
80 | The LTTng Userspace Tracer is intended to be linkable to open source software | |
81 | as well as to proprietary applications. This was accomplished by licensing | |
82 | the code that needs to be linked to the traced program as @acronym{LGPL}. | |
83 | ||
675ac4ab PMF |
84 | Components licensed as LGPL v2.1: |
85 | @itemize @bullet | |
86 | @item libust | |
87 | @item libinterfork | |
88 | @item libustcomm | |
89 | @end itemize | |
90 | ||
91 | Components licensed as GPL v2: | |
92 | @itemize @bullet | |
93 | @item ustctl | |
2298f329 | 94 | @item libustctl |
9dc7b7ff | 95 | @item ust-consumerd |
675ac4ab | 96 | @end itemize |
79f9fac7 | 97 | |
5cd1099a PMF |
98 | @node Supported platforms |
99 | @section Supported platforms | |
100 | ||
bb0496b2 PMF |
101 | UST can currently trace applications running on Linux, on the x86-32, x86-64 |
102 | and PowerPC 32 architectures. | |
5cd1099a PMF |
103 | |
104 | @node Installation | |
105 | @chapter Installation | |
106 | ||
107 | The LTTng userspace tracer is a library and a set of userspace tools. | |
108 | ||
109 | The following packages are required: | |
110 | ||
111 | @itemize @bullet | |
112 | @item | |
113 | ust | |
114 | ||
9dc7b7ff | 115 | This contains the tracing library, the ust-consumerd daemon, trace control tools |
5cd1099a PMF |
116 | and other helper tools. |
117 | ||
38ab1298 | 118 | Repository: @url{http://lttng.org/ust} |
5cd1099a PMF |
119 | |
120 | @item | |
121 | liburcu | |
122 | ||
123 | This is the userspace read-copy update library by Mathieu Desnoyers. | |
124 | ||
125 | Available in Debian as package liburcu-dev. | |
126 | ||
72d022d6 | 127 | Home page: @url{http://lttng.org/urcu} |
5cd1099a PMF |
128 | |
129 | @item | |
130 | LTTV | |
131 | ||
132 | LTTV is a graphical (and text) viewer for LTTng traces. | |
133 | ||
72d022d6 | 134 | Home page: @url{http://lttng.org} |
5cd1099a PMF |
135 | |
136 | @end itemize | |
137 | ||
72d022d6 PMF |
138 | Liburcu should be installed first. UST may then be compiled and installed. LTTV |
139 | has no dependency on the other packages; it may therefore be installed on a | |
140 | system which does not have UST installed. | |
5cd1099a PMF |
141 | |
142 | Refer to the README in each of these packages for installation instructions. | |
143 | ||
144 | @c @menu | |
145 | @c @end menu | |
146 | ||
147 | @node Quick start | |
148 | @chapter Quick start | |
149 | ||
150 | First, instrument a program with a marker. | |
151 | ||
152 | @example | |
153 | @verbatim | |
154 | ||
155 | #include <ust/marker.h> | |
156 | ||
157 | int main(int argc, char **argv) | |
158 | { | |
159 | int v; | |
160 | char *st; | |
161 | ||
162 | /* ... set values of v and st ... */ | |
163 | ||
164 | /* a marker: */ | |
e367f87d | 165 | ust_marker(myevent, "firstarg %d secondarg %s", v, st); |
5cd1099a PMF |
166 | |
167 | /* a marker without arguments: */ | |
e367f87d | 168 | ust_marker(myotherevent, MARK_NOARGS); |
5cd1099a PMF |
169 | |
170 | return 0; | |
171 | } | |
172 | ||
173 | @end verbatim | |
174 | @end example | |
175 | ||
176 | Then compile it in the regular way, linking it with libust. For example: | |
177 | ||
178 | @example | |
179 | gcc -o foo -lust foo.c | |
180 | @end example | |
181 | ||
182 | Run the program with @command{usttrace}. The @command{usttrace} output says where the trace | |
183 | was written. | |
184 | ||
185 | @example | |
186 | usttrace ./foo | |
187 | @end example | |
188 | ||
189 | Finally, open the trace in LTTV. | |
190 | ||
191 | @example | |
192 | lttv-gui -t /path/to/trace | |
193 | @end example | |
194 | ||
195 | The trace can also be dumped as text in the console: | |
196 | ||
197 | @example | |
198 | lttv -m textDump -t /path/to/trace | |
199 | @end example | |
200 | ||
201 | @node Instrumenting an application | |
202 | @chapter Instrumenting an application | |
203 | ||
204 | In order to record a trace of events occurring in a application, the | |
205 | application must be instrumented. Instrumentation points resemble function | |
206 | calls. When the program reaches an instrumentation point, an event is | |
207 | generated. | |
208 | ||
209 | There are no limitations on the type of code that may be instrumented. | |
210 | Multi-threaded programs may be instrumented without problem. Signal handlers | |
211 | may be instrumented as well. | |
212 | ||
213 | There are two APIs to instrument programs: markers and tracepoints. Markers are | |
214 | quick to add and are usually used for temporary instrumentation. Tracepoints | |
215 | provide a way to instrument code more cleanly and are suited for permanent | |
216 | instrumentation. | |
217 | ||
218 | In addition to executable programs, shared libraries may also be instrumented | |
219 | with the methods described in this chapter. | |
220 | ||
221 | @menu | |
222 | * Markers:: | |
223 | * Tracepoints:: | |
224 | @end menu | |
225 | ||
226 | @node Markers | |
227 | @section Markers | |
228 | ||
70e55dbe | 229 | Adding a marker is simply a matter of inserting one line in the program. |
5cd1099a PMF |
230 | |
231 | @example | |
232 | @verbatim | |
233 | #include <ust/marker.h> | |
234 | ||
235 | int main(int argc, char **argv) | |
236 | { | |
237 | int v; | |
238 | char *st; | |
239 | ||
240 | /* ... set values of v and st ... */ | |
241 | ||
242 | /* a marker: */ | |
e367f87d | 243 | ust_marker(myevent, "firstarg %d secondarg %s", v, st); |
5cd1099a | 244 | |
70e55dbe | 245 | /* another marker without arguments: */ |
e367f87d | 246 | ust_marker(myotherevent, MARK_NOARGS); |
5cd1099a PMF |
247 | |
248 | return 0; | |
249 | } | |
250 | @end verbatim | |
251 | @end example | |
252 | ||
e367f87d MD |
253 | The invocation of the ust_marker() macro requires at least 2 arguments. The |
254 | first, "myevent", is the name of the event. The second is a format string | |
255 | that announces the names and the types of the event arguments. Its | |
256 | format resembles that of a printf() format string; it is described | |
257 | thoroughly in Appendix x. | |
5cd1099a PMF |
258 | |
259 | A given Marker may appear more than once in the same program. Other Markers may | |
260 | have the same name and a different format string, although this might induce | |
261 | some confusion at analysis time. | |
262 | ||
263 | @node Tracepoints | |
264 | @section Tracepoints | |
265 | ||
38ab1298 MD |
266 | The Tracepoint API is superseded by TRACEPOINT_EVENT() (which uses the |
267 | Tracepoints internally). At this stage, TRACEPOINT_EVENT() is work in | |
268 | progress. While we complete this API, please use the ust_marker() API | |
269 | provided by ust/marker.h. | |
5cd1099a PMF |
270 | |
271 | @node Recording a trace | |
272 | @chapter Recording a trace | |
273 | ||
274 | @menu | |
275 | * Using @command{usttrace}:: | |
276 | * Setting up the recording manually:: | |
277 | * Using early tracing:: | |
278 | * Crash recovery:: | |
279 | * Tracing across @code{fork()} and @code{clone()}:: | |
280 | * Tracing programs and libraries that were not linked to libust:: | |
281 | @end menu | |
282 | ||
283 | @node Using @command{usttrace} | |
284 | @section Using @command{usttrace} | |
285 | ||
286 | The simplest way to record a trace is to use the @command{usttrace} script. An | |
287 | example is given in the quickstart above. | |
288 | ||
289 | The @command{usttrace} script automatically: | |
290 | @itemize @bullet | |
291 | @item creates a daemon | |
292 | @item enables all markers | |
293 | @item runs the command specified on the command line | |
294 | @item after the command ends, prints the location where the trace was saved | |
295 | @end itemize | |
296 | ||
297 | Each subdirectory of the save location contains the trace of one process that | |
298 | was generated by the command. The name of a subdirectory consists in the the PID | |
299 | of the process, followed by the timestamp of its creation. | |
300 | ||
301 | The save location also contains logs of the tracing. | |
302 | ||
303 | When using @command{usttrace}, the early tracing is always active, which means | |
304 | that the tracing is guaranteed to be started by the time the process enters its | |
4bea9b39 | 305 | @code{main()} function. |
5cd1099a PMF |
306 | |
307 | Several @command{usttrace}'s may be run simultaneously without risk of | |
308 | conflict. This facilitates the use of the tracer by idependent users on a | |
309 | system. Each instance of @command{usttrace} starts its own daemon which | |
310 | collects the events of the processes it creates. | |
311 | ||
312 | @node Setting up the recording manually | |
313 | @section Setting up the recording manually | |
314 | ||
315 | Instead of using @command{usttrace}, a trace may be recorded on an already | |
316 | running process. | |
317 | ||
318 | First the daemon must be started. | |
319 | ||
320 | @example | |
321 | @verbatim | |
322 | # Make sure the directory for the communication sockets exists. | |
323 | $ mkdir /tmp/ustsocks | |
324 | ||
9dc7b7ff | 325 | # Make sure the directory where ust-consumerd will write the trace exists. |
5cd1099a PMF |
326 | $ mkdir /tmp/trace |
327 | ||
328 | # Start the daemon | |
9dc7b7ff | 329 | $ ust-consumerd |
5cd1099a PMF |
330 | |
331 | # We assume the program we want to trace is already running and that | |
332 | # it has pid 1234. | |
333 | ||
334 | # List the available markers | |
eb78cc5f | 335 | $ ustctl list-markers 1234 |
5cd1099a PMF |
336 | # A column indicates 0 for an inactive marker and 1 for an active marker. |
337 | ||
338 | # Enable a marker | |
eb78cc5f | 339 | $ ustctl enable-marker 1234 auto ust/mymark |
5cd1099a PMF |
340 | |
341 | # Create a trace | |
eb78cc5f | 342 | $ ustctl create-trace 1234 auto |
5cd1099a PMF |
343 | |
344 | # Start tracing | |
eb78cc5f | 345 | $ ustctl start-trace 1234 auto |
5cd1099a PMF |
346 | |
347 | # Do things... | |
348 | ||
349 | # Stop tracing | |
eb78cc5f | 350 | $ ustctl stop-trace 1234 auto |
1af1f1db PMF |
351 | |
352 | # Destroy the trace | |
eb78cc5f | 353 | $ ustctl destroy-trace 1234 auto |
5cd1099a PMF |
354 | @end verbatim |
355 | @end example | |
356 | ||
c3947c25 PMF |
357 | For more information about the manual mode, see the ustctl(1) man page. |
358 | ||
5cd1099a PMF |
359 | @node Using early tracing |
360 | @section Using early tracing | |
361 | ||
362 | Early tracing consists in starting the tracing as early as possible in the | |
363 | program, so no events are lost between program start and the point where the | |
364 | command to start the tracing is given. When using early tracing, it is | |
365 | guaranteed that by the time the traced program enters its @code{main()} | |
366 | function, the tracing will be started. | |
367 | ||
368 | When using @command{usttrace}, the early tracing is always active. | |
369 | ||
370 | When using the manual mode (@command{ustctl}), early tracing is enabled using | |
371 | environment variables. Setting @env{UST_TRACE} to @code{1}, enables early | |
372 | tracing, while setting @env{UST_AUTOPROBE} to @code{1} enables all markers | |
373 | automatically. | |
374 | ||
375 | ||
376 | @node Crash recovery | |
377 | @section Crash recovery | |
378 | ||
379 | When a process being traced crashes, the daemon is able to recover all the | |
380 | events in its buffers that were successfully commited. This is possible because | |
381 | the buffers are in a shared memory segment which remains available to the | |
382 | daemon even after the termination of the traced process. | |
383 | ||
384 | @node Tracing across @code{fork()} and @code{clone()} | |
385 | @section Tracing across @code{fork()} and @code{clone()} | |
386 | ||
2669948d PMF |
387 | Tracing across @code{clone()} when the @code{CLONE_VM} flag is specified is |
388 | supported without any particular action. | |
5cd1099a PMF |
389 | |
390 | When @code{clone()} is called without @code{CLONE_VM} or @code{fork()} is | |
391 | called, a new address space is created and the tracer must be notified to | |
2669948d PMF |
392 | create new buffers for it. |
393 | ||
394 | This can be done automatically, by @env{LD_PRELOAD}'ing @file{libinterfork.so}. | |
395 | This library intercepts calls to @code{fork()} and informs the tracer it is | |
396 | being called. When using @command{usttrace}, this is accomplied by specifying | |
397 | the @option{-f} command line argument. | |
398 | ||
399 | Alternatively, the program can call @code{ust_before_fork()} before calling | |
400 | @code{fork()} or @code{clone()} with @code{CLONE_VM}. After the call, | |
401 | @code{ust_after_fork_parent()} must be called in the parent process and | |
402 | @code{ust_after_fork_child()} must be called in the child process. | |
5cd1099a | 403 | |
5cd1099a PMF |
404 | |
405 | @node Tracing programs and libraries that were not linked to libust | |
406 | @section Tracing programs and libraries that were not linked to libust | |
407 | ||
2669948d PMF |
408 | Some programs need to be traced even though they were not linked to libust |
409 | either because they were not instrumented or because it was not practical. | |
410 | ||
411 | An executable that is not instrumented can still yield interesting traces when | |
412 | at least one of its dynamic libraries is instrumented. It is also possible to | |
413 | trace certain function calls by intercepting them with a specially crafted | |
414 | library that is linked with @env{LD_PRELOAD} at program start. | |
415 | ||
416 | In any case, a program that was not linked to libust at compile time must be | |
417 | linked to it at run time with @env{LD_PRELOAD}. This can be accomplished with | |
418 | @command{usttrace}'s @option{-l} option. It can also be done by setting the | |
419 | @env{LD_PRELOAD} environment variable on the command line. For example: | |
420 | ||
421 | @example | |
422 | @verbatim | |
423 | # Run ls with usttrace, LD_PRELOAD'ing libust | |
424 | # (assuming one of the libraries used by ls is instrumented). | |
425 | $ usttrace -l ls | |
426 | ||
427 | # Run ls, manually adding the LD_PRELOAD. | |
428 | $ LD_PRELOAD=/usr/local/lib/libust.so.0 ls | |
429 | @end verbatim | |
430 | @end example | |
431 | ||
5cd1099a PMF |
432 | |
433 | @node Performance | |
434 | @chapter Performance | |
435 | ||
436 | Todo. | |
437 | ||
438 | @node Viewing traces | |
439 | @chapter Viewing traces | |
440 | ||
2669948d PMF |
441 | Traces may be viewed with LTTV. An example of command for launching LTTV is |
442 | given in the quickstart. | |
443 | ||
444 | @menu | |
445 | * Viewing multiple traces:: | |
446 | * Combined kernel-userspace tracing:: | |
447 | @end menu | |
448 | ||
449 | @node Viewing multiple traces | |
450 | @section Viewing multiple traces | |
451 | ||
452 | When tracing multi-process applications or several applications simultaneously, | |
453 | more than one trace will be obtained. LTTV can open and display all these | |
454 | traces simultaneously. | |
455 | ||
456 | @node Combined kernel-userspace tracing | |
457 | @section Combined kernel-userspace tracing | |
458 | ||
459 | In addition to multiple userspace traces, LTTV can open a kernel trace recorded | |
460 | with the LTTng kernel tracer. This provides events that enable the rendering of | |
461 | the Control Flow View and the Resource View. | |
462 | ||
463 | When doing so, it is necessary to use the same time source for the kernel | |
464 | tracer as well as the userspace tracer. Currently, the recommended method is to | |
465 | use the timestamp counter for both. The TSC can however only be used on architectures | |
466 | where it is synchronized across cores. | |
467 | ||
bb0496b2 PMF |
468 | @node Resource Usage |
469 | @chapter Resource Usage | |
470 | ||
471 | The purpose of this section is to give an overview of the resource usage of libust. For | |
472 | a developer, knowing this can be important: because libust is linked with applications, it | |
473 | needs to share some resources with it. Some applications may make some assumptions that are in | |
474 | conflict with libust's usage of resources. | |
475 | ||
476 | In practice however, libust is designed to be transparent and is compatible | |
477 | with the vast majority of applications. This means no changes are required in | |
478 | the application (or library) being linked to libust. | |
479 | ||
4bea9b39 PMF |
480 | Libust is initialized by a constructor, which by definition runs before the |
481 | @code{main()} function of the application starts. This constructor creates a | |
482 | thread called the @emph{listener thread}. The listener thread initializes a | |
9dc7b7ff | 483 | named socket and waits for connections for ust-consumerd or ustctl. |
bb0496b2 PMF |
484 | |
485 | Libust-specific code may: | |
486 | @itemize @bullet | |
487 | @item use @code{malloc()} and @code{free()} | |
488 | @item map shared memory segment in the process adress space | |
489 | @item intercept some library calls, specifically @code{fork()} and @code{clone()} | |
490 | @item do interprocess communication with the daemon or ustctl | |
491 | @item create and open named sockets | |
492 | ||
493 | @end itemize | |
494 | ||
495 | It will not: | |
496 | @itemize @bullet | |
497 | @item handle any signal (all signals are blocked in the listener thread) | |
498 | @item change any process-wide setting that could confuse the application | |
499 | @end itemize | |
500 | ||
4bea9b39 PMF |
501 | @node List of environment variables detected by libust |
502 | @appendix List of environment variables detected by libust | |
503 | ||
504 | The behavior of tracing can be influenced by setting special environment | |
505 | variables in the environment of the traced application. This section | |
506 | describes these variables. | |
507 | ||
508 | @itemize @bullet | |
509 | ||
510 | @item | |
511 | @env{UST_TRACE} | |
512 | ||
513 | If set to 1, start tracing as soon as the program starts. Tracing is | |
514 | guaranteed to be started by the time the @code{main()} function starts. | |
515 | ||
516 | @item | |
517 | @env{UST_AUTOPROBE} | |
518 | ||
519 | If set to @code{1}, enable all markers by the time the @code{main()} function starts. | |
520 | ||
521 | @item | |
522 | @env{UST_AUTOCOLLECT} | |
523 | ||
524 | If set to @code{0}, disable notification of daemon on trace start. Useful for | |
525 | performance tests. | |
526 | ||
527 | @item | |
528 | @env{UST_OVERWRITE} | |
529 | ||
530 | If set to @code{1}, enable overwriting of buffers on overrun. | |
531 | ||
532 | @item | |
533 | @env{UST_SUBBUF_NUM} | |
534 | ||
535 | If set, defines the default number of subbuffers per buffer. | |
536 | ||
537 | @item | |
538 | @env{UST_SUBBUF_SIZE} | |
539 | ||
540 | If set, defines the default size of subbuffers, in bytes. | |
541 | ||
542 | @end itemize | |
543 | ||
544 | @node GDB integration | |
545 | @appendix GDB integration | |
546 | ||
547 | GDB, the GNU Debugger, can use UST markers as GDB tracepoints (note GDB has its | |
548 | own concept of tracepoint). This feature is called GDB Static Tracepoints. When | |
549 | a GDB tracepoint is hit, GDB collects the marker arguments, as well as the | |
fe566790 | 550 | state of the registers. Support for GDB is currently work in progress. |
bb0496b2 | 551 | |
5cd1099a | 552 | @bye |