1 #LyX 1.6.1 created this file. For more info see http://www.lyx.org/
6 \use_default_options true
11 \font_typewriter default
12 \font_default_family default
19 \paperfontsize default
27 \paperorientation portrait
30 \paragraph_separation indent
32 \quotes_language english
35 \paperpagestyle default
36 \tracking_changes false
45 LTTng Userspace Tracer Manual
53 What is the LTTng Userspace Tracer?
56 \begin_layout Subsection
60 \begin_layout Standard
61 The LTTng Userspace Tracer (UST) is a tracing system for userspace applications.
62 It is designed to trace efficiently applications that produce events at
64 UST is derived from LTTng, a tracer for the Linux kernel.
65 It has the same trace format.
68 \begin_layout Standard
69 Users may choose at runtime and even at trace time what instrumentation
71 Custom probes may be used to trace events conditionally.
74 \begin_layout Subsection
79 Arbitrary number of channels
83 One buffer per process (multiple threads share the same buffer)
87 Early process tracing (from the beginning of the main() function)
91 Support for custom probes
94 \begin_layout Standard
99 Support for dynamic instrumentation
102 \begin_layout Itemize
103 Per thread or per CPU buffers
106 \begin_layout Standard
107 Complementary systems:
110 \begin_layout Itemize
111 A extension to gdb tracepoints that will allow the tracing of applications
112 with dynamic instrumentation is under development.
115 \begin_layout Subsection
119 \begin_layout Section
123 \begin_layout Subsection
127 \begin_layout Standard
128 To compile UST, you need:
131 \begin_layout Itemize
135 \begin_layout Itemize
139 \begin_layout Itemize
143 \begin_layout Subsection
147 \begin_layout Standard
148 Edit kcompat.h and uncomment the architecture of the target system.
151 \begin_layout Subsection
155 \begin_layout Standard
156 If the code was obtained through git, first generate a Makefile by running
160 \begin_layout Standard
161 Then, run ./configure, specifying the directories of liburcu and libkcompat.
165 \begin_layout Standard
166 \begin_inset listings
170 \begin_layout Plain Layout
172 ./configure --with-urcu=/home/pmf/liburcu-pmf --with-kcompat=/home/pmf/libkcompat
180 \begin_layout Subsection
184 \begin_layout Standard
185 Go to the liburcu directory and run:
188 \begin_layout Standard
189 \begin_inset listings
193 \begin_layout Plain Layout
203 \begin_layout Subsection
207 \begin_layout Standard
208 Go to the ust directory and run:
211 \begin_layout Standard
212 \begin_inset listings
216 \begin_layout Plain Layout
226 \begin_layout Section
227 Instrumenting an Application
230 \begin_layout Standard
231 In order to record a trace of events occurring in a application, the application
232 must be instrumented.
233 Instrumentation points resemble function calls.
234 When the program reaches an instrumentation point, an event is generated.
237 \begin_layout Standard
238 There are no limitations on the type of code that may be instrumented.
239 Multi-threaded programs may be instrumented without problem.
240 Signal handlers may be instrumented as well.
243 \begin_layout Standard
244 There are two APIs to instrument programs: markers and tracepoints.
245 Markers are quick to add and are usually used for temporary instrumentation.
246 Tracepoints provide a way to instrument code more cleanly and are suited
247 for permanent instrumentation.
250 \begin_layout Subsection
254 \begin_layout Standard
255 Markers were ported from the Linux Kernel Markers implementation.
256 Therefore, their usage is almost identical.
259 \begin_layout Subsubsection
263 \begin_layout Standard
264 Adding a marker is simply a matter of insert one line in the program.
267 \begin_layout Standard
268 \begin_inset listings
272 \begin_layout Plain Layout
277 \begin_layout Plain Layout
279 int main(int argc, char **argv)
282 \begin_layout Plain Layout
287 \begin_layout Plain Layout
292 \begin_layout Plain Layout
297 \begin_layout Plain Layout
301 \begin_layout Plain Layout
304 set values of v and st ...
308 \begin_layout Plain Layout
312 \begin_layout Plain Layout
317 \begin_layout Plain Layout
319 trace_mark(main, myevent,
320 \begin_inset Quotes eld
323 firstarg %d secondarg %s
324 \begin_inset Quotes erd
330 \begin_layout Plain Layout
334 \begin_layout Plain Layout
336 /* a marker without arguments: */
339 \begin_layout Plain Layout
341 trace_mark(main, myotherevent, MARK_NOARGS);
344 \begin_layout Plain Layout
348 \begin_layout Plain Layout
353 \begin_layout Plain Layout
363 \begin_layout Standard
364 The invocation of the trace_mark() macro requires at least 3 arguments.
366 \begin_inset Quotes eld
370 \begin_inset Quotes erd
373 , is the name of the event category.
374 It is also the name of the channel the event will go in.
376 \begin_inset Quotes eld
380 \begin_inset Quotes erd
383 is the name of the event.
384 The third is a format string that announces the names and the types of
386 Its format resembles that of a printf() format string; it is described
387 thoroughly in Appendix x.
390 \begin_layout Standard
391 A given Marker may appear more than once in the same program.
392 Other Markers may have the same name and a different format string, although
393 this might induce some confusion at analysis time.
396 \begin_layout Subsubsection
397 Registering the Markers
400 \begin_layout Standard
401 In order to inform to register the Markers present in the module, a macro
402 must be inserted at global scope.
403 Only one such macro is needed per exacutable or per shared object.
404 Adding it more than once, however, is harmless.
407 \begin_layout Standard
408 \begin_inset listings
412 \begin_layout Plain Layout
422 \begin_layout Subsection
426 \begin_layout Standard
427 The Tracepoints API uses the Markers, but provides a higher-level abstraction.
428 Whereas the markers API provides limited type checking, the Tracepoints
429 API provides more thorough type checking and discharges from the need to
430 insert format strings directly in the code and to have format strings appear
431 more than once if a given marker is reused.
434 \begin_layout Standard
435 A tracepoint in the code looks like this:
438 \begin_layout Standard
439 \begin_inset listings
443 \begin_layout Plain Layout
448 \begin_layout Plain Layout
452 \begin_layout Plain Layout
457 \begin_layout Plain Layout
462 \begin_layout Plain Layout
467 \begin_layout Plain Layout
472 \begin_layout Plain Layout
476 \begin_layout Plain Layout
479 set values of v and st ...
483 \begin_layout Plain Layout
487 \begin_layout Plain Layout
492 \begin_layout Plain Layout
494 trace_main_myevent(v, st);
497 \begin_layout Plain Layout
507 \begin_layout Standard
508 \begin_inset listings
512 \begin_layout Plain Layout
522 \begin_layout Standard
523 \begin_inset listings
527 \begin_layout Plain Layout
537 \begin_layout Subsection
538 Compiling the Application
541 \begin_layout Standard
543 \begin_inset Quotes eld
547 \begin_inset Quotes erd
550 directory for an example application and makefile.
553 \begin_layout Itemize
554 The compiler must have access to the include path for the libust headers.
557 \begin_layout Itemize
558 The application should be statically or dynamically linked to libust.
561 \begin_layout Section
565 \begin_layout Standard
566 These preliminary steps need to be done before recording a trace.
569 \begin_layout Standard
570 \begin_inset listings
574 \begin_layout Plain Layout
576 Create the directory for the communication sockets.
577 Eventually this should be somewhere in /var.
580 \begin_layout Plain Layout
585 \begin_layout Plain Layout
589 \begin_layout Plain Layout
591 Create the directory where ustd will write the trace.
592 This will become configurable.
595 \begin_layout Plain Layout
600 \begin_layout Plain Layout
604 \begin_layout Plain Layout
609 \begin_layout Plain Layout
619 \begin_layout Subsection
623 \begin_layout Standard
624 Tracing on a running program is controlled with the ust helper program.
625 Ust can start the tracing, stop the tracing, enable markers, disable markers
626 and list the markers.
627 Another program, ustd, is a daemon that collects the trace events and write
631 \begin_layout Standard
632 In order to record a trace, the daemon must first be started with the following
636 \begin_layout Standard
637 \begin_inset listings
641 \begin_layout Plain Layout
651 \begin_layout Standard
652 Markers can be listed with the following command (for PID 1234).
655 \begin_layout Standard
656 \begin_inset listings
660 \begin_layout Plain Layout
662 ust --list-markers 1234
670 \begin_layout Standard
671 A marker can be enabled with the following command (for PID 1234).
674 \begin_layout Standard
675 \begin_inset listings
679 \begin_layout Plain Layout
681 ust --enable-marker ust/mymark 1234
689 \begin_layout Standard
690 Then, the trace can be started.
693 \begin_layout Standard
694 \begin_inset listings
698 \begin_layout Plain Layout
700 ust --start-trace 1234
708 \begin_layout Standard
709 Eventually it can be stopped and destroyed.
712 \begin_layout Standard
713 \begin_inset listings
717 \begin_layout Plain Layout
719 ust --stop-trace 1234
722 \begin_layout Plain Layout
724 ust --destroy-trace 1234
732 \begin_layout Subsection
736 \begin_layout Standard
737 Sometimes, an application must be traced as soon as it is started.
738 In these cases, the Basic Recording method is not satisfactory, as important
739 events may be lost before the tracing is started.
742 \begin_layout Standard
743 By using the Early Tracing method, it is guaranteed that the tracing is
744 started when the execution of the main() function of the program starts.
747 \begin_layout Standard
748 Early Tracing may be enabled by defining the UST_TRACE environment variable
749 to a non-empty value when the program starts.
750 Additionally, the UST_AUTOPROBE may be set to a non-empty value to automaticall
751 y connect all markers to the default probe.
755 \begin_layout Standard
756 \begin_inset listings
760 \begin_layout Plain Layout
762 # In another terminal, start the daemon.
765 \begin_layout Plain Layout
770 \begin_layout Plain Layout
774 \begin_layout Plain Layout
777 \begin_inset Quotes eld
781 \begin_inset Quotes erd
787 \begin_layout Plain Layout
789 UST_TRACE=1 UST_AUTOPROBE=1 ./prog
797 \begin_layout Standard
798 In order for the trace to be saved, ustd must be running when the traced
802 \begin_layout Section
803 Viewing and Analyzing the Trace
806 \begin_layout Standard
807 LTTV may be used for opening the trace.
808 If an appropriate time source was used, it may be opened concurrently with
809 other application traces and kernel traces.
812 \begin_layout Section
816 \begin_layout Subsection
817 Using Custom Probes for Conditional Tracing
820 \begin_layout Subsection
821 Instrumenting Calls to Library Functions without Recompilation
824 \begin_layout Standard
825 Calls to uninstrumented libraries may be instrumented by creating a wrapper
826 library that intercepts calls to the library, trigger an instrumentation
830 \begin_layout Standard
831 Such an example can be found in the libmallocwrap directory of the ust code
835 \begin_layout Subsection
836 Tracing Programs Without Linking them to the Tracing Library
839 \begin_layout Standard
840 Programs that were not instrumented nor linked with the tracing libraries
842 In order to produce events, they must be linked to instrumented libraries
843 or use instrumented library wrappers as described in section xx.
846 \begin_layout Standard
847 \begin_inset listings
851 \begin_layout Plain Layout
862 \begin_layout Section
864 Format of Marker Format Strings
867 \begin_layout Standard
868 The format of Marker format strings is inspired from printf() format strings.
869 As with printf(), it is used to indicate to the parsing function the type
870 of the arguments that are passed.
871 Additionally, format strings indicate the name of each argument and the
872 format of that argument in the trace.
873 The structure of a typical format string is the following.
876 \begin_layout Standard
877 \begin_inset listings
881 \begin_layout Plain Layout
883 \begin_inset Quotes eld
886 field_name1 #tracetype1 %ctype1 field_name2 #tracetype2 %ctype2
887 \begin_inset Quotes erd
898 \begin_layout Description
899 field_name: The name of the field, as it will be seen when the trace is
903 \begin_layout Description
904 tracetype: The type of the argument as it will be written in the trace.
907 \begin_layout Description
908 ctype: The C type of the argument passed to the Marker (this is very similar
910 types in a printf() format string)
913 \begin_layout Standard
914 Both field_name and tracetype are optional.
915 These are all valid format strings:
918 \begin_layout Standard
919 \begin_inset listings
923 \begin_layout Plain Layout
932 \begin_layout Subsection
936 \begin_layout Standard
937 A typical Marker format string looks like this:
940 \begin_layout Standard
941 The serialization format string supports the basic printf format strings.
944 \begin_layout Standard
945 In addition, it defines new formats that can be used to serialize more
948 \begin_layout Standard
949 complex/non portable data structures.
952 \begin_layout Standard
956 \begin_layout Standard
960 \begin_layout Standard
961 field_name #tracetype %ctype
964 \begin_layout Standard
965 field_name #tracetype %ctype1 %ctype2 ...
968 \begin_layout Standard
969 A conversion is performed between format string types supported by GCC and
972 \begin_layout Standard
973 the trace type requested.
974 GCC type is used to perform type checking on format
977 \begin_layout Standard
979 Trace type is used to specify the exact binary representation
982 \begin_layout Standard
984 A mapping is done between one or more GCC types to one trace
987 \begin_layout Standard
989 Sign extension, if required by the conversion, is performed following
992 \begin_layout Standard
996 \begin_layout Standard
997 If a gcc format is not declared with a trace format, the gcc format is
1000 \begin_layout Standard
1001 also used as binary representation in the trace.
1004 \begin_layout Standard
1005 Strings are supported with %s.
1008 \begin_layout Standard
1009 A single tracetype (sequence) can take multiple c types as parameter.
1012 \begin_layout Standard
1016 \begin_layout Standard
1020 \begin_layout Standard
1021 Note: to write a uint32_t in a trace, the following expression is recommended
1024 \begin_layout Standard
1025 si it can be portable:
1028 \begin_layout Standard
1029 ("#4u%lu", (unsigned long)var)
1032 \begin_layout Standard
1036 \begin_layout Standard
1037 Serialization specific formats :
1040 \begin_layout Standard
1044 \begin_layout Standard
1048 \begin_layout Standard
1052 \begin_layout Standard
1056 \begin_layout Standard
1060 \begin_layout Standard
1064 \begin_layout Standard
1068 \begin_layout Standard
1072 \begin_layout Standard
1076 \begin_layout Standard
1080 \begin_layout Standard
1081 #1u%lu #2u%lu #4d%lu #8d%lu #llu%hu #d%lu
1084 \begin_layout Standard
1088 \begin_layout Standard
1089 n: (for network byte order)
1092 \begin_layout Standard
1096 \begin_layout Standard
1097 is written in the trace in network byte order.
1100 \begin_layout Standard
1101 i.e.: #bn4u%lu, #n%lu, #b%u
1104 \begin_layout Standard
1108 \begin_layout Standard
1109 Variable length sequence
1112 \begin_layout Standard
1113 #a #tracetype1 #tracetype2 %array_ptr %elem_size %num_elems
1116 \begin_layout Standard
1120 \begin_layout Standard
1121 #a specifies that this is a sequence
1124 \begin_layout Standard
1125 #tracetype1 is the type of elements in the sequence
1128 \begin_layout Standard
1129 #tracetype2 is the type of the element count
1132 \begin_layout Standard
1136 \begin_layout Standard
1137 array_ptr is a pointer to an array that contains members of size
1140 \begin_layout Standard
1144 \begin_layout Standard
1145 num_elems is the number of elements in the array.
1148 \begin_layout Standard
1149 i.e.: #a #lu #lu %p %lu %u
1152 \begin_layout Standard
1156 \begin_layout Standard
1157 #k callback (taken from the probe data)
1160 \begin_layout Standard
1161 The following % arguments are exepected by the callback
1164 \begin_layout Standard
1165 i.e.: #a #lu #lu #k %p
1168 \begin_layout Standard
1169 Note: No conversion is done from floats to integers, nor from integers to
1172 \begin_layout Standard
1173 floats between c types and trace types.
1174 float conversion from double to float
1177 \begin_layout Standard
1178 or from float to double is also not supported.
1181 \begin_layout Section
1182 Environment variables
1185 \begin_layout Standard
1186 The following environment variables change the behavior of libust.
1189 \begin_layout Description
1190 UST_TRACE When defined, automatically enables tracing.
1193 \begin_layout Description
1194 UST_AUTOPROBE When value is 1, enable all markers automatically.
1195 When value is a regular expression, enable all markers whose name matches
1196 the regular expression.
1199 \begin_layout Description
1200 UST_DAEMON_SOCKET If not set, the default location of the daemon socket
1202 If set, the value is used as path to the ustd socket.