doc/developer/ust.html

   1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
   2 <html><head>
   3 <meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
   4
   5
   6   <title>LTTng User-space Tracing Design</title>
   7 </head><body>
   8
   9 <h1>LTTng User-space Tracing Design</h1>
  10
  11 <p>
  12 Earlier versions of LTTng contained simple, system call based, tracepoints and fast,
  13 user-space buffered, tracepoints for user-space tracing. During the kernel inclusion phase
  14 of LTTng, extensive rework and modularization of the kernel tracing portion was undertaken.
  15 This phase is well under way and several portions have been included already in the mainline kernel.
  16 The rework of the  kernel tracing infrastructure will shortly thereafter be ported to fast user-space
  17 tracing.
  18 This fast user-space tracing scheme uses a direct function call to write events into buffers
  19 mapped in user-space. This should be an order of magnitude faster than the
  20 current Dtrace implementation (c.f.
  21 <a href="http://ltt.polymtl.ca/tracingwiki/index.php/DTrace">Dtrace information
  22 on the TracingWiki</a>) which uses a breakpoint to
  23 perform both dynamic and static user-space tracing. Performance comparison of
  24 a function call vs the int3 approach is available at
  25 <a href="http://sourceware.org/ml/systemtap/2006-q3/msg00793.html">Markers vs
  26 int3 performance comparison</a> (see "Conclusion").
  27
  28 </p><p>
  29 Libmarkers will provide applications with user-space Markers and Tracepoints
  30 declarations, such that programmers will be able to insert Markers and Tracepoints in their
  31 libraries and applications. User-space Tracepoints and Markers, analogous to Kernel Tracepoints and Markers,
  32 define program locations and arguments provided. Libmarkers will also
  33 provide utility functions to enumerate, activate and deactivate tracepoints and markers in the
  34 process, and to associate probes with any tracepoint or marker.
  35 <ul><li> Linux Kernel Tracepoints documentation: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/tracepoints.txt" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/tracepoints.txt" rel="nofollow">Documentation/tracepoints.txt</a>
  36 </li><li> Linux Kernel Tracepoints example: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=samples/tracepoints" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=samples/tracepoints" rel="nofollow">samples/tracepoints</a>
  37 </li><li> Linux Kernel Markers documentation: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/markers.txt" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/markers.txt" rel="nofollow">Documentation/markers.txt</a>
  38 </li><li> Linux Kernel Markers example: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=samples/markers" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=samples/markers" rel="nofollow">samples/markers</a>
  39 </ul>
  40
  41
  42 </p><p>
  43 Libtracing will provide the infrastructure to allocate buffers, define event types, write event
  44 metadata and data to the buffers, and get notification when buffers are full. The initial implementation
  45 will simply use one set of buffers per process. Subsequent more optimized versions will allocate one
  46 set of buffers per thread; one set of buffers per CPU would be desirable but user-space programs cannot check
  47 or control CPU migration without resorting to more costly bus locking operations or system calls. The library
  48 provides a generic probe for markers which, when connected, generates an event in the buffer each time the marker
  49 is encountered.
  50
  51 <UL>
  52 </li><li> LTTng Linux Kernel Markers Efficicient API: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=ltt/probes/ltt-type-serializer.h" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=ltt/probes/ltt-type-serializer.h" rel="nofollow">ltt/probes/ltt-type-serializer.h</a>
  53 </li><li> LTTng Linux Kernel Markers Efficicient API usage example: <a href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=ltt/probes/kernel-trace.c" class="external text" title="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=ltt/probes/kernel-trace.c" rel="nofollow">ltt/probes/kernel-trace.c</a>
  54 </li>
  55 </UL>
  56 </p><p>
  57 Finally, libtracingcontrol opens a connection allowing a separate process (e.g. LTTng daemon, Eclipse, GDB) to control the
  58 tracing within the application. Through this connection, the remote process can:
  59
  60 <UL>
  61 <li> list the available Markers and Tracepoints;
  62 </li><li> dynamically load a library (presumably containing probes to connect);
  63 </li><li> connect a probe to a Tracepoint or Marker;
  64 </li><li> activate a Tracepoint or Marker;
  65 </li><li> initialize the tracing buffers;
  66 </li><li> retrieve the content of the tracing buffers;
  67 </li><li> flush the tracing buffers;
  68 </li><li> finalize the tracing buffers;
  69 </li>
  70 </UL>
  71
  72 In the first version, libtracingcontrol will simply contain the basic functionality
  73 required to generate a trace of the Markers encountered. Subsequent versions will interact with
  74 other probing mechanisms like GDB Tracepoints, user-space SystemTap and those defined by virtual
  75 machines (e.g. Java Virtual Machine, Python, Errlang...).
  76 </p><p>
  77 Tracing of Java application is planned to be done through a JNI interface.
  78 Linking standard low-level C tracing library to the application within a JNI
  79 adaptation class will be required to trace Java events. This has been prototyped
  80 in the past. The work is available
  81 <a href="http://ltt.polymtl.ca/svn/trunk/obsolete/ltt-usertrace/java/">here</a>
  82 for older LTTng versions.
  83 <p></p>
  84 The principle of operation of libtracingcontrol is that
  85 when the instrumented application starts, a pipe is opened to allow external
  86 tracing control. Asynchronous notification is requested when commands arrive in
  87 the pipe, and a signal handler is installed for SIGIO (or a carefully chosen
  88 chainable signal number). Every time such signal is received, the runtime
  89 library checks for commands received on the external tracing control pipe.
  90 The application may also spontaneously provide information to the remote control process
  91 through the pipe:
  92
  93 </p><ul></li><li> a buffer is full;
  94 </li><li> new tracepoints or markers are appearing or disappearing, because modules are
  95   dynamically loaded or unloaded;
  96 </li></ul>
  97 <p>
  98 In addition, the tracing control application should be notified when the
  99 application exits (to save the content of buffers if the application is
 100 crashing) or forks (to trace the child as well if needed).
 101 Such notification may be obtained through utrace.
 102
 103 </p><p>
 104 In summary, a user-space application linking with libmarkers may contain static
 105 instrumentation, Tracepoints and Markers, just like the kernel with Kernel Markers and Tracepoints.
 106 The application can exploit this instrumentation itself or link with libtracing and have tracing probes
 107 connected to each Marker. Other instrumentation mechanisms, like the GCC instrument-function option, or hooks
 108 inserted by a JIT compiler, can also use libtracing to define and write event to the trace buffers.
 109 Finally, libtracingcontrol, analogous to GDB stubs, allows the remote control of the tracing by remote
 110 monitoring or debugging frameworks.
 111
 112 </p></body></html>