--- /dev/null
+*.html
+*.xml
--- /dev/null
+The LTTng Documentation
+=======================
+Philippe Proulx <pproulx@efficios.com>
+v2.5, 21 October 2016
+
+
+include::../common/copyright.txt[]
+
+
+include::../common/warning-not-maintained.txt[]
+
+
+include::../common/welcome.txt[]
+
+
+include::../common/audience.txt[]
+
+
+[[chapters]]
+=== Chapter descriptions
+
+What follows is a list of brief descriptions of this documentation's
+chapters. The latter are ordered in such a way as to make the reading
+as linear as possible.
+
+. <<nuts-and-bolts,Nuts and bolts>> explains the
+ rudiments of software tracing and the rationale behind the
+ LTTng project.
+. <<installing-lttng,Installing LTTng>> is divided into
+ sections describing the steps needed to get a working installation
+ of LTTng packages for common Linux distributions and from its
+ source.
+. <<getting-started,Getting started>> is a very concise guide to
+ get started quickly with LTTng kernel and user space tracing. This
+ chapter is recommended if you're new to LTTng or software tracing
+ in general.
+. <<understanding-lttng,Understanding LTTng>> deals with some
+ core concepts and components of the LTTng suite. Understanding
+ those is important since the next chapter assumes you're familiar
+ with them.
+. <<using-lttng,Using LTTng>> is a complete user guide of the
+ LTTng project. It shows in great details how to instrument user
+ applications and the Linux kernel, how to control tracing sessions
+ using the `lttng` command line tool and miscellaneous practical use
+ cases.
+. <<reference,Reference>> contains references of LTTng components,
+ like links to online manpages and various APIs.
+
+We recommend that you read the above chapters in this order, although
+some of them may be skipped depending on your situation. You may skip
+<<nuts-and-bolts,Nuts and bolts>> if you're familiar with tracing
+and LTTng. Also, you may jump over <<installing-lttng,Installing LTTng>>
+if LTTng is already properly installed on your target system.
+
+
+include::../common/convention.txt[]
+
+
+include::../common/acknowledgements.txt[]
+
+
+[[whats-new]]
+== What's new in LTTng {revision}?
+
+The **LTTng {revision}** toolchain introduces many interesting features,
+some of them which have been requested by users many times.
+
+It is now possible to
+<<saving-loading-tracing-session,save and restore tracing sessions>>.
+Sessions are saved to and loaded from XML files located by default in a
+subdirectory of the user's home directory. LTTng daemons are also
+configurable by configuration files as of LTTng-tools {revision}. This version
+also makes it possible to load user-defined kernel probes with the new
+session daemon's `--kmod-probes` option (or using the
+`LTTNG_KMOD_PROBES` environment variable).
+
+<<tracef,`tracef()`>> is a new instrumentation facility in LTTng-UST {revision}
+which makes it possible to insert `printf()`-like tracepoints in C/$$C++$$
+code for quick debugging. LTTng-UST {revision} also adds support for perf PMU
+counters in user space on the x86 architecture
+(see <<adding-context,Adding some context to channels>>).
+
+As of LTTng-modules {revision}, a new
+<<proc-lttng-logger-abi,LTTng logger ABI>>
+is made available, making tracing Bash scripts, for example, much more
+easier (just `echo` whatever you need to record to path:{/proc/lttng-logger}
+while tracing is active). On the kernel side, some tracepoints are
+added: state dumps of block devices, file descriptors, and file modes,
+as well as http://en.wikipedia.org/wiki/Video4Linux[V4L2] events. Linux
+3.15 is now officially supported, and system call tracing is now
+possible on the MIPS32 architecture.
+
+To learn more about the new features of LTTng {revision}, see
+http://lttng.org/blog/2014/08/04/lttng-toolchain-2-5-0-is-out/[this
+release announcement].
+
+
+[[nuts-and-bolts]]
+== Nuts and bolts
+
+What is LTTng? As its name suggests, the _Linux Trace Toolkit: next
+generation_ is a modern toolkit for tracing Linux systems and
+applications. So your first question might rather be: **what is
+tracing?**
+
+As the history of software engineering progressed and led to what
+we now take for granted--complex, numerous and
+interdependent software applications running in parallel on
+sophisticated operating systems like Linux--the authors of such
+components, or software developers, began feeling a natural
+urge of having tools to ensure the robustness and good performance
+of their masterpieces.
+
+One major achievement in this field is, inarguably, the
+https://www.gnu.org/software/gdb/[GNU debugger (GDB)], which is an
+essential tool for developers to find and fix bugs. But even the best
+debugger won't help make your software run faster, and nowadays, faster
+software means either more work done by the same hardware, or cheaper
+hardware for the same work.
+
+A _profiler_ is often the tool of choice to identify performance
+bottlenecks. Profiling is suitable to identify _where_ performance is
+lost in a given software; the profiler outputs a profile, a statistical
+summary of observed events, which you may use to know which functions
+took the most time to execute. However, a profiler won't report _why_
+some identified functions are the bottleneck. Also, bottlenecks might
+only occur when specific conditions are met. For a thorough
+investigation of software performance issues, a history of execution,
+with historical values of chosen variables, is essential. This is where
+tracing comes in handy.
+
+_Tracing_ is a technique used to understand what goes on in a running
+software system. The software used for tracing is called a _tracer_,
+which is conceptually similar to a tape recorder. When recording,
+specific points placed in the software source code generate events that
+are saved on a giant tape: a _trace_ file. Both user applications and
+the operating system may be traced at the same time, opening the
+possibility of resolving a wide range of problems that are otherwise
+extremely challenging.
+
+Tracing is often compared to _logging_. However, tracers and loggers are
+two different types of tools, serving two different purposes. Tracers
+are designed to record much lower-level events that occur much more
+frequently than log messages, often in the thousands per second range,
+with very little execution overhead. Logging is more appropriate for
+very high-level analysis of less frequent events: user accesses,
+exceptional conditions (e.g., errors, warnings), database transactions,
+instant messaging communications, etc. More formally, logging is one of
+several use cases that can be accomplished with tracing.
+
+The list of recorded events inside a trace file may be read manually
+like a log file for the maximum level of detail, but it is generally
+much more interesting to perform application-specific analyses to
+produce reduced statistics and graphs that are useful to resolve a given
+problem. Trace viewers and analysers are specialized tools which achieve
+this.
+
+So, in the end, this is what LTTng is: a powerful, open source set of
+tools to trace the Linux kernel and user applications. LTTng is composed
+of several components actively maintained and developed by its
+http://lttng.org/community/#where[community].
+
+Excluding proprietary solutions, a few competing software tracers exist
+for Linux.
+https://www.kernel.org/doc/Documentation/trace/ftrace.txt[ftrace] is the
+de facto function tracer of the Linux kernel.
+http://linux.die.net/man/1/strace[strace] is able to record all system
+calls made by a user process.
+https://sourceware.org/systemtap/[SystemTap] is a Linux kernel and user
+space tracer which uses custom user scripts to produce plain text
+traces. http://www.sysdig.org/[sysdig] also uses scripts, written in
+Lua, to trace and analyze the Linux kernel.
+
+The main distinctive features of LTTng is that it produces correlated
+kernel and user space traces, as well as doing so with the lowest
+overhead amongst other solutions. It produces trace files in the
+http://www.efficios.com/ctf[CTF] format, an optimized file format for
+production and analyses of multi-gigabyte data. LTTng is the result of
+close to 10 years of active development by a community of passionate
+developers. It is currently available on some major desktop, server, and
+embedded Linux distributions.
+
+The main interface for tracing control is a single command line tool
+named `lttng`. The latter can create several tracing sessions,
+enable/disable events on the fly, filter them efficiently with custom
+user expressions, start/stop tracing and do much more. Traces can be
+recorded on disk or sent over the network, kept totally or partially,
+and viewed once tracing is inactive or in real-time.
+
+<<installing-lttng,Install LTTng now>> and start tracing!
+
+
+[[installing-lttng]]
+== Installing LTTng
+
+**LTTng** is a set of software components which interact to allow
+instrumenting the Linux kernel and user applications and controlling
+tracing sessions (starting/stopping tracing, enabling/disabling events,
+etc.). Those components are bundled into the following packages:
+
+LTTng-tools::
+ Libraries and command line interface to control tracing sessions.
+
+LTTng-modules::
+ Linux kernel modules allowing Linux to be traced using LTTng.
+
+LTTng-UST::
+ User space tracing library.
+
+Most distributions mark the LTTng-modules and LTTng-UST packages as
+optional. In the following sections, we always provide the steps to
+install all three, but be aware that LTTng-modules is only required if
+you intend to trace the Linux kernel and LTTng-UST is only required if
+you intend to trace user space applications.
+
+This chapter shows how to install the above packages on a Linux system.
+The easiest way is to use the package manager of the system's
+distribution (<<desktop-distributions,desktop>> or
+<<embedded-distributions,embedded>>). Support is also available for
+<<enterprise-distributions,enterprise distributions>>, such as Red Hat
+Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SLES).
+Otherwise, you can
+<<building-from-source,build the LTTng packages from source>>.
+
+
+[[desktop-distributions]]
+=== Desktop distributions
+
+Official LTTng {revision} packages are available for <<ubuntu,Ubuntu>> and
+<<debian,Debian>>.
+
+More recent versions of LTTng are available for Fedora, openSUSE,
+as well as Arch Linux.
+
+Should any issue arise when following the procedures below, please
+inform the http://lttng.org/community[community] about it.
+
+
+[[ubuntu]]
+==== Ubuntu
+
+LTTng {revision} is packaged in Ubuntu 15.04 _Vivid Vervet_. For other
+releases of Ubuntu, you need to build and install LTTng
+<<building-from-source,from source>>. Ubuntu 15.10 _Wily Werewolf_
+ships with link:/docs/v2.6/[LTTng 2.6].
+
+To install LTTng {revision} from the official Ubuntu repositories,
+simply use `apt-get`:
+
+[role="term"]
+----
+sudo apt-get install lttng-tools
+sudo apt-get install lttng-modules-dkms
+sudo apt-get install liblttng-ust-dev
+----
+
+
+[[debian]]
+==== Debian
+
+Debian "jessie" has official packages of LTTng {revision}:
+
+[role="term"]
+----
+sudo apt-get install lttng-tools
+sudo apt-get install lttng-modules-dkms
+sudo apt-get install liblttng-ust-dev
+----
+
+
+[[embedded-distributions]]
+=== Embedded distributions
+
+Some developers may be interested in tracing the Linux kernel and user space
+applications running on embedded systems. LTTng is packaged by two popular
+embedded Linux distributions: <<buildroot,Buildroot>> and
+<<oe-yocto,OpenEmbedded/Yocto>>.
+
+
+[[buildroot]]
+==== Buildroot
+
+LTTng {revision} packages in Buildroot 2014.11 and 2015.02 are named
+`lttng-tools`, `lttng-modules`, and `lttng-libust`.
+
+To enable them, start the Buildroot configuration menu as usual:
+
+[role="term"]
+----
+make menuconfig
+----
+
+In:
+
+* _Kernel_: make sure _Linux kernel_ is enabled
+* _Toolchain_: make sure the following options are enabled:
+** _Enable large file (files > 2GB) support_
+** _Enable WCHAR support_
+
+In _Target packages_/_Debugging, profiling and benchmark_, enable
+_lttng-modules_ and _lttng-tools_. In
+_Target packages_/_Libraries_/_Other_, enable _lttng-libust_.
+
+
+[[oe-yocto]]
+==== OpenEmbedded/Yocto
+
+LTTng {revision} recipes are available in the `openembedded-core` layer of
+OpenEmbedded from August 15th, 2014 to February 8th, 2015 under the
+following names:
+
+* `lttng-tools`
+* `lttng-modules`
+* `lttng-ust`
+
+Using BitBake, the simplest way to include LTTng recipes in your
+target image is to add them to `IMAGE_INSTALL_append` in
+path:{conf/local.conf}:
+
+----
+IMAGE_INSTALL_append = " lttng-tools lttng-modules lttng-ust"
+----
+
+If you're using Hob, click _Edit image recipe_ once you have selected
+a machine and an image recipe. Then, in the _All recipes_ tab, search
+for `lttng` and you should find and be able to include the three LTTng
+recipes.
+
+
+[[enterprise-distributions]]
+=== Enterprise distributions (RHEL, SLES)
+
+To install LTTng on enterprise Linux distributions
+(such as RHEL and SLES), please see
+http://packages.efficios.com/[EfficiOS Enterprise Packages].
+
+
+[[building-from-source]]
+=== Building from source
+
+As <<installing-lttng,previously stated>>, LTTng is shipped as three
+packages: LTTng-tools, LTTng-modules and LTTng-UST. LTTng-tools contains
+everything needed to control tracing sessions, while LTTng-modules is
+only needed for Linux kernel tracing and LTTng-UST is only needed for
+user space tracing.
+
+The tarballs are available in the
+http://lttng.org/download#build-from-source[Download section]
+of the LTTng website.
+
+Please refer to the path:{README.md} files provided by each package to
+properly build and install them.
+
+TIP: The aforementioned path:{README.md} files are rendered as
+rich text when https://github.com/lttng[viewed on GitHub].
+
+
+[[getting-started]]
+== Getting started with LTTng
+
+This is a small guide to get started quickly with LTTng kernel and user
+space tracing. For intermediate to advanced use cases and a more
+thorough understanding of LTTng, see <<using-lttng,Using LTTng>> and
+<<understanding-lttng,Understanding LTTng>>.
+
+Before reading this guide, make sure LTTng
+<<installing-lttng,is installed>>. You will at least need LTTng-tools.
+Also install LTTng-modules for
+<<tracing-the-linux-kernel,tracing the Linux kernel>>
+and LTTng-UST for <<tracing-your-own-user-application,tracing your own
+user space applications>>. When your traces are finally written and
+complete, the
+<<viewing-and-analyzing-your-traces,Viewing and analyzing your traces>>
+section of this chapter will help you analyze your tracepoint
+events to investigate.
+
+
+[[tracing-the-linux-kernel]]
+=== Tracing the Linux kernel
+
+Make sure LTTng-tools and LTTng-modules packages
+<<installing-lttng,are installed>>.
+
+Since you're about to trace the Linux kernel itself, let's look at the
+available kernel events using the `lttng` tool, which has a
+Git-like command line structure:
+
+[role="term"]
+----
+lttng list --kernel
+----
+
+Before tracing, you need to create a session:
+
+[role="term"]
+----
+sudo lttng create my-session
+----
+
+TIP: You can avoid using `sudo` in the previous and following commands
+if your user is a member of the <<lttng-sessiond,tracing group>>.
+
+`my-session` is the tracing session name and could be anything you
+like. `auto` will be used if omitted.
+
+Let's now enable some events for this session:
+
+[role="term"]
+----
+sudo lttng enable-event --kernel sched_switch,sched_process_fork
+----
+
+or you might want to simply enable all available kernel events (beware
+that trace files will grow rapidly when doing this):
+
+[role="term"]
+----
+sudo lttng enable-event --kernel --all
+----
+
+Start tracing:
+
+[role="term"]
+----
+sudo lttng start
+----
+
+By default, traces are saved in
++\~/lttng-traces/__name__-__date__-__time__+,
+where +__name__+ is the session name.
+
+When you're done tracing:
+
+[role="term"]
+----
+sudo lttng stop
+sudo lttng destroy
+----
+
+Although `destroy` looks scary here, it doesn't actually destroy the
+outputted trace files: it only destroys the tracing session.
+
+What's next? Have a look at
+<<viewing-and-analyzing-your-traces,Viewing and analyzing your traces>>
+to view and analyze the trace you just recorded.
+
+
+[[tracing-your-own-user-application]]
+=== Tracing your own user application
+
+The previous section helped you create a trace out of Linux kernel
+events. This section steps you through a simple example showing you how
+to trace a _Hello world_ program written in C.
+
+Make sure LTTng-tools and LTTng-UST packages
+<<installing-lttng,are installed>>.
+
+Tracing is just like having `printf()` calls at specific locations of
+your source code, albeit LTTng is much faster and more flexible than
+`printf()`. In the LTTng realm, **`tracepoint()`** is analogous to
+`printf()`.
+
+Unlike `printf()`, though, `tracepoint()` does not use a format string to
+know the types of its arguments: the formats of all tracepoints must be
+defined before using them. So before even writing our _Hello world_ program,
+we need to define the format of our tracepoint. This is done by writing a
+**template file**, with a name usually ending
+with the `.tp` extension (for **t**race**p**oint),
+which the `lttng-gen-tp` tool (shipped with LTTng-UST) will use to generate
+an object file (along with a `.c` file) and a header to be
+included in our application source code.
+
+Here's the whole flow:
+
+[role="img-80"]
+.Build workflow for LTTng application tracing.
+image::lttng-lttng-gen-tp.png[]
+
+The template file format is a list of tracepoint definitions
+and other optional definition entries which we will skip for
+this quickstart. Each tracepoint is defined using the
+`TRACEPOINT_EVENT()` macro. For each tracepoint, you must provide:
+
+* a **provider name**, which is the "scope" of this tracepoint (this usually
+ includes the company and project names)
+* a **tracepoint name**
+* a **list of arguments** for the eventual `tracepoint()` call,
+ each item being:
+** the argument C type
+** the argument name
+* a **list of fields**, which will be the actual fields of the recorded events
+ for this tracepoint
+
+Here's a simple tracepoint definition example with two arguments: an integer
+and a string:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ hello_world,
+ my_first_tracepoint,
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+----
+
+The exact syntax is well explained in the
+<<c-application,C application>> instrumenting guide of the
+<<using-lttng,Using LTTng>> chapter, as well as in man:lttng-ust(3).
+
+Save the above snippet as path:{hello-tp.tp} and run:
+
+[role="term"]
+----
+lttng-gen-tp hello-tp.tp
+----
+
+The following files will be created next to path:{hello-tp.tp}:
+
+* path:{hello-tp.c}
+* path:{hello-tp.o}
+* path:{hello-tp.h}
+
+path:{hello-tp.o} is the compiled object file of path:{hello-tp.c}.
+
+Now, by including path:{hello-tp.h} in your own application, you may use the
+tracepoint defined above by properly refering to it when calling
+`tracepoint()`:
+
+[source,c]
+----
+#include <stdio.h>
+#include "hello-tp.h"
+
+int main(int argc, char* argv[])
+{
+ int x;
+
+ puts("Hello, World!\nPress Enter to continue...");
+
+ /* The following getchar() call is only placed here for the purpose
+ * of this demonstration, for pausing the application in order for
+ * you to have time to list its events. It's not needed otherwise.
+ */
+ getchar();
+
+ /* A tracepoint() call. Arguments, as defined in hello-tp.tp:
+ *
+ * 1st: provider name (always)
+ * 2nd: tracepoint name (always)
+ * 3rd: my_integer_arg (first user-defined argument)
+ * 4th: my_string_arg (second user-defined argument)
+ *
+ * Notice the provider and tracepoint names are NOT strings;
+ * they are in fact parts of variables created by macros in
+ * hello-tp.h.
+ */
+ tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");
+
+ for (x = 0; x < argc; ++x) {
+ tracepoint(hello_world, my_first_tracepoint, x, argv[x]);
+ }
+
+ puts("Quitting now!");
+
+ tracepoint(hello_world, my_first_tracepoint, x * x, "x^2");
+
+ return 0;
+}
+----
+
+Save this as path:{hello.c}, next to path:{hello-tp.tp}.
+
+Notice path:{hello-tp.h}, the header file generated by path:{lttng-gen-tp} from
+our template file path:{hello-tp.tp}, is included by path:{hello.c}.
+
+You are now ready to compile the application with LTTng-UST support:
+
+[role="term"]
+----
+gcc -o hello hello.c hello-tp.o -llttng-ust -ldl
+----
+
+If you followed the
+<<tracing-the-linux-kernel,Tracing the Linux kernel>> section, the
+following steps will look familiar.
+
+First, run the application with a few arguments:
+
+[role="term"]
+----
+./hello world and beyond
+----
+
+You should see
+
+----
+Hello, World!
+Press Enter to continue...
+----
+
+Use the `lttng` tool to list all available user space events:
+
+[role="term"]
+----
+lttng list --userspace
+----
+
+You should see the `hello_world:my_first_tracepoint` tracepoint listed
+under the `./hello` process.
+
+Create a tracing session:
+
+[role="term"]
+----
+lttng create my-userspace-session
+----
+
+Enable the `hello_world:my_first_tracepoint` tracepoint:
+
+[role="term"]
+----
+lttng enable-event --userspace hello_world:my_first_tracepoint
+----
+
+Start tracing:
+
+[role="term"]
+----
+lttng start
+----
+
+Go back to the running path:{hello} application and press Enter. All
+`tracepoint()` calls will be executed and the program will finally exit.
+
+Stop tracing:
+
+[role="term"]
+----
+lttng stop
+----
+
+Done! You may use `lttng view` to list the recorded events. This command
+starts
+http://www.efficios.com/babeltrace[`babeltrace`]
+in the background, if it is installed:
+
+[role="term"]
+----
+lttng view
+----
+
+should output something like:
+
+----
+[18:10:27.684304496] (+?.?????????) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "hi there!", my_integer_field = 23 }
+[18:10:27.684338440] (+0.000033944) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "./hello", my_integer_field = 0 }
+[18:10:27.684340692] (+0.000002252) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "world", my_integer_field = 1 }
+[18:10:27.684342616] (+0.000001924) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "and", my_integer_field = 2 }
+[18:10:27.684343518] (+0.000000902) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "beyond", my_integer_field = 3 }
+[18:10:27.684357978] (+0.000014460) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "x^2", my_integer_field = 16 }
+----
+
+When you're done, you may destroy the tracing session, which does _not_
+destroy the generated trace files, leaving them available for further
+analysis:
+
+[role="term"]
+----
+lttng destroy my-userspace-session
+----
+
+The next section presents other alternatives to view and analyze your
+LTTng traces.
+
+
+[[viewing-and-analyzing-your-traces]]
+=== Viewing and analyzing your traces
+
+This section describes how to visualize the data gathered after tracing
+the Linux kernel or a user space application.
+
+Many ways exist to read your LTTng traces:
+
+* **`babeltrace`** is a command line utility which converts trace formats;
+ it supports the format used by LTTng,
+ CTF, as well as a basic
+ text output which may be ++grep++ed. The `babeltrace` command is
+ part of the http://www.efficios.com/babeltrace[Babeltrace] project.
+* Babeltrace also includes a **Python binding** so that you may
+ easily open and read an LTTng trace with your own script, benefiting
+ from the power of Python.
+* **http://projects.eclipse.org/projects/tools.tracecompass[Trace Compass]**
+ is an Eclipse plugin used to visualize and analyze various types of
+ traces, including LTTng's. It also comes as a standalone application
+ and can be downloaded from
+ http://projects.eclipse.org/projects/tools.tracecompass/downloads[here].
+
+LTTng trace files are usually recorded in the path:{~/lttng-traces} directory.
+Let's now view the trace and perform a basic analysis using
+`babeltrace`.
+
+The simplest way to list all the recorded events of a trace is to pass its
+path to `babeltrace` with no options:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-session
+----
+
+`babeltrace` will find all traces within the given path recursively and
+output all their events, merging them intelligently.
+
+Listing all the system calls of a Linux kernel trace with their arguments is
+easy with `babeltrace` and `grep`:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session | grep sys_
+----
+
+Counting events is also straightforward:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session | grep sys_read | wc --lines
+----
+
+The text output of `babeltrace` is useful for isolating events by simple
+matching using `grep` and similar utilities. However, more elaborate filters
+such as keeping only events with a field value falling within a specific range
+are not trivial to write using a shell. Moreover, reductions and even the
+most basic computations involving multiple events are virtually impossible
+to implement.
+
+Fortunately, Babeltrace ships with a Python 3 binding which makes it
+really easy to read the events of an LTTng trace sequentially and compute
+the desired information.
+
+Here's a simple example using the Babeltrace Python binding. The following
+script accepts an LTTng Linux kernel trace path as its first argument and
+outputs the short names of the top 5 running processes on CPU 0 during the
+whole trace:
+
+[source,python]
+----
+import sys
+from collections import Counter
+import babeltrace
+
+
+def top5proc():
+ if len(sys.argv) != 2:
+ msg = 'Usage: python {} TRACEPATH'.format(sys.argv[0])
+ raise ValueError(msg)
+
+ # a trace collection holds one to many traces
+ col = babeltrace.TraceCollection()
+
+ # add the trace provided by the user
+ # (LTTng traces always have the 'ctf' format)
+ if col.add_trace(sys.argv[1], 'ctf') is None:
+ raise RuntimeError('Cannot add trace')
+
+ # this counter dict will hold execution times:
+ #
+ # task command name -> total execution time (ns)
+ exec_times = Counter()
+
+ # this holds the last `sched_switch` timestamp
+ last_ts = None
+
+ # iterate events
+ for event in col.events:
+ # keep only `sched_switch` events
+ if event.name != 'sched_switch':
+ continue
+
+ # keep only events which happened on CPU 0
+ if event['cpu_id'] != 0:
+ continue
+
+ # event timestamp
+ cur_ts = event.timestamp
+
+ if last_ts is None:
+ # we start here
+ last_ts = cur_ts
+
+ # previous task command (short) name
+ prev_comm = event['prev_comm']
+
+ # initialize entry in our dict if not yet done
+ if prev_comm not in exec_times:
+ exec_times[prev_comm] = 0
+
+ # compute previous command execution time
+ diff = cur_ts - last_ts
+
+ # update execution time of this command
+ exec_times[prev_comm] += diff
+
+ # update last timestamp
+ last_ts = cur_ts
+
+ # display top 10
+ for name, ns in exec_times.most_common(5):
+ s = ns / 1000000000
+ print('{:20}{} s'.format(name, s))
+
+
+if __name__ == '__main__':
+ top5proc()
+----
+
+Save this script as path:{top5proc.py} and run it with Python 3, providing the
+path to an LTTng Linux kernel trace as the first argument:
+
+[role="term"]
+----
+python3 top5proc.py ~/lttng-sessions/my-session-.../kernel
+----
+
+Make sure the path you provide is the directory containing actual trace
+files (path:{channel0_0}, path:{metadata}, etc.): the `babeltrace` utility
+recurses directories, but the Python binding does not.
+
+Here's an example of output:
+
+----
+swapper/0 48.607245889 s
+chromium 7.192738188 s
+pavucontrol 0.709894415 s
+Compositor 0.660867933 s
+Xorg.bin 0.616753786 s
+----
+
+Note that `swapper/0` is the "idle" process of CPU 0 on Linux; since we
+weren't using the CPU that much when tracing, its first position in the list
+makes sense.
+
+
+[[understanding-lttng]]
+== Understanding LTTng
+
+If you're going to use LTTng in any serious way, it is fundamental that
+you become familiar with its core concepts. Technical terms like
+_tracing sessions_, _domains_, _channels_ and _events_ are used over
+and over in the <<using-lttng,Using LTTng>> chapter,
+and it is assumed that you understand what they mean when reading it.
+
+LTTng, as you already know, is a _toolkit_. It would be wrong
+to call it a simple _tool_ since it is composed of multiple interacting
+components. This chapter also describes the latter, providing details
+about their respective roles and how they connect together to form
+the current LTTng ecosystem.
+
+
+[[core-concepts]]
+=== Core concepts
+
+This section explains the various elementary concepts a user has to deal
+with when using LTTng. They are:
+
+* <<tracing-session,tracing session>>
+* <<domain,domain>>
+* <<channel,channel>>
+* <<event,event>>
+
+
+[[tracing-session]]
+==== Tracing session
+
+A _tracing session_ is--like any session--a container of
+state. Anything that is done when tracing using LTTng happens in the
+scope of a tracing session. In this regard, it is analogous to a bank
+website's session: you can't interact online with your bank account
+unless you are logged in a session, except for reading a few static
+webpages (LTTng, too, can report some static information that does not
+need a created tracing session).
+
+A tracing session holds the following attributes and objects (some of
+which are described in the following sections):
+
+* a name
+* the tracing state (tracing started or stopped)
+* the trace data output path/URL (local path or sent over the network)
+* a mode (normal, snapshot or live)
+* the snapshot output paths/URLs (if applicable)
+* for each <<domain,domain>>, a list of <<channel,channels>>
+* for each channel:
+** a name
+** the channel state (enabled or disabled)
+** its parameters (event loss mode, sub-buffers size and count,
+ timer periods, output type, trace files size and count, etc.)
+** a list of added context information
+** a list of <<event,events>>
+* for each event:
+** its state (enabled or disabled)
+** a list of instrumentation points (tracepoints, system calls,
+ dynamic probes, etc.)
+** associated log levels
+** a filter expression
+
+All this information is completely isolated between tracing sessions.
+
+Conceptually, a tracing session is a per-user object; the
+<<plumbing,Plumbing>> section shows how this is actually
+implemented. Any user may create as many concurrent tracing sessions
+as desired. As you can see in the list above, even the tracing state
+is a per-tracing session attribute, so that you may trace your target
+system/application in a given tracing session with a specific
+configuration while another one stays inactive.
+
+The trace data generated in a tracing session may be either saved
+to disk, sent over the network or not saved at all (in which case
+snapshots may still be saved to disk or sent to a remote machine).
+
+
+[[domain]]
+==== Domain
+
+A tracing _domain_ is the official term the LTTng project uses to
+designate a tracer category.
+
+There are currently three known domains:
+
+* Linux kernel
+* user space
+* `java.util.logging` (JUL)
+
+Different tracers expose common features in their own interfaces, but,
+from a user's perspective, you still need to target a specific type of
+tracer to perform some actions. For example, since both kernel and user
+space tracers support named tracepoints (probes manually inserted in
+source code), you need to specify which one is concerned when enabling
+an event because both domains could have existing events with the same
+name.
+
+Some features are not available in all domains. Filtering enabled
+events using custom expressions, for example, is currently not
+supported in the kernel domain, but support could be added in the
+future.
+
+
+[[channel]]
+==== Channel
+
+A _channel_ is a set of events with specific parameters and potential
+added context information. Channels have unique names per domain within
+a tracing session. A given event is always registered to at least one
+channel; having an enabled event in two channels will produce a trace
+with this event recorded twice everytime it occurs.
+
+Channels may be individually enabled or disabled. Occurring events of
+a disabled channel will never make it to recorded events.
+
+The fundamental role of a channel is to keep a shared ring buffer, where
+events are eventually recorded by the tracer and consumed by a consumer
+daemon. This internal ring buffer is divided into many sub-buffers of
+equal size.
+
+Channels, when created, may be fine-tuned thanks to a few parameters,
+many of them related to sub-buffers. The following subsections explain
+what those parameters are and in which situations you should manually
+adjust them.
+
+
+[[channel-overwrite-mode-vs-discard-mode]]
+===== Overwrite and discard event loss modes
+
+As previously mentioned, a channel's ring buffer is divided into many
+equally sized sub-buffers.
+
+As events occur, they are serialized as trace data into a specific
+sub-buffer (yellow arc in the following animation) until it is full:
+when this happens, the sub-buffer is marked as consumable (red) and
+another, _empty_ (white) sub-buffer starts receiving the following
+events. The marked sub-buffer will be consumed eventually by a consumer
+daemon (returns to white).
+
+[NOTE]
+[role="docsvg-channel-subbuf-anim"]
+====
+{note-no-anim}
+====
+
+In an ideal world, sub-buffers are consumed faster than filled, like it
+is the case above. In the real world, however, all sub-buffers could be
+full at some point, leaving no space to record the following events. By
+design, LTTng is a _non-blocking_ tracer: when no empty sub-buffer
+exists, losing events is acceptable when the alternative would be to
+cause substantial delays in the instrumented application's execution.
+LTTng privileges performance over integrity, aiming at perturbing the
+traced system as little as possible in order to make tracing of subtle
+race conditions and rare interrupt cascades possible.
+
+When it comes to losing events because no empty sub-buffer is available,
+the channel's _event loss mode_ determines what to do amongst:
+
+Discard::
+ Drop the newest events until a sub-buffer is released.
+
+Overwrite::
+ Clear the sub-buffer containing the oldest recorded
+ events and start recording the newest events there. This mode is
+ sometimes called _flight recorder mode_ because it behaves like a
+ flight recorder: always keep a fixed amount of the latest data.
+
+Which mechanism you should choose depends on your context: prioritize
+the newest or the oldest events in the ring buffer?
+
+Beware that, in overwrite mode, a whole sub-buffer is abandoned as soon
+as a new event doesn't find an empty sub-buffer, whereas in discard
+mode, only the event that doesn't fit is discarded.
+
+Also note that a count of lost events will be incremented and saved in
+the trace itself when an event is lost in discard mode, whereas no
+information is kept when a sub-buffer gets overwritten before being
+committed.
+
+There are known ways to decrease your probability of losing events. The
+next section shows how tuning the sub-buffers count and size can be
+used to virtually stop losing events.
+
+
+[[channel-subbuf-size-vs-subbuf-count]]
+===== Sub-buffers count and size
+
+For each channel, an LTTng user may set its number of sub-buffers and
+their size.
+
+Note that there is a noticeable tracer's CPU overhead introduced when
+switching sub-buffers (marking a full one as consumable and switching
+to an empty one for the following events to be recorded). Knowing this,
+the following list presents a few practical situations along with how
+to configure sub-buffers for them:
+
+High event throughput::
+ In general, prefer bigger sub-buffers to
+ lower the risk of losing events. Having bigger sub-buffers will
+ also ensure a lower sub-buffer switching frequency. The number of
+ sub-buffers is only meaningful if the channel is in overwrite mode:
+ in this case, if a sub-buffer overwrite happens, you will still have
+ the other sub-buffers left unaltered.
+
+Low event throughput::
+ In general, prefer smaller sub-buffers
+ since the risk of losing events is already low. Since events
+ happen less frequently, the sub-buffer switching frequency should
+ remain low and thus the tracer's overhead should not be a problem.
+
+Low memory system::
+ If your target system has a low memory
+ limit, prefer fewer first, then smaller sub-buffers. Even if the
+ system is limited in memory, you want to keep the sub-buffers as
+ big as possible to avoid a high sub-buffer switching frequency.
+
+You should know that LTTng uses CTF as its trace format, which means
+event data is very compact. For example, the average LTTng Linux kernel
+event weights about 32{nbsp}bytes. A sub-buffer size of 1{nbsp}MiB is
+thus considered big.
+
+The previous situations highlight the major trade-off between a few big
+sub-buffers and more, smaller sub-buffers: sub-buffer switching
+frequency vs. how much data is lost in overwrite mode. Assuming a
+constant event throughput and using the overwrite mode, the two
+following configurations have the same ring buffer total size:
+
+[NOTE]
+[role="docsvg-channel-subbuf-size-vs-count-anim"]
+====
+{note-no-anim}
+====
+
+* **2 sub-buffers of 4 MiB each** lead to a very low sub-buffer
+ switching frequency, but if a sub-buffer overwrite happens, half of
+ the recorded events so far (4{nbsp}MiB) are definitely lost.
+* **8 sub-buffers of 1 MiB each** lead to 4{nbsp}times the tracer's
+ overhead as the previous configuration, but if a sub-buffer
+ overwrite happens, only the eighth of events recorded so far are
+ definitely lost.
+
+In discard mode, the sub-buffers count parameter is pointless: use two
+sub-buffers and set their size according to the requirements of your
+situation.
+
+
+[[channel-switch-timer]]
+===== Switch timer
+
+The _switch timer_ period is another important configurable feature of
+channels to ensure periodic sub-buffer flushing.
+
+When the _switch timer_ fires, a sub-buffer switch happens. This timer
+may be used to ensure that event data is consumed and committed to
+trace files periodically in case of a low event throughput:
+
+[NOTE]
+[role="docsvg-channel-switch-timer"]
+====
+{note-no-anim}
+====
+
+It's also convenient when big sub-buffers are used to cope with
+sporadic high event throughput, even if the throughput is normally
+lower.
+
+
+[[channel-buffering-schemes]]
+===== Buffering schemes
+
+In the user space tracing domain, two **buffering schemes** are
+available when creating a channel:
+
+Per-PID buffering::
+ Keep one ring buffer per process.
+
+Per-UID buffering::
+ Keep one ring buffer for all processes of a single user.
+
+The per-PID buffering scheme will consume more memory than the per-UID
+option if more than one process is instrumented for LTTng-UST. However,
+per-PID buffering ensures that one process having a high event
+throughput won't fill all the shared sub-buffers, only its own.
+
+The Linux kernel tracing domain only has one available buffering scheme
+which is to use a single ring buffer for the whole system.
+
+
+[[event]]
+==== Event
+
+An _event_, in LTTng's realm, is a term often used metonymically,
+having multiple definitions depending on the context:
+
+. When tracing, an event is a _point in space-time_. Space, in a
+ tracing context, is the set of all executable positions of a
+ compiled application by a logical processor. When a program is
+ executed by a processor and some instrumentation point, or
+ _probe_, is encountered, an event occurs. This event is accompanied
+ by some contextual payload (values of specific variables at this
+ point of execution) which may or may not be recorded.
+. In the context of a recorded trace file, the term _event_ implies
+ a _recorded event_.
+. When configuring a tracing session, _enabled events_ refer to
+ specific rules which could lead to the transfer of actual
+ occurring events (1) to recorded events (2).
+
+The whole <<core-concepts,Core concepts>> section focuses on the
+third definition. An event is always registered to _one or more_
+channels and may be enabled or disabled at will per channel. A disabled
+event will never lead to a recorded event, even if its channel
+is enabled.
+
+An event (3) is enabled with a few conditions that must _all_ be met
+when an event (1) happens in order to generate a recorded event (2):
+
+. A _probe_ or group of probes in the traced application must be
+ executed.
+. **Optionally**, the probe must have a log level matching a
+ log level range specified when enabling the event.
+. **Optionally**, the occurring event must satisfy a custom
+ expression, or _filter_, specified when enabling the event.
+
+The following illustration summarizes how tracing sessions, domains,
+channels and events are related:
+
+[role="img-90"]
+.Core concepts.
+image::core-concepts.png[]
+
+This diagram also shows how events may be individually enabled/disabled
+(green/grey) and how a given event may be registered to more than one
+channel.
+
+
+[[plumbing]]
+=== Plumbing
+
+The previous section described the concepts at the heart of LTTng.
+This section summarizes LTTng's implementation: how those objects are
+managed by different applications and libraries working together to
+form the toolkit.
+
+
+[[plumbing-overview]]
+==== Overview
+
+As <<installing-lttng,mentioned previously>>, the whole LTTng suite
+is made of the following packages: LTTng-tools, LTTng-UST, and
+LTTng-modules. Together, they provide different daemons, libraries,
+kernel modules and command line interfaces. The following tree shows
+which usable component belongs to which package:
+
+* **LTTng-tools**:
+** session daemon (`lttng-sessiond`)
+** consumer daemon (`lttng-consumerd`)
+** relay daemon (`lttng-relayd`)
+** tracing control library (`liblttng-ctl`)
+** tracing control command line tool (`lttng`)
+* **LTTng-UST**:
+** user space tracing library (`liblttng-ust`) and its headers
+** preloadable user space tracing helpers
+ (`liblttng-ust-libc-wrapper`, `liblttng-ust-pthread-wrapper`,
+ `liblttng-ust-cyg-profile`, `liblttng-ust-cyg-profile-fast`
+ and `liblttng-ust-dl`)
+** user space tracepoint code generator command line tool
+ (`lttng-gen-tp`)
+** `java.util.logging` tracepoint provider (`liblttng-ust-jul-jni`)
+ and JAR file (path:{liblttng-ust-jul.jar})
+* **LTTng-modules**:
+** LTTng Linux kernel tracer module
+** tracing ring buffer kernel modules
+** many LTTng probe kernel modules
+
+The following diagram shows how the most important LTTng components
+interact. Plain black arrows represent trace data paths while dashed
+red arrows indicate control communications. The LTTng relay daemon is
+shown running on a remote system, although it could as well run on the
+target (monitored) system.
+
+[role="img-90"]
+.LTTng plumbing.
+image::plumbing.png[]
+
+Each component is described in the following subsections.
+
+
+[[lttng-sessiond]]
+==== Session daemon
+
+At the heart of LTTng's plumbing is the _session daemon_, often called
+by its command name, `lttng-sessiond`.
+
+The session daemon is responsible for managing tracing sessions and
+what they logically contain (channel properties, enabled/disabled
+events, etc.). By communicating locally with instrumented applications
+(using LTTng-UST) and with the LTTng Linux kernel modules
+(LTTng-modules), it oversees all tracing activities.
+
+One of the many things that `lttng-sessiond` does is to keep
+track of the available event types. User space applications and
+libraries actively connect and register to the session daemon when they
+start. By contrast, `lttng-sessiond` seeks out and loads the appropriate
+LTTng kernel modules as part of its own initialization. Kernel event
+types are _pulled_ by `lttng-sessiond`, whereas user space event types
+are _pushed_ to it by the various user space tracepoint providers.
+
+Using a specific inter-process communication protocol with Linux kernel
+and user space tracers, the session daemon can send channel information
+so that they are initialized, enable/disable specific probes based on
+enabled/disabled events by the user, send event filters information to
+LTTng tracers so that filtering actually happens at the tracer site,
+start/stop tracing a specific application or the Linux kernel, etc.
+
+The session daemon is not useful without some user controlling it,
+because it's only a sophisticated control interchange and thus
+doesn't make any decision on its own. `lttng-sessiond` opens a local
+socket for controlling it, albeit the preferred way to control it is
+using `liblttng-ctl`, an installed C library hiding the communication
+protocol behind an easy-to-use API. The `lttng` tool makes use of
+`liblttng-ctl` to implement a user-friendly command line interface.
+
+`lttng-sessiond` does not receive any trace data from instrumented
+applications; the _consumer daemons_ are the programs responsible for
+collecting trace data using shared ring buffers. However, the session
+daemon is the one that must spawn a consumer daemon and establish
+a control communication with it.
+
+Session daemons run on a per-user basis. Knowing this, multiple
+instances of `lttng-sessiond` may run simultaneously, each belonging
+to a different user and each operating independently of the others.
+Only `root`'s session daemon, however, may control LTTng kernel modules
+(i.e. the kernel tracer). With that in mind, if a user has no root
+access on the target system, he cannot trace the system's kernel, but
+should still be able to trace its own instrumented applications.
+
+It has to be noted that, although only `root`'s session daemon may
+control the kernel tracer, the `lttng-sessiond` command has a `--group`
+option which may be used to specify the name of a special user group
+allowed to communicate with `root`'s session daemon and thus record
+kernel traces. By default, this group is named `tracing`.
+
+If not done yet, the `lttng` tool, by default, automatically starts a
+session daemon. `lttng-sessiond` may also be started manually:
+
+[role="term"]
+----
+lttng-sessiond
+----
+
+This will start the session daemon in foreground. Use
+
+[role="term"]
+----
+lttng-sessiond --daemonize
+----
+
+to start it as a true daemon.
+
+To kill the current user's session daemon, `pkill` may be used:
+
+[role="term"]
+----
+pkill lttng-sessiond
+----
+
+The default `SIGTERM` signal will terminate it cleanly.
+
+Several other options are available and described in
+man:lttng-sessiond(8) or by running `lttng-sessiond --help`.
+
+
+[[lttng-consumerd]]
+==== Consumer daemon
+
+The _consumer daemon_, or `lttng-consumerd`, is a program sharing some
+ring buffers with user applications or the LTTng kernel modules to
+collect trace data and output it at some place (on disk or sent over
+the network to an LTTng relay daemon).
+
+Consumer daemons are created by a session daemon as soon as events are
+enabled within a tracing session, well before tracing is activated
+for the latter. Entirely managed by session daemons,
+consumer daemons survive session destruction to be reused later,
+should a new tracing session be created. Consumer daemons are always
+owned by the same user as their session daemon. When its owner session
+daemon is killed, the consumer daemon also exits. This is because
+the consumer daemon is always the child process of a session daemon.
+Consumer daemons should never be started manually. For this reason,
+they are not installed in one of the usual locations listed in the
+`PATH` environment variable. `lttng-sessiond` has, however, a
+bunch of options (see man:lttng-sessiond(8)) to
+specify custom consumer daemon paths if, for some reason, a consumer
+daemon other than the default installed one is needed.
+
+There are up to two running consumer daemons per user, whereas only one
+session daemon may run per user. This is because each process has
+independent bitness: if the target system runs a mixture of 32-bit and
+64-bit processes, it is more efficient to have separate corresponding
+32-bit and 64-bit consumer daemons. The `root` user is an exception: it
+may have up to _three_ running consumer daemons: 32-bit and 64-bit
+instances for its user space applications and one more reserved for
+collecting kernel trace data.
+
+As new tracing domains are added to LTTng, the development community's
+intent is to minimize the need for additionnal consumer daemon instances
+dedicated to them. For instance, the `java.util.logging` (JUL) domain
+events are in fact mapped to the user space domain, thus tracing this
+particular domain is handled by existing user space domain consumer
+daemons.
+
+
+[[lttng-relayd]]
+==== Relay daemon
+
+When a tracing session is configured to send its trace data over the
+network, an LTTng _relay daemon_ must be used at the other end to
+receive trace packets and serialize them to trace files. This setup
+makes it possible to trace a target system without ever committing trace
+data to its local storage, a feature which is useful for embedded
+systems, amongst others. The command implementing the relay daemon
+is `lttng-relayd`.
+
+The basic use case of `lttng-relayd` is to transfer trace data received
+over the network to trace files on the local file system. The relay
+daemon must listen on two TCP ports to achieve this: one control port,
+used by the target session daemon, and one data port, used by the
+target consumer daemon. The relay and session daemons agree on common
+default ports when custom ones are not specified.
+
+Since the communication transport protocol for both ports is standard
+TCP, the relay daemon may be started either remotely or locally (on the
+target system).
+
+While two instances of consumer daemons (32-bit and 64-bit) may run
+concurrently for a given user, `lttng-relayd` needs only be of its
+host operating system's bitness.
+
+The other important feature of LTTng's relay daemon is the support of
+_LTTng live_. LTTng live is an application protocol to view events as
+they arrive. The relay daemon will still record events in trace files,
+but a _tee_ may be created to inspect incoming events. Using LTTng live
+locally thus requires to run a local relay daemon.
+
+
+[[liblttng-ctl-lttng]]
+==== [[lttng-cli]]Control library and command line interface
+
+The LTTng control library, `liblttng-ctl`, can be used to communicate
+with the session daemon using a C API that hides the underlying
+protocol's details. `liblttng-ctl` is part of LTTng-tools.
+
+`liblttng-ctl` may be used by including its "master" header:
+
+[source,c]
+----
+#include <lttng/lttng.h>
+----
+
+Some objects are referred by name (C string), such as tracing sessions,
+but most of them require creating a handle first using
+`lttng_create_handle()`. The best available developer documentation for
+`liblttng-ctl` is, for the moment, its installed header files as such.
+Every function/structure is thoroughly documented.
+
+The `lttng` program is the _de facto_ standard user interface to
+control LTTng tracing sessions. `lttng` uses `liblttng-ctl` to
+communicate with session daemons behind the scenes.
+Its man page, man:lttng(1), is exhaustive, as well as its command
+line help (+lttng _cmd_ --help+, where +_cmd_+ is the command name).
+
+The <<controlling-tracing,Controlling tracing>> section is a feature
+tour of the `lttng` tool.
+
+
+[[lttng-ust]]
+==== User space tracing library
+
+The user space tracing part of LTTng is possible thanks to the user
+space tracing library, `liblttng-ust`, which is part of the LTTng-UST
+package.
+
+`liblttng-ust` provides header files containing macros used to define
+tracepoints and create tracepoint providers, as well as a shared object
+that must be linked to individual applications to connect to and
+communicate with a session daemon and a consumer daemon as soon as the
+application starts.
+
+The exact mechanism by which an application is registered to the
+session daemon is beyond the scope of this documentation. The only thing
+you need to know is that, since the library constructor does this job
+automatically, tracepoints may be safely inserted anywhere in the source
+code without prior manual initialization of `liblttng-ust`.
+
+The `liblttng-ust`-session daemon collaboration also provides an
+interesting feature: user space events may be enabled _before_
+applications actually start. By doing this and starting tracing before
+launching the instrumented application, you make sure that even the
+earliest occurring events can be recorded.
+
+The <<c-application,C application>> instrumenting guide of the
+<<using-lttng,Using LTTng>> chapter focuses on using `liblttng-ust`:
+instrumenting, building/linking and running a user application.
+
+
+[[lttng-modules]]
+==== LTTng kernel modules
+
+The LTTng Linux kernel modules provide everything needed to trace the
+Linux kernel: various probes, a ring buffer implementation for a
+consumer daemon to read trace data and the tracer itself.
+
+Only in exceptional circumstances should you ever need to load the
+LTTng kernel modules manually: it is normally the responsability of
+`root`'s session daemon to do so. If you were to develop your own LTTng
+probe module, however--for tracing a custom kernel or some kernel
+module (this topic is covered in the
+<<instrumenting-linux-kernel,Linux kernel>> instrumenting guide of
+the <<using-lttng,Using LTTng>> chapter)--you should either
+load it manually, or use the `--kmod-probes` option of the session
+daemon to load a specific list of kernel probes (beware, however,
+that the `--kmod-probes` option specifies an _absolute_ list, which
+means you also have to specify the default probes you need). The
+session and consumer daemons of regular users do not interact with the
+LTTng kernel modules at all.
+
+LTTng kernel modules are installed, by default, in
++/usr/lib/modules/_release_/extra+, where +_release_+ is the
+kernel release (see `uname --kernel-release`).
+
+
+[[using-lttng]]
+== Using LTTng
+
+Using LTTng involves two main activities: **instrumenting** and
+**controlling tracing**.
+
+_<<instrumenting,Instrumenting>>_ is the process of inserting probes
+into some source code. It can be done manually, by writing tracepoint
+calls at specific locations in the source code of the program to trace,
+or more automatically using dynamic probes (address in assembled code,
+symbol name, function entry/return, etc.).
+
+It has to be noted that, as an LTTng user, you may not have to worry
+about the instrumentation process. Indeed, you may want to trace a
+program already instrumented. As an example, the Linux kernel is
+thoroughly instrumented, which is why you can trace it without caring
+about adding probes.
+
+_<<controlling-tracing,Controlling tracing>>_ is everything
+that can be done by the LTTng session daemon, which is controlled using
+`liblttng-ctl` or its command line utility, `lttng`: creating tracing
+sessions, listing tracing sessions and events, enabling/disabling
+events, starting/stopping the tracers, taking snapshots, etc.
+
+This chapter is a complete user guide of both activities,
+with common use cases of LTTng exposed throughout the text. It is
+assumed that you are familiar with LTTng's concepts (events, channels,
+domains, tracing sessions) and that you understand the roles of its
+components (daemons, libraries, command line tools); if not, we invite
+you to read the <<understanding-lttng,Understanding LTTng>> chapter
+before you begin reading this one.
+
+If you're new to LTTng, we suggest that you rather start with the
+<<getting-started,Getting started>> small guide first, then come
+back here to broaden your knowledge.
+
+If you're only interested in tracing the Linux kernel with its current
+instrumentation, you may skip the
+<<instrumenting,Instrumenting>> section.
+
+
+[[instrumenting]]
+=== Instrumenting
+
+There are many examples of tracing and monitoring in our everyday life.
+You have access to real-time and historical weather reports and forecasts
+thanks to weather stations installed around the country. You know your
+possibly hospitalized friends' and family's hearts are safe thanks to
+electrocardiography. You make sure not to drive your car too fast
+and have enough fuel to reach your destination thanks to gauges visible
+on your dashboard.
+
+All the previous examples have something in common: they rely on
+**probes**. Without electrodes attached to the surface of a body's
+skin, cardiac monitoring would be futile.
+
+LTTng, as a tracer, is no different from the real life examples above.
+If you're about to trace a software system, i.e. record its history of
+execution, you better have probes in the subject you're
+tracing: the actual software. Various ways were developed to do this.
+The most straightforward one is to manually place probes, called
+_tracepoints_, in the software's source code. The Linux kernel tracing
+domain also allows probes added dynamically.
+
+If you're only interested in tracing the Linux kernel, it may very well
+be that your tracing needs are already appropriately covered by LTTng's
+built-in Linux kernel tracepoints and other probes. Or you may be in
+possession of a user space application which has already been
+instrumented. In such cases, the work will reside entirely in the design
+and execution of tracing sessions, allowing you to jump to
+<<controlling-tracing,Controlling tracing>> right now.
+
+This chapter focuses on the following use cases of instrumentation:
+
+* <<c-application,C>> and <<cxx-application,$$C++$$>> applications
+* <<prebuilt-ust-helpers,prebuilt user space tracing helpers>>
+* <<java-application,Java application>>
+* <<instrumenting-linux-kernel,Linux kernel>> module or the
+ kernel itself
+* the <<proc-lttng-logger-abi,path:{/proc/lttng-logger} ABI>>
+
+Some advanced techniques are also presented at the very end of this
+chapter.
+
+
+[[c-application]]
+==== C application
+
+Instrumenting a C (or $$C++$$) application, be it an executable program or
+a library, implies using LTTng-UST, the
+user space tracing component of LTTng. For C/$$C++$$ applications, the
+LTTng-UST package includes a dynamically loaded library
+(`liblttng-ust`), C headers and the `lttng-gen-tp` command line utility.
+
+Since C and $$C++$$ are the base languages of virtually all other
+programming languages
+(Java virtual machine, Python, Perl, PHP and Node.js interpreters, etc.),
+implementing user space tracing for an unsupported language is just a
+matter of using the LTTng-UST C API at the right places.
+
+The usual work flow to instrument a user space C application with
+LTTng-UST is:
+
+. Define tracepoints (actual probes)
+. Write tracepoint providers
+. Insert tracepoints into target source code
+. Package (build) tracepoint providers
+. Build user application and link it with tracepoint providers
+
+The steps above are discussed in greater detail in the following
+subsections.
+
+
+[[tracepoint-provider]]
+===== Tracepoint provider
+
+Before jumping into defining tracepoints and inserting
+them into the application source code, you must understand what a
+_tracepoint provider_ is.
+
+For the sake of this guide, consider the following two files:
+
+[source,c]
+.path:{tp.h}
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER my_provider
+
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./tp.h"
+
+#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _TP_H
+
+#include <lttng/tracepoint.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_first_tracepoint,
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_other_tracepoint,
+ TP_ARGS(
+ int, my_int
+ ),
+ TP_FIELDS(
+ ctf_integer(int, some_field, my_int)
+ )
+)
+
+#endif /* _TP_H */
+
+#include <lttng/tracepoint-event.h>
+----
+
+[source,c]
+.path:{tp.c}
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+The two files above are defining a _tracepoint provider_. A tracepoint
+provider is some sort of namespace for _tracepoint definitions_. Tracepoint
+definitions are written above with the `TRACEPOINT_EVENT()` macro, and allow
+eventual `tracepoint()` calls respecting their definitions to be inserted
+into the user application's C source code (we explore this in a
+later section).
+
+Many tracepoint definitions may be part of the same tracepoint provider
+and many tracepoint providers may coexist in a user space application. A
+tracepoint provider is packaged either:
+
+* directly into an existing user application's C source file
+* as an object file
+* as a static library
+* as a shared library
+
+The two files above, path:{tp.h} and path:{tp.c}, show a typical template for
+writing a tracepoint provider. LTTng-UST was designed so that two
+tracepoint providers should not be defined in the same header file.
+
+We will now go through the various parts of the above files and
+give them a meaning. As you may have noticed, the LTTng-UST API for
+C/$$C++$$ applications is some preprocessor sorcery. The LTTng-UST macros
+used in your application and those in the LTTng-UST headers are
+combined to produce actual source code needed to make tracing possible
+using LTTng.
+
+Let's start with the header file, path:{tp.h}. It begins with
+
+[source,c]
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER my_provider
+----
+
+`TRACEPOINT_PROVIDER` defines the name of the provider to which the
+following tracepoint definitions will belong. It is used internally by
+LTTng-UST headers and _must_ be defined. Since `TRACEPOINT_PROVIDER`
+could have been defined by another header file also included by the same
+C source file, the best practice is to undefine it first.
+
+NOTE: Names in LTTng-UST follow the C
+_identifier_ syntax (starting with a letter and containing either
+letters, numbers or underscores); they are _not_ C strings
+(not surrounded by double quotes). This is because LTTng-UST macros
+use those identifier-like strings to create symbols (named types and
+variables).
+
+The tracepoint provider is a group of tracepoint definitions; its chosen
+name should reflect this. A hierarchy like Java packages is recommended,
+using underscores instead of dots, e.g., `org_company_project_component`.
+
+Next is `TRACEPOINT_INCLUDE`:
+
+[source,c]
+----
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./tp.h"
+----
+
+This little bit of instrospection is needed by LTTng-UST to include
+your header at various predefined places.
+
+Include guard follows:
+
+[source,c]
+----
+#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _TP_H
+----
+
+Add these precompiler conditionals to ensure the tracepoint event
+generation can include this file more than once.
+
+The `TRACEPOINT_EVENT()` macro is defined in a LTTng-UST header file which
+must be included:
+
+[source,c]
+----
+#include <lttng/tracepoint.h>
+----
+
+This will also allow the application to use the `tracepoint()` macro.
+
+Next is a list of `TRACEPOINT_EVENT()` macro calls which create the
+actual tracepoint definitions. We will skip this for the moment and
+come back to how to use `TRACEPOINT_EVENT()`
+<<defining-tracepoints,in a later section>>. Just pay attention to
+the first argument: it's always the name of the tracepoint provider
+being defined in this header file.
+
+End of include guard:
+
+[source,c]
+----
+#endif /* _TP_H */
+----
+
+Finally, include `<lttng/tracepoint-event.h>` to expand the macros:
+
+[source,c]
+----
+#include <lttng/tracepoint-event.h>
+----
+
+That's it for path:{tp.h}. Of course, this is only a header file; it must be
+included in some C source file to actually use it. This is the job of
+path:{tp.c}:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+When `TRACEPOINT_CREATE_PROBES` is defined, the macros used in path:{tp.h},
+which is included just after, will actually create the source code for
+LTTng-UST probes (global data structures and functions) out of your
+tracepoint definitions. How exactly this is done is out of this text's scope.
+`TRACEPOINT_CREATE_PROBES` is discussed further
+in
+<<building-tracepoint-providers-and-user-application,Building/linking
+tracepoint providers and the user application>>.
+
+You could include other header files like path:{tp.h} here to create the probes
+of different tracepoint providers, e.g.:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp1.h"
+#include "tp2.h"
+----
+
+The rule is: probes of a given tracepoint provider
+must be created in exactly one source file. This source file could be one
+of your project's; it doesn't have to be on its own like
+path:{tp.c}, although
+<<building-tracepoint-providers-and-user-application,a later section>>
+shows that doing so allows packaging the tracepoint providers
+independently and keep them out of your application, also making it
+possible to reuse them between projects.
+
+The following sections explain how to define tracepoints, how to use the
+`tracepoint()` macro to instrument your user space C application and how
+to build/link tracepoint providers and your application with LTTng-UST
+support.
+
+
+[[lttng-gen-tp]]
+===== Using `lttng-gen-tp`
+
+LTTng-UST ships with `lttng-gen-tp`, a handy command line utility for
+generating most of the stuff discussed above. It takes a _template file_,
+with a name usually ending with the `.tp` extension, containing only
+tracepoint definitions, and outputs a tracepoint provider (either a C
+source file or a precompiled object file) with its header file.
+
+`lttng-gen-tp` should suffice in <<static-linking,static linking>>
+situations. When using it, write a template file containing a list of
+`TRACEPOINT_EVENT()` macro calls. The tool will find the provider names
+used and generate the appropriate files which are going to look a lot
+like path:{tp.h} and path:{tp.c} above.
+
+Just call `lttng-gen-tp` like this:
+
+[role="term"]
+----
+lttng-gen-tp my-template.tp
+----
+
+path:{my-template.c}, path:{my-template.o} and path:{my-template.h}
+will be created in the same directory.
+
+You may specify custom C flags passed to the compiler invoked by
+`lttng-gen-tp` using the `CFLAGS` environment variable:
+
+[role="term"]
+----
+CFLAGS=-I/custom/include/path lttng-gen-tp my-template.tp
+----
+
+For more information on `lttng-gen-tp`, see man:lttng-gen-tp(1).
+
+
+[[defining-tracepoints]]
+===== Defining tracepoints
+
+As written in <<tracepoint-provider,Tracepoint provider>>,
+tracepoints are defined using the
+`TRACEPOINT_EVENT()` macro. Each tracepoint, when called using the
+`tracepoint()` macro in the actual application's source code, generates
+a specific event type with its own fields.
+
+Let's have another look at the example above, with a few added comments:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ /* tracepoint provider name */
+ my_provider,
+
+ /* tracepoint/event name */
+ my_first_tracepoint,
+
+ /* list of tracepoint arguments */
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+
+ /* list of fields of eventual event */
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+----
+
+The tracepoint provider name must match the name of the tracepoint
+provider in which this tracepoint is defined
+(see <<tracepoint-provider,Tracepoint provider>>). In other words,
+always use the same string as the value of `TRACEPOINT_PROVIDER` above.
+
+The tracepoint name will become the event name once events are recorded
+by the LTTng-UST tracer. It must follow the tracepoint provider name
+syntax: start with a letter and contain either letters, numbers or
+underscores. Two tracepoints under the same provider cannot have the
+same name, i.e. you cannot overload a tracepoint like you would
+overload functions and methods in $$C++$$/Java.
+
+NOTE: The concatenation of the tracepoint
+provider name and the tracepoint name cannot exceed 254 characters. If
+it does, the instrumented application will compile and run, but LTTng
+will issue multiple warnings and you could experience serious problems.
+
+The list of tracepoint arguments gives this tracepoint its signature:
+see it like the declaration of a C function. The format of `TP_ARGS()`
+arguments is: C type, then argument name; repeat as needed, up to ten
+times. For example, if we were to replicate the signature of C standard
+library's `fseek()`, the `TP_ARGS()` part would look like:
+
+[source,c]
+----
+ TP_ARGS(
+ FILE*, stream,
+ long int, offset,
+ int, origin
+ ),
+----
+
+Of course, you will need to include appropriate header files before
+the `TRACEPOINT_EVENT()` macro calls if any argument has a complex type.
+
+`TP_ARGS()` may not be omitted, but may be empty. `TP_ARGS(void)` is
+also accepted.
+
+The list of fields is where the fun really begins. The fields defined
+in this list will be the fields of the events generated by the execution
+of this tracepoint. Each tracepoint field definition has a C
+_argument expression_ which will be evaluated when the execution reaches
+the tracepoint. Tracepoint arguments _may be_ used freely in those
+argument expressions, but they _don't_ have to.
+
+There are several types of tracepoint fields available. The macros to
+define them are given and explained in the
+<<liblttng-ust-tp-fields,LTTng-UST library reference>> section.
+
+Field names must follow the standard C identifier syntax: letter, then
+optional sequence of letters, numbers or underscores. Each field must have
+a different name.
+
+Those `ctf_*()` macros are added to the `TP_FIELDS()` part of
+`TRACEPOINT_EVENT()`. Note that they are not delimited by commas.
+`TP_FIELDS()` may be empty, but the `TP_FIELDS(void)` form is _not_
+accepted.
+
+The following snippet shows how argument expressions may be used in
+tracepoint fields and how they may refer freely to tracepoint arguments.
+
+[source,c]
+----
+/* for struct stat */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, my_int_arg,
+ char*, my_str_arg,
+ struct stat*, st
+ ),
+ TP_FIELDS(
+ /* simple integer field with constant value */
+ ctf_integer(
+ int, /* field C type */
+ my_constant_field, /* field name */
+ 23 + 17 /* argument expression */
+ )
+
+ /* my_int_arg tracepoint argument */
+ ctf_integer(
+ int,
+ my_int_arg_field,
+ my_int_arg
+ )
+
+ /* my_int_arg squared */
+ ctf_integer(
+ int,
+ my_int_arg_field2,
+ my_int_arg * my_int_arg
+ )
+
+ /* sum of first 4 characters of my_str_arg */
+ ctf_integer(
+ int,
+ sum4,
+ my_str_arg[0] + my_str_arg[1] +
+ my_str_arg[2] + my_str_arg[3]
+ )
+
+ /* my_str_arg as string field */
+ ctf_string(
+ my_str_arg_field, /* field name */
+ my_str_arg /* argument expression */
+ )
+
+ /* st_size member of st tracepoint argument, hexadecimal */
+ ctf_integer_hex(
+ off_t, /* field C type */
+ size_field, /* field name */
+ st->st_size /* argument expression */
+ )
+
+ /* st_size member of st tracepoint argument, as double */
+ ctf_float(
+ double, /* field C type */
+ size_dbl_field, /* field name */
+ (double) st->st_size /* argument expression */
+ )
+
+ /* half of my_str_arg string as text sequence */
+ ctf_sequence_text(
+ char, /* element C type */
+ half_my_str_arg_field, /* field name */
+ my_str_arg, /* argument expression */
+ size_t, /* length expression C type */
+ strlen(my_str_arg) / 2 /* length expression */
+ )
+ )
+)
+----
+
+As you can see, having a custom argument expression for each field
+makes tracepoints very flexible for tracing a user space C application.
+This tracepoint definition is reused later in this guide, when
+actually using tracepoints in a user space application.
+
+
+[[using-tracepoint-classes]]
+===== Using tracepoint classes
+
+In LTTng-UST, a _tracepoint class_ is a class of tracepoints sharing the
+same field types and names. A _tracepoint instance_ is one instance of
+such a declared tracepoint class, with its own event name and tracepoint
+provider name.
+
+What is documented in <<defining-tracepoints,Defining tracepoints>>
+is actually how to declare a _tracepoint class_ and define a
+_tracepoint instance_ at the same time. Without revealing the internals
+of LTTng-UST too much, it has to be noted that one serialization
+function is created for each tracepoint class. A serialization
+function is responsible for serializing the fields of a tracepoint
+into a sub-buffer when tracing. For various performance reasons, when
+your situation requires multiple tracepoints with different names, but
+with the same fields layout, the best practice is to manually create
+a tracepoint class and instantiate as many tracepoint instances as
+needed. One positive effect of such a design, amongst other advantages,
+is that all tracepoint instances of the same tracepoint class will
+reuse the same serialization function, thus reducing cache pollution.
+
+As an example, here are three tracepoint definitions as we know them:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ my_app,
+ get_account,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+----
+
+In this case, three tracepoint classes are created, with one tracepoint
+instance for each of them: `get_account`, `get_settings` and
+`get_transaction`. However, they all share the same field names and
+types. Declaring one tracepoint class and three tracepoint instances of
+the latter is a better design choice:
+
+[source,c]
+----
+/* the tracepoint class */
+TRACEPOINT_EVENT_CLASS(
+ /* tracepoint provider name */
+ my_app,
+
+ /* tracepoint class name */
+ my_class,
+
+ /* arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+
+ /* fields */
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+/* the tracepoint instances */
+TRACEPOINT_EVENT_INSTANCE(
+ /* tracepoint provider name */
+ my_app,
+
+ /* tracepoint class name */
+ my_class,
+
+ /* tracepoint/event name */
+ get_account,
+
+ /* arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+----
+
+Of course, all those names and `TP_ARGS()` invocations are redundant,
+but some C preprocessor magic can solve this:
+
+[source,c]
+----
+#define MY_TRACEPOINT_ARGS \
+ TP_ARGS( \
+ int, userid, \
+ size_t, len \
+ )
+
+TRACEPOINT_EVENT_CLASS(
+ my_app,
+ my_class,
+ MY_TRACEPOINT_ARGS,
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+#define MY_APP_TRACEPOINT_INSTANCE(name) \
+ TRACEPOINT_EVENT_INSTANCE( \
+ my_app, \
+ my_class, \
+ name, \
+ MY_TRACEPOINT_ARGS \
+ )
+
+MY_APP_TRACEPOINT_INSTANCE(get_account)
+MY_APP_TRACEPOINT_INSTANCE(get_settings)
+MY_APP_TRACEPOINT_INSTANCE(get_transaction)
+----
+
+
+[[assigning-log-levels]]
+===== Assigning log levels to tracepoints
+
+Optionally, a log level can be assigned to a defined tracepoint.
+Assigning different levels of importance to tracepoints can be useful;
+when controlling tracing sessions,
+<<controlling-tracing,you can choose>> to only enable tracepoints
+falling into a specific log level range.
+
+Log levels are assigned to defined tracepoints using the
+`TRACEPOINT_LOGLEVEL()` macro. The latter must be used _after_ having
+used `TRACEPOINT_EVENT()` for a given tracepoint. The
+`TRACEPOINT_LOGLEVEL()` macro has the following construct:
+
+[source,c]
+----
+TRACEPOINT_LOGLEVEL(PROVIDER_NAME, TRACEPOINT_NAME, LOG_LEVEL)
+----
+
+where the first two arguments are the same as the first two arguments
+of `TRACEPOINT_EVENT()` and `LOG_LEVEL` is one
+of the values given in the
+<<liblttng-ust-tracepoint-loglevel,LTTng-UST library reference>>
+section.
+
+As an example, let's assign a `TRACE_DEBUG_UNIT` log level to our
+previous tracepoint definition:
+
+[source,c]
+----
+TRACEPOINT_LOGLEVEL(my_provider, my_tracepoint, TRACE_DEBUG_UNIT)
+----
+
+
+[[probing-the-application-source-code]]
+===== Probing the application's source code
+
+Once tracepoints are properly defined within a tracepoint provider,
+they may be inserted into the user application to be instrumented
+using the `tracepoint()` macro. Its first argument is the tracepoint
+provider name and its second is the tracepoint name. The next, optional
+arguments are defined by the `TP_ARGS()` part of the definition of
+the tracepoint to use.
+
+As an example, let us again take the following tracepoint definition:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ /* tracepoint provider name */
+ my_provider,
+
+ /* tracepoint/event name */
+ my_first_tracepoint,
+
+ /* list of tracepoint arguments */
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+
+ /* list of fields of eventual event */
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+----
+
+Assuming this is part of a file named path:{tp.h} which defines the tracepoint
+provider and which is included by path:{tp.c}, here's a complete C application
+calling this tracepoint (multiple times):
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+int main(int argc, char* argv[])
+{
+ int i;
+
+ tracepoint(my_provider, my_first_tracepoint, 23, "Hello, World!");
+
+ for (i = 0; i < argc; ++i) {
+ tracepoint(my_provider, my_first_tracepoint, i, argv[i]);
+ }
+
+ return 0;
+}
+----
+
+For each tracepoint provider, `TRACEPOINT_DEFINE` must be defined into
+exactly one translation unit (C source file) of the user application,
+before including the tracepoint provider header file. In other words,
+for a given tracepoint provider, you cannot define `TRACEPOINT_DEFINE`,
+and then include its header file in two separate C source files of
+the same application. `TRACEPOINT_DEFINE` is discussed further in
+<<building-tracepoint-providers-and-user-application,Building/linking
+tracepoint providers and the user application>>.
+
+As another example, remember this definition we wrote in a previous
+section (comments are stripped):
+
+[source,c]
+----
+/* for struct stat */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, my_int_arg,
+ char*, my_str_arg,
+ struct stat*, st
+ ),
+ TP_FIELDS(
+ ctf_integer(int, my_constant_field, 23 + 17)
+ ctf_integer(int, my_int_arg_field, my_int_arg)
+ ctf_integer(int, my_int_arg_field2, my_int_arg * my_int_arg)
+ ctf_integer(int, sum4_field, my_str_arg[0] + my_str_arg[1] +
+ my_str_arg[2] + my_str_arg[3])
+ ctf_string(my_str_arg_field, my_str_arg)
+ ctf_integer_hex(off_t, size_field, st->st_size)
+ ctf_float(double, size_dbl_field, (double) st->st_size)
+ ctf_sequence_text(char, half_my_str_arg_field, my_str_arg,
+ size_t, strlen(my_str_arg) / 2)
+ )
+)
+----
+
+Here's an example of calling it:
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+int main(void)
+{
+ struct stat s;
+
+ stat("/etc/fstab", &s);
+
+ tracepoint(my_provider, my_tracepoint, 23, "Hello, World!", &s);
+
+ return 0;
+}
+----
+
+When viewing the trace, assuming the file size of path:{/etc/fstab} is
+301{nbsp}bytes, the event generated by the execution of this tracepoint
+should have the following fields, in this order:
+
+----
+my_constant_field 40
+my_int_arg_field 23
+my_int_arg_field2 529
+sum4_field 389
+my_str_arg_field "Hello, World!"
+size_field 0x12d
+size_dbl_field 301.0
+half_my_str_arg_field "Hello,"
+----
+
+
+[[building-tracepoint-providers-and-user-application]]
+===== Building/linking tracepoint providers and the user application
+
+The final step of using LTTng-UST for tracing a user space C application
+(beside running the application) is building and linking tracepoint
+providers and the application itself.
+
+As discussed above, the macros used by the user-written tracepoint provider
+header file are useless until actually used to create probes code
+(global data structures and functions) in a translation unit (C source file).
+This is accomplished by defining `TRACEPOINT_CREATE_PROBES` in a translation
+unit and then including the tracepoint provider header file.
+When `TRACEPOINT_CREATE_PROBES` is defined, macros used and included by
+the tracepoint provider header will output actual source code needed by any
+application using the defined tracepoints. Defining
+`TRACEPOINT_CREATE_PROBES` produces code used when registering
+tracepoint providers when the tracepoint provider package loads.
+
+The other important definition is `TRACEPOINT_DEFINE`. This one creates
+global, per-tracepoint structures referencing the tracepoint providers
+data. Those structures are required by the actual functions inserted
+where `tracepoint()` macros are placed and need to be defined by the
+instrumented application.
+
+Both `TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` need to be defined
+at some places in order to trace a user space C application using LTTng.
+Although explaining their exact mechanism is beyond the scope of this
+document, the reason they both exist separately is to allow the trace
+providers to be packaged as a shared object (dynamically loaded library).
+
+There are two ways to compile and link the tracepoint providers
+with the application: _<<static-linking,statically>>_ or
+_<<dynamic-linking,dynamically>>_. Both methods are covered in the
+following subsections.
+
+
+[[static-linking]]
+===== Static linking the tracepoint providers to the application
+
+With the static linking method, compiled tracepoint providers are copied
+into the target application. There are three ways to do this:
+
+. Use one of your **existing C source files** to create probes.
+. Create probes in a separate C source file and build it as an
+ **object file** to be linked with the application (more decoupled).
+. Create probes in a separate C source file, build it as an
+ object file and archive it to create a **static library**
+ (more decoupled, more portable).
+
+The first approach is to define `TRACEPOINT_CREATE_PROBES` and include
+your tracepoint provider(s) header file(s) directly into an existing C
+source file. Here's an example:
+
+[source,c]
+----
+#include <stdlib.h>
+#include <stdio.h>
+/* ... */
+
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+/* ... */
+
+int my_func(int a, const char* b)
+{
+ /* ... */
+
+ tracepoint(my_provider, my_tracepoint, buf, sz, limit, &tt)
+
+ /* ... */
+}
+
+/* ... */
+----
+
+Again, before including a given tracepoint provider header file,
+`TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` must be defined in
+one, **and only one**, translation unit. Other C source files of the
+same application may include path:{tp.h} to use tracepoints with
+the `tracepoint()` macro, but must not define
+`TRACEPOINT_CREATE_PROBES`/`TRACEPOINT_DEFINE` again.
+
+This translation unit may be built as an object file by making sure to
+add `.` to the include path:
+
+[role="term"]
+----
+gcc -c -I. file.c
+----
+
+The second approach is to isolate the tracepoint provider code into a
+separate object file by using a dedicated C source file to create probes:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+`TRACEPOINT_DEFINE` must be defined by a translation unit of the
+application. Since we're talking about static linking here, it could as
+well be defined directly in the file above, before `#include "tp.h"`:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+
+#include "tp.h"
+----
+
+This is actually what <<lttng-gen-tp,`lttng-gen-tp`>> does, and is
+the recommended practice.
+
+Build the tracepoint provider:
+
+[role="term"]
+----
+gcc -c -I. tp.c
+----
+
+Finally, the resulting object file may be archived to create a
+more portable tracepoint provider static library:
+
+[role="term"]
+----
+ar rc tp.a tp.o
+----
+
+Using a static library does have the advantage of centralising the
+tracepoint providers objects so they can be shared between multiple
+applications. This way, when the tracepoint provider is modified, the
+source code changes don't have to be patched into each application's source
+code tree. The applications need to be relinked after each change, but need
+not to be otherwise recompiled (unless the tracepoint provider's API
+changes).
+
+Regardless of which method you choose, you end up with an object file
+(potentially archived) containing the trace providers assembled code.
+To link this code with the rest of your application, you must also link
+with `liblttng-ust` and `libdl`:
+
+[role="term"]
+----
+gcc -o app tp.o other.o files.o of.o your.o app.o -llttng-ust -ldl
+----
+
+or
+
+[role="term"]
+----
+gcc -o app tp.a other.o files.o of.o your.o app.o -llttng-ust -ldl
+----
+
+If you're using a BSD
+system, replace `-ldl` with `-lc`:
+
+[role="term"]
+----
+gcc -o app tp.a other.o files.o of.o your.o app.o -llttng-ust -lc
+----
+
+The application can be started as usual, e.g.:
+
+[role="term"]
+----
+./app
+----
+
+The `lttng` command line tool can be used to
+<<controlling-tracing,control tracing>>.
+
+
+[[dynamic-linking]]
+===== Dynamic linking the tracepoint providers to the application
+
+The second approach to package the tracepoint providers is to use
+dynamic linking: the library and its member functions are explicitly
+sought, loaded and unloaded at runtime using `libdl`.
+
+It has to be noted that, for a variety of reasons, the created shared
+library will be dynamically _loaded_, as opposed to dynamically
+_linked_. The tracepoint provider shared object is, however, linked
+with `liblttng-ust`, so that `liblttng-ust` is guaranteed to be loaded
+as soon as the tracepoint provider is. If the tracepoint provider is
+not loaded, since the application itself is not linked with
+`liblttng-ust`, the latter is not loaded at all and the tracepoint calls
+become inert.
+
+The process to create the tracepoint provider shared object is pretty
+much the same as the static library method, except that:
+
+* since the tracepoint provider is not part of the application
+ anymore, `TRACEPOINT_DEFINE` _must_ be defined, for each tracepoint
+ provider, in exactly one translation unit (C source file) of the
+ _application_;
+* `TRACEPOINT_PROBE_DYNAMIC_LINKAGE` must be defined next to
+ `TRACEPOINT_DEFINE`.
+
+Regarding `TRACEPOINT_DEFINE` and `TRACEPOINT_PROBE_DYNAMIC_LINKAGE`,
+the recommended practice is to use a separate C source file in your
+application to define them, and then include the tracepoint provider
+header files afterwards, e.g.:
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+
+/* include the header files of one or more tracepoint providers below */
+#include "tp1.h"
+#include "tp2.h"
+#include "tp3.h"
+----
+
+`TRACEPOINT_PROBE_DYNAMIC_LINKAGE` makes the macros included afterwards
+(by including the tracepoint provider header, which itself includes
+LTTng-UST headers) aware that the tracepoint provider is to be loaded
+dynamically and not part of the application's executable.
+
+The tracepoint provider object file used to create the shared library
+is built like it is using the static library method, only with the
+`-fpic` option added:
+
+[role="term"]
+----
+gcc -c -fpic -I. tp.c
+----
+
+It is then linked as a shared library like this:
+
+[role="term"]
+----
+gcc -shared -Wl,--no-as-needed -o tp.so -llttng-ust tp.o
+----
+
+As previously stated, this tracepoint provider shared object isn't
+linked with the user application: it will be loaded manually. This is
+why the application is built with no mention of this tracepoint
+provider, but still needs `libdl`:
+
+[role="term"]
+----
+gcc -o app other.o files.o of.o your.o app.o -ldl
+----
+
+Now, to make LTTng-UST tracing available to the application, the
+`LD_PRELOAD` environment variable is used to preload the tracepoint
+provider shared library _before_ the application actually starts:
+
+[role="term"]
+----
+LD_PRELOAD=/path/to/tp.so ./app
+----
+
+[NOTE]
+====
+It is not safe to use
+`dlclose()` on a tracepoint provider shared object that
+is being actively used for tracing, due to a lack of reference
+counting from LTTng-UST to the shared object.
+
+For example, statically linking a tracepoint provider to a
+shared object which is to be dynamically loaded by an application
+(e.g., a plugin) is not safe: the shared object, which contains the
+tracepoint provider, could be dynamically closed
+(`dlclose()`) at any time by the application.
+
+To instrument a shared object, either:
+
+* Statically link the tracepoint provider to the _application_, or
+* Build the tracepoint provider as a shared object (following
+ the procedure shown in this section), and preload it when
+ tracing is needed using the `LD_PRELOAD`
+ environment variable.
+====
+
+Your application will still work without this preloading, albeit without
+LTTng-UST tracing support:
+
+[role="term"]
+----
+./app
+----
+
+
+[[using-lttng-ust-with-daemons]]
+===== Using LTTng-UST with daemons
+
+Some extra care is needed when using `liblttng-ust` with daemon
+applications that call `fork()`, `clone()` or BSD's `rfork()` without
+a following `exec()` family system call. The `liblttng-ust-fork`
+library must be preloaded for the application.
+
+Example:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-fork.so ./app
+----
+
+Or, if you're using a tracepoint provider shared library:
+
+[role="term"]
+----
+LD_PRELOAD="liblttng-ust-fork.so /path/to/tp.so" ./app
+----
+
+
+[[lttng-ust-pkg-config]]
+===== Using pkg-config
+
+On some distributions, LTTng-UST is shipped with a pkg-config metadata
+file, so that you may use the `pkg-config` tool:
+
+[role="term"]
+----
+pkg-config --libs lttng-ust
+----
+
+This will return `-llttng-ust -ldl` on Linux systems.
+
+You may also check the LTTng-UST version using `pkg-config`:
+
+[role="term"]
+----
+pkg-config --modversion lttng-ust
+----
+
+For more information about pkg-config, see
+http://linux.die.net/man/1/pkg-config[its manpage].
+
+
+[[tracef]]
+===== Using `tracef()`
+
+`tracef()` is a small LTTng-UST API to avoid defining your own
+tracepoints and tracepoint providers. The signature of `tracef()` is
+the same as `printf()`'s.
+
+The `tracef()` utility function was developed to make user space tracing
+super simple, albeit with notable disadvantages compared to custom,
+full-fledged tracepoint providers:
+
+* All generated events have the same provider/event names, respectively
+ `lttng_ust_tracef` and `event`.
+* There's no static type checking.
+* The only event field you actually get, named `msg`, is a string
+ potentially containing the values you passed to the function
+ using your own format. This also means that you cannot use filtering
+ using a custom expression at runtime because there are no isolated
+ fields.
+* Since `tracef()` uses C standard library's `vasprintf()` function
+ in the background to format the strings at runtime, its
+ expected performance is lower than using custom tracepoint providers
+ with typed fields, which do not require a conversion to a string.
+
+Thus, `tracef()` is useful for quick prototyping and debugging, but
+should not be considered for any permanent/serious application
+instrumentation.
+
+To use `tracef()`, first include `<lttng/tracef.h>` in the C source file
+where you need to insert probes:
+
+[source,c]
+----
+#include <lttng/tracef.h>
+----
+
+Use `tracef()` like you would use `printf()` in your source code, e.g.:
+
+[source,c]
+----
+ /* ... */
+
+ tracef("my message, my integer: %d", my_integer);
+
+ /* ... */
+----
+
+Link your application with `liblttng-ust`:
+
+[role="term"]
+----
+gcc -o app app.c -llttng-ust
+----
+
+Execute the application as usual:
+
+[role="term"]
+----
+./app
+----
+
+Voilà! Use the `lttng` command line tool to
+<<controlling-tracing,control tracing>>. You can enable `tracef()`
+events like this:
+
+[role="term"]
+----
+lttng enable-event --userspace 'lttng_ust_tracef:*'
+----
+
+
+[[lttng-ust-environment-variables-compiler-flags]]
+===== LTTng-UST environment variables and special compilation flags
+
+A few special environment variables and compile flags may affect the
+behavior of LTTng-UST.
+
+LTTng-UST's debugging can be activated by setting the environment
+variable `LTTNG_UST_DEBUG` to `1` when launching the application. It
+can also be enabled at compile time by defining `LTTNG_UST_DEBUG` when
+compiling LTTng-UST (using the `-DLTTNG_UST_DEBUG` compiler option).
+
+The environment variable `LTTNG_UST_REGISTER_TIMEOUT` can be used to
+specify how long the application should wait for the
+<<lttng-sessiond,session daemon>>'s _registration done_ command
+before proceeding to execute the main program. The timeout value is
+specified in milliseconds. 0 means _don't wait_. -1 means
+_wait forever_. Setting this environment variable to 0 is recommended
+for applications with time contraints on the process startup time.
+
+The default value of `LTTNG_UST_REGISTER_TIMEOUT` (when not defined)
+is **3000{nbsp}ms**.
+
+The compilation definition `LTTNG_UST_DEBUG_VALGRIND` should be enabled
+at build time (`-DLTTNG_UST_DEBUG_VALGRIND`) to allow `liblttng-ust`
+to be used with http://valgrind.org/[Valgrind].
+The side effect of defining `LTTNG_UST_DEBUG_VALGRIND` is that per-CPU
+buffering is disabled.
+
+
+[[cxx-application]]
+==== $$C++$$ application
+
+Because of $$C++$$'s cross-compatibility with the C language, $$C++$$
+applications can be readily instrumented with the LTTng-UST C API.
+
+Follow the <<c-application,C application>> user guide above. It
+should be noted that, in this case, tracepoint providers should have
+the typical `.cpp`, `.cxx` or `.cc` extension and be built with `g++`
+instead of `gcc`. This is the easiest way of avoiding linking errors
+due to symbol name mangling incompatibilities between both languages.
+
+
+[[prebuilt-ust-helpers]]
+==== Prebuilt user space tracing helpers
+
+The LTTng-UST package provides a few helpers that one may find
+useful in some situations. They all work the same way: you must
+preload the appropriate shared object before running the user
+application (using the `LD_PRELOAD` environment variable).
+
+The shared objects are normally found in dir:{/usr/lib}.
+
+The current installed helpers are:
+
+path:{liblttng-ust-libc-wrapper.so} and path:{liblttng-ust-pthread-wrapper.so}::
+ <<liblttng-ust-libc-pthread-wrapper,C{nbsp}standard library
+ and POSIX threads tracing>>.
+
+path:{liblttng-ust-cyg-profile.so} and path:{liblttng-ust-cyg-profile-fast.so}::
+ <<liblttng-ust-cyg-profile,Function tracing>>.
+
+path:{liblttng-ust-dl.so}::
+ <<liblttng-ust-dl,Dynamic linker tracing>>.
+
+The following subsections document what helpers instrument exactly
+and how to use them.
+
+
+[[liblttng-ust-libc-pthread-wrapper]]
+===== C standard library and POSIX threads tracing
+
+path:{liblttng-ust-libc-wrapper.so} and path:{liblttng-ust-pthread-wrapper.so}
+can add instrumentation to respectively some C standard library and
+POSIX threads functions.
+
+The following functions are traceable by path:{liblttng-ust-libc-wrapper.so}:
+
+[role="growable"]
+.Functions instrumented by path:{liblttng-ust-libc-wrapper.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.6+|`ust_libc` |`malloc` |`malloc()`
+ |`calloc` |`calloc()`
+ |`realloc` |`realloc()`
+ |`free` |`free()`
+ |`memalign` |`memalign()`
+ |`posix_memalign` |`posix_memalign()`
+|====
+
+The following functions are traceable by
+path:{liblttng-ust-pthread-wrapper.so}:
+
+[role="growable"]
+.Functions instrumented by path:{liblttng-ust-pthread-wrapper.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.4+|`ust_pthread` |`pthread_mutex_lock_req` |`pthread_mutex_lock()` (request time)
+ |`pthread_mutex_lock_acq` |`pthread_mutex_lock()` (acquire time)
+ |`pthread_mutex_trylock` |`pthread_mutex_trylock()`
+ |`pthread_mutex_unlock` |`pthread_mutex_unlock()`
+|====
+
+All tracepoints have fields corresponding to the arguments of the
+function they instrument.
+
+To use one or the other with any user application, independently of
+how the latter is built, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
+----
+
+or
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-pthread-wrapper.so my-app
+----
+
+To use both, do:
+
+[role="term"]
+----
+LD_PRELOAD="liblttng-ust-libc-wrapper.so liblttng-ust-pthread-wrapper.so" my-app
+----
+
+When the shared object is preloaded, it effectively replaces the
+functions listed in the above tables by wrappers which add tracepoints
+and call the replaced functions.
+
+Of course, like any other tracepoint, the ones above need to be enabled
+in order for LTTng-UST to generate events. This is done using the
+`lttng` command line tool
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[[liblttng-ust-cyg-profile]]
+===== Function tracing
+
+Function tracing is the recording of which functions are entered and
+left during the execution of an application. Like with any LTTng event,
+the precise time at which this happens is also kept.
+
+GCC and clang have an option named
+https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/Code-Gen-Options.html[`-finstrument-functions`]
+which generates instrumentation calls for entry and exit to functions.
+The LTTng-UST function tracing helpers, path:{liblttng-ust-cyg-profile.so}
+and path:{liblttng-ust-cyg-profile-fast.so}, take advantage of this feature
+to add instrumentation to the two generated functions (which contain
+`cyg_profile` in their names, hence the shared object's name).
+
+In order to use LTTng-UST function tracing, the translation units to
+instrument must be built using the `-finstrument-functions` compiler
+flag.
+
+LTTng-UST function tracing comes in two flavors, each providing
+different trade-offs: path:{liblttng-ust-cyg-profile-fast.so} and
+path:{liblttng-ust-cyg-profile.so}.
+
+**path:{liblttng-ust-cyg-profile-fast.so}** is a lightweight variant that
+should only be used where it can be _guaranteed_ that the complete event
+stream is recorded without any missing events. Any kind of duplicate
+information is left out. This version registers the following
+tracepoints:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-cyg-profile-fast.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`lttng_ust_cyg_profile_fast`
+
+|`func_entry`
+a|Function entry
+
+`addr`::
+ Address of called function.
+
+|`func_exit`
+|Function exit
+|====
+
+Assuming no event is lost, having only the function addresses on entry
+is enough for creating a call graph (remember that a recorded event
+always contains the ID of the CPU that generated it). A tool like
+https://sourceware.org/binutils/docs/binutils/addr2line.html[`addr2line`]
+may be used to convert function addresses back to source files names
+and line numbers.
+
+The other helper,
+**path:{liblttng-ust-cyg-profile.so}**,
+is a more robust variant which also works for use cases where
+events might get discarded or not recorded from application startup.
+In these cases, the trace analyzer needs extra information to be
+able to reconstruct the program flow. This version registers the
+following tracepoints:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-cyg-profile.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`lttng_ust_cyg_profile`
+
+|`func_entry`
+a|Function entry
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+
+|`func_exit`
+a|Function exit
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+|====
+
+To use one or the other variant with any user application, assuming at
+least one translation unit of the latter is compiled with the
+`-finstrument-functions` option, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-cyg-profile-fast.so my-app
+----
+
+or
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-cyg-profile.so my-app
+----
+
+It might be necessary to limit the number of source files where
+`-finstrument-functions` is used to prevent excessive amount of trace
+data to be generated at runtime.
+
+TIP: When using GCC, at least, you can use
+ the `-finstrument-functions-exclude-function-list`
+ option to avoid instrumenting entries and exits of specific
+ symbol names.
+
+All events generated from LTTng-UST function tracing are provided on
+log level `TRACE_DEBUG_FUNCTION`, which is useful to easily enable
+function tracing events in your tracing session using the
+`--loglevel-only` option of `lttng enable-event`
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[[liblttng-ust-dl]]
+===== Dynamic linker tracing
+
+This LTTng-UST helper causes all calls to `dlopen()` and `dlclose()`
+in the target application to be traced with LTTng.
+
+The helper's shared object, path:{liblttng-ust-dl.so}, registers the
+following tracepoints when preloaded:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-dl.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`ust_baddr`
+
+|`push`
+a|`dlopen()` call
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+
+`sopath`::
+ File system path to the loaded shared object.
+
+`size`::
+ File size of the the loaded shared object.
+
+`mtime`::
+ Last modification time (seconds since Epoch time) of the loaded shared
+ object.
+
+|`pop`
+a|Function exit
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+|====
+
+To use this LTTng-UST helper with any user application, independently of
+how the latter is built, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-dl.so my-app
+----
+
+Of course, like any other tracepoint, the ones above need to be enabled
+in order for LTTng-UST to generate events. This is done using the
+`lttng` command line tool
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[[java-application]]
+==== Java application
+
+LTTng-UST provides a _logging_ back-end for Java applications using
+http://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html[`java.util.logging`]
+(JUL). This back-end is called the _LTTng-UST JUL agent_ and is
+responsible for communications with an LTTng session daemon.
+
+From the user's point of view, once the LTTng-UST JUL agent has been
+initialized, JUL loggers may be created and used as usual. The agent
+adds its own handler to the _root logger_, so that all loggers may
+generate LTTng events with no effort.
+
+Common JUL features are supported using the `lttng` tool
+(see <<controlling-tracing,Controlling tracing>>):
+
+* listing all logger names
+* enabling/disabling events per logger name
+* JUL log levels
+
+Here's an example:
+
+[source,java]
+----
+import java.util.logging.Logger;
+import org.lttng.ust.jul.LTTngAgent;
+
+public class Test
+{
+ public static void main(String[] argv) throws Exception
+ {
+ // create a logger
+ Logger logger = Logger.getLogger("jello");
+
+ // call this as soon as possible (before logging)
+ LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+
+ // log at will!
+ logger.info("some info");
+ logger.warning("some warning");
+ Thread.sleep(500);
+ logger.finer("finer information...");
+ Thread.sleep(123);
+ logger.severe("error!");
+
+ // not mandatory, but cleaner
+ lttngAgent.dispose();
+ }
+}
+----
+
+The LTTng-UST JUL agent Java classes are packaged in a JAR file named
+path:{liblttng-ust-jul.jar}. It is typically located in
+dir:{/usr/lib/lttng/java}. To compile the snippet above
+(saved as path:{Test.java}), do:
+
+[role="term"]
+----
+javac -cp /usr/lib/lttng/java/liblttng-ust-jul.jar Test.java
+----
+
+You can run the resulting compiled class:
+
+[role="term"]
+----
+java -cp /usr/lib/lttng/java/liblttng-ust-jul.jar:. Test
+----
+
+NOTE: http://openjdk.java.net/[OpenJDK] 7 is used for development and
+continuous integration, thus this version is directly supported.
+However, the LTTng-UST JUL agent has also been tested with OpenJDK 6.
+
+
+[[instrumenting-linux-kernel]]
+==== Linux kernel
+
+The Linux kernel can be instrumented for LTTng tracing, either its core
+source code or a kernel module. It has to be noted that Linux is
+readily traceable using LTTng since many parts of its source code are
+already instrumented: this is the job of the upstream
+http://git.lttng.org/?p=lttng-modules.git[LTTng-modules]
+package. This section presents how to add LTTng instrumentation where it
+does not currently exist and how to instrument custom kernel modules.
+
+All LTTng instrumentation in the Linux kernel is based on an existing
+infrastructure which bears the name of its main macro, `TRACE_EVENT()`.
+This macro is used to define tracepoints,
+each tracepoint having a name, usually with the
++__subsys_____name__+ format,
++_subsys_+ being the subsystem name and
++_name_+ the specific event name.
+
+Tracepoints defined with `TRACE_EVENT()` may be inserted anywhere in
+the Linux kernel source code, after what callbacks, called _probes_,
+may be registered to execute some action when a tracepoint is
+executed. This mechanism is directly used by ftrace and perf,
+but cannot be used as is by LTTng: an adaptation layer is added to
+satisfy LTTng's specific needs.
+
+With that in mind, this documentation does not cover the `TRACE_EVENT()`
+format and how to use it, but it is mandatory to understand it and use
+it to instrument Linux for LTTng. A series of
+LWN articles explain
+`TRACE_EVENT()` in details:
+http://lwn.net/Articles/379903/[part 1],
+http://lwn.net/Articles/381064/[part 2], and
+http://lwn.net/Articles/383362/[part 3].
+Once you master `TRACE_EVENT()` enough for your use case, continue
+reading this section so that you can add the LTTng adaptation layer of
+instrumentation.
+
+This section first discusses the general method of instrumenting the
+Linux kernel for LTTng. This method is then reused for the specific
+case of instrumenting a kernel module.
+
+
+[[instrumenting-linux-kernel-itself]]
+===== Instrumenting the Linux kernel for LTTng
+
+The following subsections explain strictly how to add custom LTTng
+instrumentation to the Linux kernel. They do not explain how the
+macros actually work and the internal mechanics of the tracer.
+
+You should have a Linux kernel source code tree to work with.
+Throughout this section, all file paths are relative to the root of
+this tree unless otherwise stated.
+
+You will need a copy of the LTTng-modules Git repository:
+
+[role="term"]
+----
+git clone git://git.lttng.org/lttng-modules.git
+----
+
+The steps to add custom LTTng instrumentation to a Linux kernel
+involves defining and using the mainline `TRACE_EVENT()` tracepoints
+first, then writing and using the LTTng adaptation layer.
+
+
+[[mainline-trace-event]]
+===== Defining/using tracepoints with mainline `TRACE_EVENT()` infrastructure
+
+The first step is to define tracepoints using the mainline Linux
+`TRACE_EVENT()` macro and insert tracepoints where you want them.
+Your tracepoint definitions reside in a header file in
+dir:{include/trace/events}. If you're adding tracepoints to an existing
+subsystem, edit its appropriate header file.
+
+As an example, the following header file (let's call it
+path:{include/trace/events/hello.h}) defines one tracepoint using
+`TRACE_EVENT()`:
+
+[source,c]
+----
+/* subsystem name is "hello" */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hello
+
+#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HELLO_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(
+ /* "hello" is the subsystem name, "world" is the event name */
+ hello_world,
+
+ /* tracepoint function prototype */
+ TP_PROTO(int foo, const char* bar),
+
+ /* arguments for this tracepoint */
+ TP_ARGS(foo, bar),
+
+ /* LTTng doesn't need those */
+ TP_STRUCT__entry(),
+ TP_fast_assign(),
+ TP_printk("", 0)
+);
+
+#endif
+
+/* this part must be outside protection */
+#include <trace/define_trace.h>
+----
+
+Notice that we don't use any of the last three arguments: they
+are left empty here because LTTng doesn't need them. You would only fill
+`TP_STRUCT__entry()`, `TP_fast_assign()` and `TP_printk()` if you were
+to also use this tracepoint for ftrace/perf.
+
+Once this is done, you may place calls to `trace_hello_world()`
+wherever you want in the Linux source code. As an example, let us place
+such a tracepoint in the `usb_probe_device()` static function
+(path:{drivers/usb/core/driver.c}):
+
+[source,c]
+----
+/* called from driver core with dev locked */
+static int usb_probe_device(struct device *dev)
+{
+ struct usb_device_driver *udriver = to_usb_device_driver(dev->driver);
+ struct usb_device *udev = to_usb_device(dev);
+ int error = 0;
+
+ trace_hello_world(udev->devnum, udev->product);
+
+ /* ... */
+}
+----
+
+This tracepoint should fire every time a USB device is plugged in.
+
+At the top of path:{driver.c}, we need to include our actual tracepoint
+definition and, in this case (one place per subsystem), define
+`CREATE_TRACE_POINTS`, which will create our tracepoint:
+
+[source,c]
+----
+/* ... */
+
+#include "usb.h"
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/hello.h>
+
+/* ... */
+----
+
+Build your custom Linux kernel. In order to use LTTng, make sure the
+following kernel configuration options are enabled:
+
+* `CONFIG_MODULES` (loadable module support)
+* `CONFIG_KALLSYMS` (load all symbols for debugging/kksymoops)
+* `CONFIG_HIGH_RES_TIMERS` (high resolution timer support)
+* `CONFIG_TRACEPOINTS` (kernel tracepoint instrumentation)
+
+Boot the custom kernel. The directory
+dir:{/sys/kernel/debug/tracing/events/hello} should exist if everything
+went right, with a dir:{hello_world} subdirectory.
+
+
+[[lttng-adaptation-layer]]
+===== Adding the LTTng adaptation layer
+
+The steps to write the LTTng adaptation layer are, in your
+LTTng-modules copy's source code tree:
+
+. In dir:{instrumentation/events/lttng-module},
+ add a header +__subsys__.h+ for your custom
+ subsystem +__subsys__+ and write your
+ tracepoint definitions using LTTng-modules macros in it.
+ Those macros look like the mainline kernel equivalents,
+ but they present subtle, yet important differences.
+. In dir:{probes}, create the C source file of the LTTng probe kernel
+ module for your subsystem. It should be named
+ +lttng-probe-__subsys__.c+.
+. Edit path:{probes/Makefile} so that the LTTng-modules project
+ builds your custom LTTng probe kernel module.
+. Build and install LTTng kernel modules.
+
+Following our `hello_world` event example, here's the content of
+path:{instrumentation/events/lttng-module/hello.h}:
+
+[source,c]
+----
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hello
+
+#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HELLO_H
+
+#include <linux/tracepoint.h>
+
+LTTNG_TRACEPOINT_EVENT(
+ /* format identical to mainline version for those */
+ hello_world,
+ TP_PROTO(int foo, const char* bar),
+ TP_ARGS(foo, bar),
+
+ /* possible differences */
+ TP_STRUCT__entry(
+ __field(int, my_int)
+ __field(char, char0)
+ __field(char, char1)
+ __string(product, bar)
+ ),
+
+ /* notice the use of tp_assign()/tp_strcpy() and no semicolons */
+ TP_fast_assign(
+ tp_assign(my_int, foo)
+ tp_assign(char0, bar[0])
+ tp_assign(char1, bar[1])
+ tp_strcpy(product, bar)
+ ),
+
+ /* This one is actually not used by LTTng either, but must be
+ * present for the moment.
+ */
+ TP_printk("", 0)
+
+/* no semicolon after this either */
+)
+
+#endif
+
+/* other difference: do NOT include <trace/define_trace.h> */
+#include "../../../probes/define_trace.h"
+----
+
+Some possible entries for `TP_STRUCT__entry()` and `TP_fast_assign()`,
+in the case of LTTng-modules, are shown in the
+<<lttng-modules-ref,LTTng-modules reference>> section.
+
+The best way to learn how to use the above macros is to inspect
+existing LTTng tracepoint definitions in
+dir:{instrumentation/events/lttng-module} header files. Compare
+them with the Linux kernel mainline versions in
+dir:{include/trace/events}.
+
+The next step is writing the LTTng probe kernel module C source file.
+This one is named +lttng-probe-__subsys__.c+
+in dir:{probes}. You may always use the following template:
+
+[source,c]
+----
+#include <linux/module.h>
+#include "../lttng-tracer.h"
+
+/* Build time verification of mismatch between mainline TRACE_EVENT()
+ * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
+ */
+#include <trace/events/hello.h>
+
+/* create LTTng tracepoint probes */
+#define LTTNG_PACKAGE_BUILD
+#define CREATE_TRACE_POINTS
+#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module
+
+#include "../instrumentation/events/lttng-module/hello.h"
+
+MODULE_LICENSE("GPL and additional rights");
+MODULE_AUTHOR("Your name <your-email>");
+MODULE_DESCRIPTION("LTTng hello probes");
+MODULE_VERSION(__stringify(LTTNG_MODULES_MAJOR_VERSION) "."
+ __stringify(LTTNG_MODULES_MINOR_VERSION) "."
+ __stringify(LTTNG_MODULES_PATCHLEVEL_VERSION)
+ LTTNG_MODULES_EXTRAVERSION);
+----
+
+Just replace `hello` with your subsystem name. In this example,
+`<trace/events/hello.h>`, which is the original mainline tracepoint
+definition header, is included for verification purposes: the
+LTTng-modules build system is able to emit an error at build time when
+the arguments of the mainline `TRACE_EVENT()` definitions do not match
+the ones of the LTTng-modules adaptation layer
+(`LTTNG_TRACEPOINT_EVENT()`).
+
+Edit path:{probes/Makefile} and add your new kernel module object
+next to existing ones:
+
+[source,make]
+----
+# ...
+
+obj-m += lttng-probe-module.o
+obj-m += lttng-probe-power.o
+
+obj-m += lttng-probe-hello.o
+
+# ...
+----
+
+Time to build! Point to your custom Linux kernel source tree using
+the `KERNELDIR` variable:
+
+[role="term"]
+----
+make KERNELDIR=/path/to/custom/linux
+----
+
+Finally, install modules:
+
+[role="term"]
+----
+sudo make modules_install
+----
+
+
+[[instrumenting-linux-kernel-tracing]]
+===== Tracing
+
+The <<controlling-tracing,Controlling tracing>> section explains
+how to use the `lttng` tool to create and control tracing sessions.
+Although the `lttng` tool will load the appropriate _known_ LTTng kernel
+modules when needed (by launching `root`'s session daemon), it won't
+load your custom `lttng-probe-hello` module by default. You need to
+manually load the `lttng-probe-hello` module, and start an LTTng session
+daemon as `root`:
+
+[role="term"]
+----
+sudo pkill -u root lttng-sessiond
+sudo modprobe lttng_probe_hello
+sudo lttng-sessiond
+----
+
+The first command makes sure any existing instance is killed. If
+you're not interested in using the default probes, or if you only
+want to use a few of them, you can use the `--kmod-probes` option
+of `lttng-sessiond` instead, which specifies an absolute list of
+probes to load (without the `lttng-probe-` prefix):
+
+[role="term"]
+----
+sudo lttng-sessiond --kmod-probes=hello,ext4,net,block,signal,sched
+----
+
+Confirm the custom probe module is loaded:
+
+[role="term"]
+----
+lsmod | grep lttng_probe_hello
+----
+
+The `hello_world` event should appear in the list when doing
+
+[role="term"]
+----
+lttng list --kernel | grep hello
+----
+
+You may now create an LTTng tracing session, enable the `hello_world`
+kernel event (and others if you wish) and start tracing:
+
+[role="term"]
+----
+sudo lttng create my-session
+sudo lttng enable-event --kernel hello_world
+sudo lttng start
+----
+
+Plug a few USB devices, then stop tracing and inspect the trace (if
+http://diamon.org/babeltrace[Babeltrace]
+is installed):
+
+[role="term"]
+----
+sudo lttng stop
+sudo lttng view
+----
+
+Here's a sample output:
+
+----
+[15:30:34.835895035] (+?.?????????) hostname hello_world: { cpu_id = 1 }, { my_int = 8, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
+[15:30:42.262781421] (+7.426886386) hostname hello_world: { cpu_id = 1 }, { my_int = 9, char0 = 80, char1 = 97, product = "Patriot Memory" }
+[15:30:48.175621778] (+5.912840357) hostname hello_world: { cpu_id = 1 }, { my_int = 10, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
+----
+
+Two USB flash drives were used for this test.
+
+You may change your LTTng custom probe, rebuild it and reload it at
+any time when not tracing. Make sure you remove the old module
+(either by killing the root LTTng session daemon which loaded the
+module in the first place (if you used `--kmod-probes`), or by
+using `modprobe --remove` directly) before loading the updated one.
+
+
+[[instrumenting-out-of-tree-linux-kernel]]
+===== Advanced: Instrumenting an out-of-tree Linux kernel module for LTTng
+
+Instrumenting a custom Linux kernel module for LTTng follows the exact
+same steps as
+<<instrumenting-linux-kernel-itself,adding instrumentation
+to the Linux kernel itself>>,
+the only difference being that your mainline tracepoint definition
+header doesn't reside in the mainline source tree, but in your
+kernel module source tree.
+
+The only reference to this mainline header is in the LTTng custom
+probe's source code (path:{probes/lttng-probe-hello.c} in our example),
+for build time verification:
+
+[source,c]
+----
+/* ... */
+
+/* Build time verification of mismatch between mainline TRACE_EVENT()
+ * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
+ */
+#include <trace/events/hello.h>
+
+/* ... */
+----
+
+The preferred, flexible way to include your module's mainline
+tracepoint definition header is to put it in a specific directory
+relative to your module's root, e.g., dir:{tracepoints}, and include it
+relative to your module's root directory in the LTTng custom probe's
+source:
+
+[source,c]
+----
+#include <tracepoints/hello.h>
+----
+
+You may then build LTTng-modules by adding your module's root
+directory as an include path to the extra C flags:
+
+[role="term"]
+----
+make ccflags-y=-I/path/to/kernel/module KERNELDIR=/path/to/custom/linux
+----
+
+Using `ccflags-y` allows you to move your kernel module to another
+directory and rebuild the LTTng-modules project with no change to
+source files.
+
+
+[[proc-lttng-logger-abi]]
+==== LTTng logger ABI
+
+The `lttng-tracer` Linux kernel module, installed by the LTTng-modules
+package, creates a special LTTng logger ABI file path:{/proc/lttng-logger}
+when loaded. Writing text data to this file generates an LTTng kernel
+domain event named `lttng_logger`.
+
+Unlike other kernel domain events, `lttng_logger` may be enabled by
+any user, not only root users or members of the tracing group.
+
+To use the LTTng logger ABI, simply write a string to
+path:{/proc/lttng-logger}:
+
+[role="term"]
+----
+echo -n 'Hello, World!' > /proc/lttng-logger
+----
+
+The `msg` field of the `lttng_logger` event contains the recorded
+message.
+
+NOTE: Messages are split in chunks of 1024{nbsp}bytes.
+
+The LTTng logger ABI is a quick and easy way to trace some events from
+user space through the kernel tracer. However, it is much more basic
+than LTTng-UST: it's slower (involves system call round-trip to the
+kernel and only supports logging strings). The LTTng logger ABI is
+particularly useful for recording logs as LTTng traces from shell
+scripts, potentially combining them with other Linux kernel/user space
+events.
+
+
+[[instrumenting-32-bit-app-on-64-bit-system]]
+==== Advanced: Instrumenting a 32-bit application on a 64-bit system
+
+[[advanced-instrumenting-techniques]]In order to trace a 32-bit
+application running on a 64-bit system,
+LTTng must use a dedicated 32-bit
+<<lttng-consumerd,consumer daemon>>. This section discusses how to
+build that daemon (which is _not_ part of the default 64-bit LTTng
+build) and the LTTng 32-bit tracing libraries, and how to instrument
+a 32-bit application in that context.
+
+Make sure you install all 32-bit versions of LTTng dependencies.
+Their names can be found in the path:{README.md} files of each LTTng package
+source. How to find and install them will vary depending on your target
+Linux distribution. `gcc-multilib` is a common package name for the
+multilib version of GCC, which you will also need.
+
+The following packages will be built for 32-bit support on a 64-bit
+system: http://urcu.so/[Userspace RCU],
+LTTng-UST and LTTng-tools.
+
+
+[[building-32-bit-userspace-rcu]]
+===== Building 32-bit Userspace RCU
+
+Follow this:
+
+[role="term"]
+----
+git clone git://git.urcu.so/urcu.git
+cd urcu
+./bootstrap
+./configure --libdir=/usr/lib32 CFLAGS=-m32
+make
+sudo make install
+sudo ldconfig
+----
+
+The `-m32` C compiler flag creates 32-bit object files and `--libdir`
+indicates where to install the resulting libraries.
+
+
+[[building-32-bit-lttng-ust]]
+===== Building 32-bit LTTng-UST
+
+Follow this:
+
+[role="term"]
+----
+git clone http://git.lttng.org/lttng-ust.git
+cd lttng-ust
+./bootstrap
+./configure --prefix=/usr \
+ --libdir=/usr/lib32 \
+ CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS=-L/usr/lib32
+make
+sudo make install
+sudo ldconfig
+----
+
+`-L/usr/lib32` is required for the build to find the 32-bit versions
+of Userspace RCU and other dependencies.
+
+[NOTE]
+====
+Depending on your Linux distribution,
+32-bit libraries could be installed at a different location than
+dir:{/usr/lib32}. For example, Debian is known to install
+some 32-bit libraries in dir:{/usr/lib/i386-linux-gnu}.
+
+In this case, make sure to set `LDFLAGS` to all the
+relevant 32-bit library paths, e.g.,
+`LDFLAGS="-L/usr/lib32 -L/usr/lib/i386-linux-gnu"`.
+====
+
+NOTE: You may add options to path:{./configure} if you need them, e.g., for
+Java and SystemTap support. Look at `./configure --help` for more
+information.
+
+
+[[building-32-bit-lttng-tools]]
+===== Building 32-bit LTTng-tools
+
+Since the host is a 64-bit system, most 32-bit binaries and libraries of
+LTTng-tools are not needed; the host will use their 64-bit counterparts.
+The required step here is building and installing a 32-bit consumer
+daemon.
+
+Follow this:
+
+[role="term"]
+----
+git clone http://git.lttng.org/lttng-tools.git
+cd lttng-ust
+./bootstrap
+./configure --prefix=/usr \
+ --libdir=/usr/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS=-L/usr/lib32
+make
+cd src/bin/lttng-consumerd
+sudo make install
+sudo ldconfig
+----
+
+The above commands build all the LTTng-tools project as 32-bit
+applications, but only installs the 32-bit consumer daemon.
+
+
+[[building-64-bit-lttng-tools]]
+===== Building 64-bit LTTng-tools
+
+Finally, you need to build a 64-bit version of LTTng-tools which is
+aware of the 32-bit consumer daemon previously built and installed:
+
+[role="term"]
+----
+make clean
+./bootstrap
+./configure --prefix=/usr \
+ --with-consumerd32-libdir=/usr/lib32 \
+ --with-consumerd32-bin=/usr/lib32/lttng/libexec/lttng-consumerd
+make
+sudo make install
+sudo ldconfig
+----
+
+Henceforth, the 64-bit session daemon will automatically find the
+32-bit consumer daemon if required.
+
+
+[[building-instrumented-32-bit-c-application]]
+===== Building an instrumented 32-bit C application
+
+Let us reuse the _Hello world_ example of
+<<tracing-your-own-user-application,Tracing your own user application>>
+(<<getting-started,Getting started>> chapter).
+
+The instrumentation process is unaltered.
+
+First, a typical 64-bit build (assuming you're running a 64-bit system):
+
+[role="term"]
+----
+gcc -o hello64 -I. hello.c hello-tp.c -ldl -llttng-ust
+----
+
+Now, a 32-bit build:
+
+[role="term"]
+----
+gcc -o hello32 -I. -m32 hello.c hello-tp.c -L/usr/lib32 \
+ -ldl -llttng-ust -Wl,-rpath,/usr/lib32
+----
+
+The `-rpath` option, passed to the linker, will make the dynamic loader
+check for libraries in dir:{/usr/lib32} before looking in its default paths,
+where it should find the 32-bit version of `liblttng-ust`.
+
+
+[[running-32-bit-and-64-bit-c-applications]]
+===== Running 32-bit and 64-bit versions of an instrumented C application
+
+Now, both 32-bit and 64-bit versions of the _Hello world_ example above
+can be traced in the same tracing session. Use the `lttng` tool as usual
+to create a tracing session and start tracing:
+
+[role="term"]
+----
+lttng create session-3264
+lttng enable-event -u -a
+./hello32
+./hello64
+lttng stop
+----
+
+Use `lttng view` to verify both processes were
+successfully traced.
+
+
+[[controlling-tracing]]
+=== Controlling tracing
+
+Once you're in possession of a software that is properly
+<<instrumenting,instrumented>> for LTTng tracing, be it thanks to
+the built-in LTTng probes for the Linux kernel, a custom user
+application or a custom Linux kernel, all that is left is actually
+tracing it. As a user, you control LTTng tracing using a single command
+line interface: the `lttng` tool. This tool uses `liblttng-ctl` behind
+the scene to connect to and communicate with session daemons. LTTng
+session daemons may either be started manually (`lttng-sessiond`) or
+automatically by the `lttng` command when needed. Trace data may
+be forwarded to the network and used elsewhere using an LTTng relay
+daemon (`lttng-relayd`).
+
+The manpages of `lttng`, `lttng-sessiond` and `lttng-relayd` are pretty
+complete, thus this section is not an online copy of the latter (we
+leave this contents for the
+<<online-lttng-manpages,Online LTTng manpages>> section).
+This section is rather a tour of LTTng
+features through practical examples and tips.
+
+If not already done, make sure you understand the core concepts
+and how LTTng components connect together by reading the
+<<understanding-lttng,Understanding LTTng>> chapter; this section
+assumes you are familiar with them.
+
+
+[[creating-destroying-tracing-sessions]]
+==== Creating and destroying tracing sessions
+
+Whatever you want to do with `lttng`, it has to happen inside a
+**tracing session**, created beforehand. A session, in general, is a
+per-user container of state. A tracing session is no different; it
+keeps a specific state of stuff like:
+
+* session name
+* enabled/disabled channels with associated parameters
+* enabled/disabled events with associated log levels and filters
+* context information added to channels
+* tracing activity (started or stopped)
+
+and more.
+
+A single user may have many active tracing sessions. LTTng session
+daemons are the ultimate owners and managers of tracing sessions. For
+user space tracing, each user has its own session daemon. Since Linux
+kernel tracing requires root privileges, only `root`'s session daemon
+may enable and trace kernel events. However, `lttng` has a `--group`
+option (which is passed to `lttng-sessiond` when starting it) to
+specify the name of a _tracing group_ which selected users may be part
+of to be allowed to communicate with `root`'s session daemon. By
+default, the tracing group name is `tracing`.
+
+To create a tracing session, do:
+
+[role="term"]
+----
+lttng create my-session
+----
+
+This will create a new tracing session named `my-session` and make it
+the current one. If you don't specify any name (calling only
+`lttng create`), your tracing session will be named `auto`. Traces
+are written in +\~/lttng-traces/__session__-+ followed
+by the tracing session's creation date/time by default, where
++__session__+ is the tracing session name. To save them
+at a different location, use the `--output` option:
+
+[role="term"]
+----
+lttng create --output /tmp/some-directory my-session
+----
+
+You may create as many tracing sessions as you wish:
+
+[role="term"]
+----
+lttng create other-session
+lttng create yet-another-session
+----
+
+You may view all existing tracing sessions using the `list` command:
+
+[role="term"]
+----
+lttng list
+----
+
+The state of a _current tracing session_ is kept in path:{~/.lttngrc}. Each
+invocation of `lttng` reads this file to set its current tracing
+session name so that you don't have to specify a session name for each
+command. You could edit this file manually, but the preferred way to
+set the current tracing session is to use the `set-session` command:
+
+[role="term"]
+----
+lttng set-session other-session
+----
+
+Most `lttng` commands accept a `--session` option to specify the name
+of the target tracing session.
+
+Any existing tracing session may be destroyed using the `destroy`
+command:
+
+[role="term"]
+----
+lttng destroy my-session
+----
+
+Providing no argument to `lttng destroy` will destroy the current
+tracing session. Destroying a tracing session will stop any tracing
+running within the latter. Destroying a tracing session frees resources
+acquired by the session daemon and tracer side, making sure to flush
+all trace data.
+
+You can't do much with LTTng using only the `create`, `set-session`
+and `destroy` commands of `lttng`, but it is essential to know them in
+order to control LTTng tracing, which always happen within the scope of
+a tracing session.
+
+
+[[enabling-disabling-events]]
+==== Enabling and disabling events
+
+Inside a tracing session, individual events may be enabled or disabled
+so that tracing them may or may not generate trace data.
+
+We sometimes use the term _event_ metonymically throughout this text to
+refer to a specific condition, or _rule_, that could lead, when
+satisfied, to an actual occurring event (a point at a specific position
+in source code/binary program, logical processor and time capturing
+some payload) being recorded as trace data. This specific condition is
+composed of:
+
+. A **domain** (kernel, user space or `java.util.logging`) (required).
+. One or many **instrumentation points** in source code or binary
+ program (tracepoint name, address, symbol name, function name,
+ logger name, etc.) to be executed (required).
+. A **log level** (each instrumentation point declares its own log
+ level) or log level range to match (optional; only valid for user
+ space domain).
+. A **custom user expression**, or **filter**, that must evaluate to
+ _true_ when a tracepoint is executed (optional; only valid for user
+ space domain).
+
+All conditions are specified using arguments passed to the
+`enable-event` command of the `lttng` tool.
+
+Condition 1 is specified using either `--kernel/-k` (kernel),
+`--userspace/-u` (user space) or `--jul/-j`
+(JUL). Exactly one of those
+three arguments must be specified.
+
+Condition 2 is specified using one of:
+
+`--tracepoint`::
+ Tracepoint.
+
+`--probe`::
+ Dynamic probe (address, symbol name or combination
+ of both in binary program; only valid for kernel domain).
+
+`--function`::
+ function entry/exit (address, symbol name or
+ combination of both in binary program; only valid for kernel domain).
+
+`--syscall`::
+ System call entry/exit (only valid for kernel domain).
+
+When none of the above is specified, `enable-event` defaults to
+using `--tracepoint`.
+
+Condition 3 is specified using one of:
+
+`--loglevel`::
+ Log level range from the specified level to the most severe
+ level.
+
+`--loglevel-only`::
+ Specific log level.
+
+See `lttng enable-event --help` for the complete list of log level
+names.
+
+Condition 4 is specified using the `--filter` option. This filter is
+a C-like expression, potentially reading real-time values of event
+fields, that has to evaluate to _true_ for the condition to be satisfied.
+Event fields are read using plain identifiers while context fields
+must be prefixed with `$ctx.`. See `lttng enable-event --help` for
+all usage details.
+
+The aforementioned arguments are combined to create and enable events.
+Each unique combination of arguments leads to a different
+_enabled event_. The log level and filter arguments are optional, their
+default values being respectively all log levels and a filter which
+always returns _true_.
+
+Here are a few examples (you must
+<<creating-destroying-tracing-sessions,create a tracing session>>
+first):
+
+[role="term"]
+----
+lttng enable-event -u --tracepoint my_app:hello_world
+lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_WARNING
+lttng enable-event -u --tracepoint 'my_other_app:*'
+lttng enable-event -u --tracepoint my_app:foo_bar \
+ --filter 'some_field <= 23 && !other_field'
+lttng enable-event -k --tracepoint sched_switch
+lttng enable-event -k --tracepoint gpio_value
+lttng enable-event -k --function usb_probe_device usb_probe_device
+lttng enable-event -k --syscall --all
+----
+
+The wildcard symbol, `*`, matches _anything_ and may only be used at
+the end of the string when specifying a _tracepoint_. Make sure to
+use it between single quotes in your favorite shell to avoid
+undesired shell expansion.
+
+You can see a list of events (enabled or disabled) using
+
+[role="term"]
+----
+lttng list some-session
+----
+
+where `some-session` is the name of the desired tracing session.
+
+What you're actually doing when enabling events with specific conditions
+is creating a **whitelist** of traceable events for a given channel.
+Thus, the following case presents redundancy:
+
+[role="term"]
+----
+lttng enable-event -u --tracepoint my_app:hello_you
+lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_DEBUG
+----
+
+The second command, matching a log level range, is useless since the first
+command enables all tracepoints matching the same name,
+`my_app:hello_you`.
+
+Disabling an event is simpler: you only need to provide the event
+name to the `disable-event` command:
+
+[role="term"]
+----
+lttng disable-event --userspace my_app:hello_you
+----
+
+This name has to match a name previously given to `enable-event` (it
+has to be listed in the output of `lttng list some-session`).
+The `*` wildcard is supported, as long as you also used it in a
+previous `enable-event` invocation.
+
+Disabling an event does not add it to some blacklist: it simply removes
+it from its channel's whitelist. This is why you cannot disable an event
+which wasn't previously enabled.
+
+A disabled event will not generate any trace data, even if all its
+specified conditions are met.
+
+Events may be enabled and disabled at will, either when LTTng tracers
+are active or not. Events may be enabled before a user space application
+is even started.
+
+
+[[basic-tracing-session-control]]
+==== Basic tracing session control
+
+Once you have
+<<creating-destroying-tracing-sessions,created a tracing session>>
+and <<enabling-disabling-events,enabled one or more events>>,
+you may activate the LTTng tracers for the current tracing session at
+any time:
+
+[role="term"]
+----
+lttng start
+----
+
+Subsequently, you may stop the tracers:
+
+[role="term"]
+----
+lttng stop
+----
+
+LTTng is very flexible: user space applications may be launched before
+or after the tracers are started. Events will only be recorded if they
+are properly enabled and if they occur while tracers are started.
+
+A tracing session name may be passed to both the `start` and `stop`
+commands to start/stop tracing a session other than the current one.
+
+
+[[enabling-disabling-channels]]
+==== Enabling and disabling channels
+
+<<event,As mentioned>> in the
+<<understanding-lttng,Understanding LTTng>> chapter, enabled
+events are contained in a specific channel, itself contained in a
+specific tracing session. A channel is a group of events with
+tunable parameters (event loss mode, sub-buffer size, number of
+sub-buffers, trace file sizes and count, etc.). A given channel may
+only be responsible for enabled events belonging to one domain: either
+kernel or user space.
+
+If you only used the `create`, `enable-event` and `start`/`stop`
+commands of the `lttng` tool so far, one or two channels were
+automatically created for you (one for the kernel domain and/or one
+for the user space domain). The default channels are both named
+`channel0`; channels from different domains may have the same name.
+
+The current channels of a given tracing session can be viewed with
+
+[role="term"]
+----
+lttng list some-session
+----
+
+where `some-session` is the name of the desired tracing session.
+
+To create and enable a channel, use the `enable-channel` command:
+
+[role="term"]
+----
+lttng enable-channel --kernel my-channel
+----
+
+This will create a kernel domain channel named `my-channel` with
+default parameters in the current tracing session.
+
+[NOTE]
+====
+Because of a current limitation, all
+channels must be _created_ prior to beginning tracing in a
+given tracing session, i.e. before the first time you do
+`lttng start`.
+
+Since a channel is automatically created by
+`enable-event` only for the specified domain, you cannot,
+for example, enable a kernel domain event, start tracing and then
+enable a user space domain event because no user space channel
+exists yet and it's too late to create one.
+
+For this reason, make sure to configure your channels properly
+before starting the tracers for the first time!
+====
+
+Here's another example:
+
+[role="term"]
+----
+lttng enable-channel --userspace --session other-session --overwrite \
+ --tracefile-size 1048576 1mib-channel
+----
+
+This will create a user space domain channel named `1mib-channel` in
+the tracing session named `other-session` that loses new events by
+overwriting previously recorded events (instead of the default mode of
+discarding newer ones) and saves trace files with a maximum size of
+1{nbsp}MiB each.
+
+Note that channels may also be created using the `--channel` option of
+the `enable-event` command when the provided channel name doesn't exist
+for the specified domain:
+
+[role="term"]
+----
+lttng enable-event --kernel --channel some-channel sched_switch
+----
+
+If no kernel domain channel named `some-channel` existed before calling
+the above command, it would be created with default parameters.
+
+You may enable the same event in two different channels:
+
+[role="term"]
+----
+lttng enable-event --userspace --channel my-channel app:tp
+lttng enable-event --userspace --channel other-channel app:tp
+----
+
+If both channels are enabled, the occurring `app:tp` event will
+generate two recorded events, one for each channel.
+
+Disabling a channel is done with the `disable-event` command:
+
+[role="term"]
+----
+lttng disable-event --kernel some-channel
+----
+
+The state of a channel precedes the individual states of events within
+it: events belonging to a disabled channel, even if they are
+enabled, won't be recorded.
+
+
+
+[[fine-tuning-channels]]
+===== Fine-tuning channels
+
+There are various parameters that may be fine-tuned with the
+`enable-channel` command. The latter are well documented in
+man:lttng(1) and in the <<channel,Channel>> section of the
+<<understanding-lttng,Understanding LTTng>> chapter. For basic
+tracing needs, their default values should be just fine, but here are a
+few examples to break the ice.
+
+As the frequency of recorded events increases--either because the
+event throughput is actually higher or because you enabled more events
+than usual—__event loss__ might be experienced. Since LTTng never
+waits, by design, for sub-buffer space availability (non-blocking
+tracer), when a sub-buffer is full and no empty sub-buffers are left,
+there are two possible outcomes: either the new events that do not fit
+are rejected, or they start replacing the oldest recorded events.
+The choice of which algorithm to use is a per-channel parameter, the
+default being discarding the newest events until there is some space
+left. If your situation always needs the latest events at the expense
+of writing over the oldest ones, create a channel with the `--overwrite`
+option:
+
+[role="term"]
+----
+lttng enable-channel --kernel --overwrite my-channel
+----
+
+When an event is lost, it means no space was available in any
+sub-buffer to accommodate it. Thus, if you want to cope with sporadic
+high event throughput situations and avoid losing events, you need to
+allocate more room for storing them in memory. This can be done by
+either increasing the size of sub-buffers or by adding sub-buffers.
+The following example creates a user space domain channel with
+16{nbsp}sub-buffers of 512{nbsp}kiB each:
+
+[role="term"]
+----
+lttng enable-channel --userspace --num-subbuf 16 --subbuf-size 512k big-channel
+----
+
+Both values need to be powers of two, otherwise they are rounded up
+to the next one.
+
+Two other interesting available parameters of `enable-channel` are
+`--tracefile-size` and `--tracefile-count`, which respectively limit
+the size of each trace file and the their count for a given channel.
+When the number of written trace files reaches its limit for a given
+channel-CPU pair, the next trace file will overwrite the very first
+one. The following example creates a kernel domain channel with a
+maximum of three trace files of 1{nbsp}MiB each:
+
+[role="term"]
+----
+lttng enable-channel --kernel --tracefile-size 1M --tracefile-count 3 my-channel
+----
+
+An efficient way to make sure lots of events are generated is enabling
+all kernel events in this channel and starting the tracer:
+
+[role="term"]
+----
+lttng enable-event --kernel --all --channel my-channel
+lttng start
+----
+
+After a few seconds, look at trace files in your tracing session
+output directory. For two CPUs, it should look like:
+
+----
+my-channel_0_0 my-channel_1_0
+my-channel_0_1 my-channel_1_1
+my-channel_0_2 my-channel_1_2
+----
+
+Amongst the files above, you might see one in each group with a size
+lower than 1{nbsp}MiB: they are the files currently being written.
+
+Since all those small files are valid LTTng trace files, LTTng trace
+viewers may read them. It is the viewer's responsibility to properly
+merge the streams so as to present an ordered list to the user.
+http://diamon.org/babeltrace[Babeltrace]
+merges LTTng trace files correctly and is fast at doing it.
+
+
+[[adding-context]]
+==== Adding some context to channels
+
+If you read all the sections of
+<<controlling-tracing,Controlling tracing>> so far, you should be
+able to create tracing sessions, create and enable channels and events
+within them and start/stop the LTTng tracers. Event fields recorded in
+trace files provide important information about occurring events, but
+sometimes external context may help you solve a problem faster. This
+section discusses how to add context information to events of a
+specific channel using the `lttng` tool.
+
+There are various available context values which can accompany events
+recorded by LTTng, for example:
+
+* **process information**:
+** identifier (PID)
+** name
+** priority
+** scheduling priority (niceness)
+** thread identifier (TID)
+* the **hostname** of the system on which the event occurred
+* plenty of **performance counters** using perf:
+** CPU cycles, stalled cycles, idle cycles, etc.
+** cache misses
+** branch instructions, misses, loads, etc.
+** CPU faults
+** etc.
+
+The full list is available in the output of `lttng add-context --help`.
+Some of them are reserved for a specific domain (kernel or
+user space) while others are available for both.
+
+To add context information to one or all channels of a given tracing
+session, use the `add-context` command:
+
+[role="term"]
+----
+lttng add-context --userspace --type vpid --type perf:thread:cpu-cycles
+----
+
+The above example adds the virtual process identifier and per-thread
+CPU cycles count values to all recorded user space domain events of the
+current tracing session. Use the `--channel` option to select a specific
+channel:
+
+[role="term"]
+----
+lttng add-context --kernel --channel my-channel --type tid
+----
+
+adds the thread identifier value to all recorded kernel domain events
+in the channel `my-channel` of the current tracing session.
+
+Beware that context information cannot be removed from channels once
+it's added for a given tracing session.
+
+
+[[saving-loading-tracing-session]]
+==== Saving and loading tracing session configurations
+
+Configuring a tracing session may be long: creating and enabling
+channels with specific parameters, enabling kernel and user space
+domain events with specific log levels and filters, adding context
+to some channels, etc. If you're going to use LTTng to solve real
+world problems, chances are you're going to have to record events using
+the same tracing session setup over and over, modifying a few variables
+each time in your instrumented program or environment. To avoid
+constant tracing session reconfiguration, the `lttng` tool is able to
+save and load tracing session configurations to/from XML files.
+
+To save a given tracing session configuration, do:
+
+[role="term"]
+----
+lttng save my-session
+----
+
+where `my-session` is the name of the tracing session to save. Tracing
+session configurations are saved to dir:{~/.lttng/sessions} by default;
+use the `--output-path` option to change this destination directory.
+
+All configuration parameters are saved:
+
+* tracing session name
+* trace data output path
+* channels with their state and all their parameters
+* context information added to channels
+* events with their state, log level and filter
+* tracing activity (started or stopped)
+
+To load a tracing session, simply do:
+
+[role="term"]
+----
+lttng load my-session
+----
+
+or, if you used a custom path:
+
+[role="term"]
+----
+lttng load --input-path /path/to/my-session.lttng
+----
+
+Your saved tracing session will be restored as if you just configured
+it manually.
+
+
+[[sending-trace-data-over-the-network]]
+==== Sending trace data over the network
+
+The possibility of sending trace data over the network comes as a
+built-in feature of LTTng-tools. For this to be possible, an LTTng
+_relay daemon_ must be executed and listening on the machine where
+trace data is to be received, and the user must create a tracing
+session using appropriate options to forward trace data to the remote
+relay daemon.
+
+The relay daemon listens on two different TCP ports: one for control
+information and the other for actual trace data.
+
+Starting the relay daemon on the remote machine is as easy as:
+
+[role="term"]
+----
+lttng-relayd
+----
+
+This will make it listen to its default ports: 5342 for control and
+5343 for trace data. The `--control-port` and `--data-port` options may
+be used to specify different ports.
+
+Traces written by `lttng-relayd` are written to
++\~/lttng-traces/__hostname__/__session__+ by
+default, where +__hostname__+ is the host name of the
+traced (monitored) system and +__session__+ is the
+tracing session name. Use the `--output` option to write trace data
+outside dir:{~/lttng-traces}.
+
+On the sending side, a tracing session must be created using the
+`lttng` tool with the `--set-url` option to connect to the distant
+relay daemon:
+
+[role="term"]
+----
+lttng create my-session --set-url net://distant-host
+----
+
+The URL format is described in the output of `lttng create --help`.
+The above example will use the default ports; the `--ctrl-url` and
+`--data-url` options may be used to set the control and data URLs
+individually.
+
+Once this basic setup is completed and the connection is established,
+you may use the `lttng` tool on the target machine as usual; everything
+you do will be transparently forwarded to the remote machine if needed.
+For example, a parameter changing the maximum size of trace files will
+have an effect on the distant relay daemon actually writing the trace.
+
+
+[[lttng-live]]
+==== Viewing events as they arrive
+
+We have seen how trace files may be produced by LTTng out of generated
+application and Linux kernel events. We have seen that those trace files
+may be either recorded locally by consumer daemons or remotely using
+a relay daemon. And we have seen that the maximum size and count of
+trace files is configurable for each channel. With all those features,
+it's still not possible to read a trace file as it is being written
+because it could be incomplete and appear corrupted to the viewer.
+There is a way to view events as they arrive, however: using
+_LTTng live_.
+
+LTTng live is implemented, in LTTng, solely on the relay daemon side.
+As trace data is sent over the network to a relay daemon by a (possibly
+remote) consumer daemon, a _tee_ may be created: trace data will be
+recorded to trace files _as well as_ being transmitted to a
+connected live viewer:
+
+[role="img-90"]
+.LTTng live and the relay daemon.
+image::lttng-live-relayd.png[]
+
+In order to use this feature, a tracing session must created in live
+mode on the target system:
+
+[role="term"]
+----
+lttng create --live
+----
+
+An optional parameter may be passed to `--live` to set the interval
+of time (in microseconds) between flushes to the network
+(1{nbsp}second is the default):
+
+[role="term"]
+----
+lttng create --live 100000
+----
+
+will flush every 100{nbsp}ms.
+
+If no network output is specified to the `create` command, a local
+relay daemon will be spawned. In this very common case, viewing a live
+trace is easy: enable events and start tracing as usual, then use
+`lttng view` to start the default live viewer:
+
+[role="term"]
+----
+lttng view
+----
+
+The correct arguments will be passed to the live viewer so that it
+may connect to the local relay daemon and start reading live events.
+
+You may also wish to use a live viewer not running on the target
+system. In this case, you should specify a network output when using
+the `create` command (`--set-url` or `--ctrl-url`/`--data-url` options).
+A distant LTTng relay daemon should also be started to receive control
+and trace data. By default, `lttng-relayd` listens on 127.0.0.1:5344
+for an LTTng live connection. Otherwise, the desired URL may be
+specified using its `--live-port` option.
+
+The
+http://diamon.org/babeltrace[`babeltrace`]
+viewer supports LTTng live as one of its input formats. `babeltrace` is
+the default viewer when using `lttng view`. To use it manually, first
+list active tracing sessions by doing the following (assuming the relay
+daemon to connect to runs on the same host):
+
+[role="term"]
+----
+babeltrace --input-format lttng-live net://localhost
+----
+
+Then, choose a tracing session and start viewing events as they arrive
+using LTTng live, e.g.:
+
+[role="term"]
+----
+babeltrace --input-format lttng-live net://localhost/host/hostname/my-session
+----
+
+
+[[taking-a-snapshot]]
+==== Taking a snapshot
+
+The normal behavior of LTTng is to record trace data as trace files.
+This is ideal for keeping a long history of events that occurred on
+the target system and applications, but may be too much data in some
+situations. For example, you may wish to trace your application
+continuously until some critical situation happens, in which case you
+would only need the latest few recorded events to perform the desired
+analysis, not multi-gigabyte trace files.
+
+LTTng has an interesting feature called _snapshots_. When creating
+a tracing session in snapshot mode, no trace files are written; the
+tracers' sub-buffers are constantly overwriting the oldest recorded
+events with the newest. At any time, either when the tracers are started
+or stopped, you may take a snapshot of those sub-buffers.
+
+There is no difference between the format of a normal trace file and the
+format of a snapshot: viewers of LTTng traces will also support LTTng
+snapshots. By default, snapshots are written to disk, but they may also
+be sent over the network.
+
+To create a tracing session in snapshot mode, do:
+
+[role="term"]
+----
+lttng create --snapshot my-snapshot-session
+----
+
+Next, enable channels, events and add context to channels as usual.
+Once a tracing session is created in snapshot mode, channels will be
+forced to use the
+<<channel-overwrite-mode-vs-discard-mode,overwrite>> mode
+(`--overwrite` option of the `enable-channel` command; also called
+_flight recorder mode_) and have an `mmap()` channel type
+(`--output mmap`).
+
+Start tracing. When you're ready to take a snapshot, do:
+
+[role="term"]
+----
+lttng snapshot record --name my-snapshot
+----
+
+This will record a snapshot named `my-snapshot` of all channels of
+all domains of the current tracing session. By default, snapshots files
+are recorded in the path returned by `lttng snapshot list-output`. You
+may change this path or decide to send snapshots over the network
+using either:
+
+. an output path/URL specified when creating the tracing session
+ (`lttng create`)
+. an added snapshot output path/URL using
+ `lttng snapshot add-output`
+. an output path/URL provided directly to the
+ `lttng snapshot record` command
+
+Method 3 overrides method 2 which overrides method 1. When specifying
+a URL, a relay daemon must be listening on some machine (see
+<<sending-trace-data-over-the-network,Sending trace data over the network>>).
+
+If you need to make absolutely sure that the output file won't be
+larger than a certain limit, you can set a maximum snapshot size when
+taking it with the `--max-size` option:
+
+[role="term"]
+----
+lttng snapshot record --name my-snapshot --max-size 2M
+----
+
+Older recorded events will be discarded in order to respect this
+maximum size.
+
+
+[[reference]]
+== Reference
+
+This chapter presents various references for LTTng packages such as links
+to online manpages, tables needed by the rest of the text, descriptions
+of library functions, etc.
+
+
+[[online-lttng-manpages]]
+=== Online LTTng manpages
+
+LTTng packages currently install the following manpages, available
+online using the links below:
+
+* **LTTng-tools**
+** man:lttng(1)
+** man:lttng-sessiond(8)
+** man:lttng-relayd(8)
+* **LTTng-UST**
+** man:lttng-gen-tp(1)
+** man:lttng-ust(3)
+** man:lttng-ust-cyg-profile(3)
+** man:lttng-ust-dl(3)
+
+
+[[lttng-ust-ref]]
+=== LTTng-UST
+
+This section presents references of the LTTng-UST package.
+
+
+[[liblttng-ust]]
+==== LTTng-UST library (+liblttng‑ust+)
+
+The LTTng-UST library, or `liblttng-ust`, is the main shared object
+against which user applications are linked to make LTTng user space
+tracing possible.
+
+The <<c-application,C application>> guide shows the complete
+process to instrument, build and run a C/$$C++$$ application using
+LTTng-UST, while this section contains a few important tables.
+
+
+[[liblttng-ust-tp-fields]]
+===== Tracepoint fields macros (for `TP_FIELDS()`)
+
+The available macros to define tracepoint fields, which should be listed
+within `TP_FIELDS()` in `TRACEPOINT_EVENT()`, are:
+
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+.Available macros to define LTTng-UST tracepoint fields
+|====
+|Macro |Description and parameters
+
+|
++ctf_integer(__t__, __n__, __e__)+
+
++ctf_integer_nowrite(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `long`, `size_t`, etc.).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_hex(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network(__t__, __n__, __e__)+
+|
+Integer in network byte order (big endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network_hex(__t__, __n__, __e__)+
+|
+Integer in network byte order, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_float(__t__, __n__, __e__)+
+
++ctf_float_nowrite(__t__, __n__, __e__)+
+|
+Floating point number.
+
++__t__+::
+ Floating point number C type (`float` or `double`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_string(__n__, __e__)+
+
++ctf_string_nowrite(__n__, __e__)+
+|
+Null-terminated string; undefined behavior if +__e__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_array(__t__, __n__, __e__, __s__)+
+
++ctf_array_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array of integers
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_array_text(__t__, __n__, __e__, __s__)+
+
++ctf_array_text_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array, printed as text.
+
+The string does not need to be null-terminated.
+
++__t__+::
+ Array element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_sequence(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers.
+
+The type of +__E__+ needs to be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|
++ctf_sequence_text(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array, displayed as text.
+
+The string does not need to be null-terminated.
+
+The type of +__E__+ needs to be unsigned.
+
+The behaviour is undefined if +__e__+ is `NULL`.
+
++__t__+::
+ Sequence element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+|====
+
+The `_nowrite` versions omit themselves from the session trace, but are
+otherwise identical. This means the `_nowrite` fields won't be written
+in the recorded trace. Their primary purpose is to make some
+of the event context available to the
+<<enabling-disabling-events,event filters>> without having to
+commit the data to sub-buffers.
+
+
+[[liblttng-ust-tracepoint-loglevel]]
+===== Tracepoint log levels (for `TRACEPOINT_LOGLEVEL()`)
+
+The following table shows the available log level values for the
+`TRACEPOINT_LOGLEVEL()` macro:
+
+`TRACE_EMERG`::
+ System is unusable.
+
+`TRACE_ALERT`::
+ Action must be taken immediately.
+
+`TRACE_CRIT`::
+ Critical conditions.
+
+`TRACE_ERR`::
+ Error conditions.
+
+`TRACE_WARNING`::
+ Warning conditions.
+
+`TRACE_NOTICE`::
+ Normal, but significant, condition.
+
+`TRACE_INFO`::
+ Informational message.
+
+`TRACE_DEBUG_SYSTEM`::
+ Debug information with system-level scope (set of programs).
+
+`TRACE_DEBUG_PROGRAM`::
+ Debug information with program-level scope (set of processes).
+
+`TRACE_DEBUG_PROCESS`::
+ Debug information with process-level scope (set of modules).
+
+`TRACE_DEBUG_MODULE`::
+ Debug information with module (executable/library) scope (set of units).
+
+`TRACE_DEBUG_UNIT`::
+ Debug information with compilation unit scope (set of functions).
+
+`TRACE_DEBUG_FUNCTION`::
+ Debug information with function-level scope.
+
+`TRACE_DEBUG_LINE`::
+ Debug information with line-level scope (TRACEPOINT_EVENT default).
+
+`TRACE_DEBUG`::
+ Debug-level message.
+
+Log levels `TRACE_EMERG` through `TRACE_INFO` and `TRACE_DEBUG` match
+http://man7.org/linux/man-pages/man3/syslog.3.html[syslog]
+level semantics. Log levels `TRACE_DEBUG_SYSTEM` through `TRACE_DEBUG`
+offer more fine-grained selection of debug information.
+
+
+[[lttng-modules-ref]]
+=== LTTng-modules
+
+This section presents references of the LTTng-modules package.
+
+
+[[lttng-modules-tp-struct-entry]]
+==== Tracepoint fields macros (for `TP_STRUCT__entry()`)
+
+This table describes possible entries for the `TP_STRUCT__entry()` part
+of `LTTNG_TRACEPOINT_EVENT()`:
+
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+.Available entries for `TP_STRUCT__entry()` (in `LTTNG_TRACEPOINT_EVENT()`)
+|====
+|Macro |Description and parameters
+
+|+\__field(__t__, __n__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `unsigned char`, `size_t`, etc.).
+
++__n__+::
+ Field name.
+
+|+\__field_hex(__t__, __n__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_oct(__t__, __n__)+
+|
+Standard integer, displayed in base 8.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_network(__t__, __n__)+
+|
+Integer in network byte order (big endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_network_hex(__t__, __n__)+
+|
+Integer in network byte order (big endian), displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__array(__t__, __n__, __s__)+
+|
+Statically-sized array, elements displayed in base 10.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Number of elements.
+
+|+\__array_hex(__t__, __n__, __s__)+
+|
+Statically-sized array, elements displayed in base 16.
+
++__t__+::
+ array element C type.
++__n__+::
+ field name.
++__s__+::
+ number of elements.
+
+|+\__array_text(__t__, __n__, __s__)+
+|
+Statically-sized array, displayed as text.
+
++__t__+::
+ Array element C type (always char).
+
++__n__+::
+ Field name.
+
++__s__+::
+ Number of elements.
+
+|+\__dynamic_array(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed in base 10.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__dynamic_array_hex(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed in base 16.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__dynamic_array_text(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed as text.
+
++__t__+::
+ Array element C type (always char).
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__string(n, __s__)+
+|
+Null-terminated string.
+
+The behaviour is undefined behavior if +__s__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__s__+::
+ String source (pointer).
+|====
+
+The above macros should cover the majority of cases. For advanced items,
+see path:{probes/lttng-events.h}.
+
+
+[[lttng-modules-tp-fast-assign]]
+==== Tracepoint assignment macros (for `TP_fast_assign()`)
+
+This table describes possible entries for the `TP_fast_assign()` part
+of `LTTNG_TRACEPOINT_EVENT()`:
+
+.Available entries for `TP_fast_assign()` (in `LTTNG_TRACEPOINT_EVENT()`)
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+|====
+|Macro |Description and parameters
+
+|+tp_assign(__d__, __s__)+
+|
+Assignment of C expression +__s__+ to tracepoint field +__d__+.
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
+|+tp_memcpy(__d__, __s__, __l__)+
+|
+Memory copy of +__l__+ bytes from +__s__+ to tracepoint field
++__d__+ (use with array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_memcpy_from_user(__d__, __s__, __l__)+
+|
+Memory copy of +__l__+ bytes from user space +__s__+ to tracepoint
+field +__d__+ (use with array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_memcpy_dyn(__d__, __s__)+
+|
+Memory copy of dynamically-sized array from +__s__+ to tracepoint field
++__d__+.
+
+The number of bytes is known from the field's length expression
+(use with dynamically-sized array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_strcpy(__d__, __s__)+
+|
+String copy of +__s__+ to tracepoint field +__d__+ (use with string
+fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+|====
--- /dev/null
+The LTTng Documentation
+=======================
+Philippe Proulx <pproulx@efficios.com>
+v2.6, May 26, 2016
+
+
+include::../common/copyright.txt[]
+
+
+include::../common/warning-not-maintained.txt[]
+
+
+include::../common/welcome.txt[]
+
+
+include::../common/audience.txt[]
+
+
+[[chapters]]
+=== Chapter descriptions
+
+What follows is a list of brief descriptions of this documentation's
+chapters. The latter are ordered in such a way as to make the reading
+as linear as possible.
+
+. <<nuts-and-bolts,Nuts and bolts>> explains the
+ rudiments of software tracing and the rationale behind the
+ LTTng project.
+. <<installing-lttng,Installing LTTng>> is divided into
+ sections describing the steps needed to get a working installation
+ of LTTng packages for common Linux distributions and from its
+ source.
+. <<getting-started,Getting started>> is a very concise guide to
+ get started quickly with LTTng kernel and user space tracing. This
+ chapter is recommended if you're new to LTTng or software tracing
+ in general.
+. <<understanding-lttng,Understanding LTTng>> deals with some
+ core concepts and components of the LTTng suite. Understanding
+ those is important since the next chapter assumes you're familiar
+ with them.
+. <<using-lttng,Using LTTng>> is a complete user guide of the
+ LTTng project. It shows in great details how to instrument user
+ applications and the Linux kernel, how to control tracing sessions
+ using the `lttng` command line tool and miscellaneous practical use
+ cases.
+. <<reference,Reference>> contains references of LTTng components,
+ like links to online manpages and various APIs.
+
+We recommend that you read the above chapters in this order, although
+some of them may be skipped depending on your situation. You may skip
+<<nuts-and-bolts,Nuts and bolts>> if you're familiar with tracing
+and LTTng. Also, you may jump over <<installing-lttng,Installing LTTng>>
+if LTTng is already properly installed on your target system.
+
+
+include::../common/convention.txt[]
+
+
+include::../common/acknowledgements.txt[]
+
+
+[[whats-new]]
+== What's new in LTTng {revision}?
+
+Most of the changes of LTTng {revision} are bug fixes, making the toolchain
+more stable than ever before. Still, LTTng {revision} adds some interesting
+features to the project.
+
+LTTng 2.5 already supported the instrumentation and tracing of
+<<java-application,Java applications>> through `java.util.logging`
+(JUL). LTTng {revision} goes one step further by supporting
+https://logging.apache.org/log4j/1.2/[Apache log4j 1.2].
+The new log4j domain is selected using the `--log4j` option in various
+commands of the `lttng` tool.
+
+LTTng-modules has supported system call tracing for a long time,
+but until now, it was only possible to record either all of them,
+or none of them. LTTng {revision} allows the user to record specific
+system call events, for example:
+
+[role="term"]
+----
+lttng enable-event --kernel --syscall open,fork,chdir,pipe
+----
+
+Finally, the `lttng` command line tool is not only able to communicate
+with humans as it used to do, but also with machines thanks to its new
+<<mi,machine interface>> feature.
+
+To learn more about the new features of LTTng {revision}, see the
+http://lttng.org/blog/2015/02/27/lttng-2.6-released/[release announcement].
+
+
+[[nuts-and-bolts]]
+== Nuts and bolts
+
+What is LTTng? As its name suggests, the _Linux Trace Toolkit: next
+generation_ is a modern toolkit for tracing Linux systems and
+applications. So your first question might rather be: **what is
+tracing?**
+
+
+[[what-is-tracing]]
+=== What is tracing?
+
+As the history of software engineering progressed and led to what
+we now take for granted--complex, numerous and
+interdependent software applications running in parallel on
+sophisticated operating systems like Linux--the authors of such
+components, or software developers, began feeling a natural
+urge of having tools to ensure the robustness and good performance
+of their masterpieces.
+
+One major achievement in this field is, inarguably, the
+https://www.gnu.org/software/gdb/[GNU debugger (GDB)],
+which is an essential tool for developers to find and fix
+bugs. But even the best debugger won't help make your software run
+faster, and nowadays, faster software means either more work done by
+the same hardware, or cheaper hardware for the same work.
+
+A _profiler_ is often the tool of choice to identify performance
+bottlenecks. Profiling is suitable to identify _where_ performance is
+lost in a given software; the profiler outputs a profile, a
+statistical summary of observed events, which you may use to discover
+which functions took the most time to execute. However, a profiler
+won't report _why_ some identified functions are the bottleneck.
+Bottlenecks might only occur when specific conditions are met, sometimes
+almost impossible to capture by a statistical profiler, or impossible to
+reproduce with an application altered by the overhead of an event-based
+profiler. For a thorough investigation of software performance issues,
+a history of execution, with the recorded values of chosen variables
+and context, is essential. This is where tracing comes in handy.
+
+_Tracing_ is a technique used to understand what goes on in a running
+software system. The software used for tracing is called a _tracer_,
+which is conceptually similar to a tape recorder. When recording,
+specific probes placed in the software source code generate events
+that are saved on a giant tape: a _trace_ file. Both user applications
+and the operating system may be traced at the same time, opening the
+possibility of resolving a wide range of problems that are otherwise
+extremely challenging.
+
+Tracing is often compared to _logging_. However, tracers and loggers
+are two different tools, serving two different purposes. Tracers are
+designed to record much lower-level events that occur much more
+frequently than log messages, often in the thousands per second range,
+with very little execution overhead. Logging is more appropriate for
+very high-level analysis of less frequent events: user accesses,
+exceptional conditions (errors and warnings, for example), database
+transactions, instant messaging communications, and such. More formally,
+logging is one of several use cases that can be accomplished with
+tracing.
+
+The list of recorded events inside a trace file may be read manually
+like a log file for the maximum level of detail, but it is generally
+much more interesting to perform application-specific analyses to
+produce reduced statistics and graphs that are useful to resolve a
+given problem. Trace viewers and analysers are specialized tools
+designed to do this.
+
+So, in the end, this is what LTTng is: a powerful, open source set of
+tools to trace the Linux kernel and user applications at the same time.
+LTTng is composed of several components actively maintained and
+developed by its link:/community/#where[community].
+
+
+[[lttng-alternatives]]
+=== Alternatives to LTTng
+
+Excluding proprietary solutions, a few competing software tracers
+exist for Linux:
+
+* https://www.kernel.org/doc/Documentation/trace/ftrace.txt[ftrace]
+ is the de facto function tracer of the Linux kernel. Its user
+ interface is a set of special files in sysfs.
+* https://perf.wiki.kernel.org/[perf] is
+ a performance analyzing tool for Linux which supports hardware
+ performance counters, tracepoints, as well as other counters and
+ types of probes. perf's controlling utility is the `perf` command
+ line/curses tool.
+* http://linux.die.net/man/1/strace[strace]
+ is a command line utility which records system calls made by a
+ user process, as well as signal deliveries and changes of process
+ state. strace makes use of https://en.wikipedia.org/wiki/Ptrace[ptrace]
+ to fulfill its function.
+* https://sourceware.org/systemtap/[SystemTap]
+ is a Linux kernel and user space tracer which uses custom user scripts
+ to produce plain text traces. Scripts are converted to the C language,
+ then compiled as Linux kernel modules which are loaded to produce
+ trace data. SystemTap's primary user interface is the `stap`
+ command line tool.
+* http://www.sysdig.org/[sysdig], like
+ SystemTap, uses scripts to analyze Linux kernel events. Scripts,
+ or _chisels_ in sysdig's jargon, are written in Lua and executed
+ while the system is being traced, or afterwards. sysdig's interface
+ is the `sysdig` command line tool as well as the curses-based
+ `csysdig` tool.
+
+The main distinctive features of LTTng is that it produces correlated
+kernel and user space traces, as well as doing so with the lowest
+overhead amongst other solutions. It produces trace files in the
+http://diamon.org/ctf[CTF] format, an optimized file format
+for production and analyses of multi-gigabyte data. LTTng is the
+result of close to 10 years of
+active development by a community of passionate developers. LTTng {revision}
+is currently available on some major desktop, server, and embedded Linux
+distributions.
+
+The main interface for tracing control is a single command line tool
+named `lttng`. The latter can create several tracing sessions,
+enable/disable events on the fly, filter them efficiently with custom
+user expressions, start/stop tracing, and do much more. Traces can be
+recorded on disk or sent over the network, kept totally or partially,
+and viewed once tracing becomes inactive or in real-time.
+
+<<installing-lttng,Install LTTng now>> and start tracing!
+
+
+[[installing-lttng]]
+== Installing LTTng
+
+**LTTng** is a set of software components which interact to allow
+instrumenting the Linux kernel and user applications as well as
+controlling tracing sessions (starting/stopping tracing,
+enabling/disabling events, and more). Those components are bundled into
+the following packages:
+
+LTTng-tools::
+ Libraries and command line interface to control tracing sessions.
+
+LTTng-modules::
+ Linux kernel modules for tracing the kernel.
+
+LTTng-UST::
+ User space tracing library.
+
+Most distributions mark the LTTng-modules and LTTng-UST packages as
+optional. In the following sections, the steps to install all three are
+always provided, but note that LTTng-modules is only required if
+you intend to trace the Linux kernel and LTTng-UST is only required if
+you intend to trace user space applications.
+
+This chapter shows how to install the above packages on a Linux system.
+The easiest way is to use the package manager of the system's
+distribution (<<desktop-distributions,desktop>> or
+<<embedded-distributions,embedded>>). Support is also available for
+<<enterprise-distributions,enterprise distributions>>, such as Red Hat
+Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SLES).
+Otherwise, you can
+<<building-from-source,build the LTTng packages from source>>.
+
+
+[[desktop-distributions]]
+=== Desktop distributions
+
+Official LTTng {revision} packages are available for
+<<ubuntu,Ubuntu>>, <<fedora,Fedora>>, and
+<<opensuse,openSUSE>> (and other RPM-based distributions).
+
+More recent versions of LTTng are available for Debian and Arch Linux.
+
+Should any issue arise when
+following the procedures below, please inform the
+link:/community[community] about it.
+
+
+[[ubuntu]]
+==== Ubuntu
+
+LTTng {revision} is packaged in Ubuntu 15.10 _Wily Werewolf_. For other
+releases of Ubuntu, you need to build and install LTTng {revision}
+<<building-from-source,from source>>. Ubuntu 15.04 _Vivid Vervet_
+ships with link:/docs/v2.5/[LTTng 2.5], whilst
+Ubuntu 16.04 _Xenial Xerus_ ships with
+link:/docs/v2.7/[LTTng 2.7].
+
+To install LTTng {revision} from the official Ubuntu repositories,
+simply use `apt-get`:
+
+[role="term"]
+----
+sudo apt-get install lttng-tools
+sudo apt-get install lttng-modules-dkms
+sudo apt-get install liblttng-ust-dev
+----
+
+If you need to trace
+<<java-application,Java applications>>,
+you need to install the LTTng-UST Java agent also:
+
+[role="term"]
+----
+sudo apt-get install liblttng-ust-agent-java
+----
+
+
+[[fedora]]
+==== Fedora
+
+Fedora 22 and Fedora 23 ship with official LTTng-tools {revision} and
+LTTng-UST {revision} packages. Simply use `yum`:
+
+[role="term"]
+----
+sudo yum install lttng-tools
+sudo yum install lttng-ust
+sudo yum install lttng-ust-devel
+----
+
+LTTng-modules {revision} still needs to be built and installed from
+source. For that, make sure that the `kernel-devel` package is
+already installed beforehand:
+
+[role="term"]
+----
+sudo yum install kernel-devel
+----
+
+Proceed on to fetch
+<<building-from-source,LTTng-modules {revision}'s source>>. Build and
+install it as follows:
+
+[role="term"]
+----
+KERNELDIR=/usr/src/kernels/$(uname -r) make
+sudo make modules_install
+----
+
+NOTE: If you need to trace <<java-application,Java applications>> on
+Fedora, you need to build and install LTTng-UST {revision}
+<<building-from-source,from source>> and use the
+`--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options.
+
+
+[[opensuse]]
+==== openSUSE/RPM
+
+openSUSE 13.1 and openSUSE 13.2 have LTTng {revision} packages. To install
+LTTng {revision}, you first need to add an entry to your repository
+configuration. All LTTng repositories are available
+http://download.opensuse.org/repositories/devel:/tools:/lttng/[here].
+For example, the following commands adds the LTTng repository for
+openSUSE{nbsp}13.1:
+
+[role="term"]
+----
+sudo zypper addrepo http://download.opensuse.org/repositories/devel:/tools:/lttng/openSUSE_13.1/devel:tools:lttng.repo
+----
+
+Then, refresh the package database:
+
+[role="term"]
+----
+sudo zypper refresh
+----
+
+and install `lttng-tools`, `lttng-modules` and `lttng-ust-devel`:
+
+[role="term"]
+----
+sudo zypper install lttng-tools
+sudo zypper install lttng-modules
+sudo zypper install lttng-ust-devel
+----
+
+NOTE: If you need to trace <<java-application,Java applications>> on
+openSUSE, you need to build and install LTTng-UST {revision}
+<<building-from-source,from source>> and use the
+`--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options.
+
+
+[[embedded-distributions]]
+=== Embedded distributions
+
+LTTng is packaged by two popular
+embedded Linux distributions: <<buildroot,Buildroot>> and
+<<oe-yocto,OpenEmbedded/Yocto>>.
+
+
+[[buildroot]]
+==== Buildroot
+
+LTTng {revision} is available in Buildroot since Buildroot 2015.05. The
+LTTng packages are named `lttng-tools`, `lttng-modules`, and `lttng-libust`.
+
+To enable them, start the Buildroot configuration menu as usual:
+
+[role="term"]
+----
+make menuconfig
+----
+
+In:
+
+* _Kernel_: make sure _Linux kernel_ is enabled
+* _Toolchain_: make sure the following options are enabled:
+** _Enable large file (files > 2GB) support_
+** _Enable WCHAR support_
+
+In _Target packages_/_Debugging, profiling and benchmark_, enable
+_lttng-modules_ and _lttng-tools_. In
+_Target packages_/_Libraries_/_Other_, enable _lttng-libust_.
+
+NOTE: If you need to trace <<java-application,Java applications>> on
+Buildroot, you need to build and install LTTng-UST {revision}
+<<building-from-source,from source>> and use the
+`--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options.
+
+
+[[oe-yocto]]
+==== OpenEmbedded/Yocto
+
+LTTng {revision} recipes are available in the
+http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/[`openembedded-core`]
+layer of OpenEmbedded since February 8th, 2015 under the following names:
+
+* `lttng-tools`
+* `lttng-modules`
+* `lttng-ust`
+
+Using BitBake, the simplest way to include LTTng recipes in your
+target image is to add them to `IMAGE_INSTALL_append` in
+path:{conf/local.conf}:
+
+----
+IMAGE_INSTALL_append = " lttng-tools lttng-modules lttng-ust"
+----
+
+If you're using Hob, click _Edit image recipe_ once you have selected
+a machine and an image recipe. Then, under the _All recipes_ tab, search
+for `lttng` and include the three LTTng recipes.
+
+NOTE: If you need to trace <<java-application,Java applications>> on
+OpenEmbedded/Yocto, you need to build and install LTTng-UST {revision}
+<<building-from-source,from source>> and use the
+`--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options.
+
+
+[[enterprise-distributions]]
+=== Enterprise distributions (RHEL, SLES)
+
+To install LTTng on enterprise Linux distributions
+(such as RHEL and SLES), please see
+http://packages.efficios.com/[EfficiOS Enterprise Packages].
+
+
+[[building-from-source]]
+=== Building from source
+
+As <<installing-lttng,previously stated>>, LTTng is shipped as
+three packages: LTTng-tools, LTTng-modules, and LTTng-UST. LTTng-tools
+contains everything needed to control tracing sessions, while
+LTTng-modules is only needed for Linux kernel tracing and LTTng-UST is
+only needed for user space tracing.
+
+The tarballs are available in the
+http://lttng.org/download#build-from-source[Download section]
+of the LTTng website.
+
+Please refer to the path:{README.md} files provided by each package to
+properly build and install them.
+
+TIP: The aforementioned path:{README.md} files
+are rendered as rich text when https://github.com/lttng[viewed on GitHub].
+
+
+[[getting-started]]
+== Getting started with LTTng
+
+This is a small guide to get started quickly with LTTng kernel and user
+space tracing. For a more thorough understanding of LTTng and intermediate
+to advanced use cases and, see <<understanding-lttng,Understanding LTTng>>
+and <<using-lttng,Using LTTng>>.
+
+Before reading this guide, make sure LTTng
+<<installing-lttng,is installed>>. LTTng-tools is required. Also install
+LTTng-modules for
+<<tracing-the-linux-kernel,tracing the Linux kernel>> and LTTng-UST
+for
+<<tracing-your-own-user-application,tracing your own user space applications>>.
+When the traces are finally written and complete, the
+<<viewing-and-analyzing-your-traces,Viewing and analyzing your traces>>
+section of this chapter will help you analyze your tracepoint events
+to investigate.
+
+
+[[tracing-the-linux-kernel]]
+=== Tracing the Linux kernel
+
+Make sure LTTng-tools and LTTng-modules packages
+<<installing-lttng,are installed>>.
+
+Since you're about to trace the Linux kernel itself, let's look at the
+available kernel events using the `lttng` tool, which has a
+Git-like command line structure:
+
+[role="term"]
+----
+lttng list --kernel
+----
+
+Before tracing, you need to create a session:
+
+[role="term"]
+----
+sudo lttng create
+----
+
+TIP: You can avoid using `sudo` in the previous and following commands
+if your user is a member of the <<lttng-sessiond,tracing group>>.
+
+Let's now enable some events for this session:
+
+[role="term"]
+----
+sudo lttng enable-event --kernel sched_switch,sched_process_fork
+----
+
+Or you might want to simply enable all available kernel events (beware
+that trace files grow rapidly when doing this):
+
+[role="term"]
+----
+sudo lttng enable-event --kernel --all
+----
+
+Start tracing:
+
+[role="term"]
+----
+sudo lttng start
+----
+
+By default, traces are saved in
++\~/lttng-traces/__name__-__date__-__time__+,
+where +__name__+ is the session name.
+
+When you're done tracing:
+
+[role="term"]
+----
+sudo lttng stop
+sudo lttng destroy
+----
+
+Although `destroy` looks scary here, it doesn't actually destroy the
+written trace files: it only destroys the tracing session.
+
+What's next? Have a look at
+<<viewing-and-analyzing-your-traces,Viewing and analyzing your traces>>
+to view and analyze the trace you just recorded.
+
+
+[[tracing-your-own-user-application]]
+=== Tracing your own user application
+
+The previous section helped you create a trace out of Linux kernel
+events. This section steps you through a simple example showing you how
+to trace a _Hello world_ program written in C.
+
+Make sure the LTTng-tools and LTTng-UST packages
+<<installing-lttng,are installed>>.
+
+Tracing is just like having `printf()` calls at specific locations of
+your source code, albeit LTTng is much faster and more flexible than
+`printf()`. In the LTTng realm, **`tracepoint()`** is analogous to
+`printf()`.
+
+Unlike `printf()`, though, `tracepoint()` does not use a format string to
+know the types of its arguments: the formats of all tracepoints must be
+defined before using them. So before even writing our _Hello world_ program,
+we need to define the format of our tracepoint. This is done by creating a
+**tracepoint provider**, which consists of a tracepoint provider header
+(`.h` file) and a tracepoint provider definition (`.c` file).
+
+The tracepoint provider header contains some boilerplate as well as a
+list of tracepoint definitions and other optional definition entries
+which we skip for this quickstart. Each tracepoint is defined using the
+`TRACEPOINT_EVENT()` macro. For each tracepoint, you must provide:
+
+* a **provider name**, which is the "scope" or namespace of this
+ tracepoint (this usually includes the company and project names)
+* a **tracepoint name**
+* a **list of arguments** for the eventual `tracepoint()` call, each
+ item being:
+** the argument C type
+** the argument name
+* a **list of fields**, which correspond to the actual fields of the
+ recorded events for this tracepoint
+
+Here's an example of a simple tracepoint provider header with two
+arguments: an integer and a string:
+
+[source,c]
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER hello_world
+
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./hello-tp.h"
+
+#if !defined(_HELLO_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _HELLO_TP_H
+
+#include <lttng/tracepoint.h>
+
+TRACEPOINT_EVENT(
+ hello_world,
+ my_first_tracepoint,
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+
+#endif /* _HELLO_TP_H */
+
+#include <lttng/tracepoint-event.h>
+----
+
+The exact syntax is well explained in the
+<<c-application,C application>> instrumentation guide of the
+<<using-lttng,Using LTTng>> chapter, as well as in
+man:lttng-ust(3).
+
+Save the above snippet as path:{hello-tp.h}.
+
+Write the tracepoint provider definition as path:{hello-tp.c}:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+
+#include "hello-tp.h"
+----
+
+Create the tracepoint provider:
+
+[role="term"]
+----
+gcc -c -I. hello-tp.c
+----
+
+Now, by including path:{hello-tp.h} in your own application, you may use the
+tracepoint defined above by properly refering to it when calling
+`tracepoint()`:
+
+[source,c]
+----
+#include <stdio.h>
+#include "hello-tp.h"
+
+int main(int argc, char *argv[])
+{
+ int x;
+
+ puts("Hello, World!\nPress Enter to continue...");
+
+ /*
+ * The following getchar() call is only placed here for the purpose
+ * of this demonstration, for pausing the application in order for
+ * you to have time to list its events. It's not needed otherwise.
+ */
+ getchar();
+
+ /*
+ * A tracepoint() call. Arguments, as defined in hello-tp.h:
+ *
+ * 1st: provider name (always)
+ * 2nd: tracepoint name (always)
+ * 3rd: my_integer_arg (first user-defined argument)
+ * 4th: my_string_arg (second user-defined argument)
+ *
+ * Notice the provider and tracepoint names are NOT strings;
+ * they are in fact parts of variables created by macros in
+ * hello-tp.h.
+ */
+ tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");
+
+ for (x = 0; x < argc; ++x) {
+ tracepoint(hello_world, my_first_tracepoint, x, argv[x]);
+ }
+
+ puts("Quitting now!");
+
+ tracepoint(hello_world, my_first_tracepoint, x * x, "x^2");
+
+ return 0;
+}
+----
+
+Save this as path:{hello.c}, next to path:{hello-tp.c}.
+
+Notice path:{hello-tp.h}, the tracepoint provider header, is included
+by path:{hello.c}.
+
+You are now ready to compile the application with LTTng-UST support:
+
+[role="term"]
+----
+gcc -c hello.c
+gcc -o hello hello.o hello-tp.o -llttng-ust -ldl
+----
+
+Here's the whole build process:
+
+[role="img-100"]
+.User space tracing's build process.
+image::ust-flow.png[]
+
+If you followed the
+<<tracing-the-linux-kernel,Tracing the Linux kernel>> tutorial, the
+following steps should look familiar.
+
+First, run the application with a few arguments:
+
+[role="term"]
+----
+./hello world and beyond
+----
+
+You should see
+
+----
+Hello, World!
+Press Enter to continue...
+----
+
+Use the `lttng` tool to list all available user space events:
+
+[role="term"]
+----
+lttng list --userspace
+----
+
+You should see the `hello_world:my_first_tracepoint` tracepoint listed
+under the `./hello` process.
+
+Create a tracing session:
+
+[role="term"]
+----
+lttng create
+----
+
+Enable the `hello_world:my_first_tracepoint` tracepoint:
+
+[role="term"]
+----
+lttng enable-event --userspace hello_world:my_first_tracepoint
+----
+
+Start tracing:
+
+[role="term"]
+----
+lttng start
+----
+
+Go back to the running `hello` application and press Enter. All `tracepoint()`
+calls are executed and the program finally exits.
+
+Stop tracing:
+
+[role="term"]
+----
+lttng stop
+----
+
+Done! You may use `lttng view` to list the recorded events. This command
+starts http://diamon.org/babeltrace[`babeltrace`]
+in the background, if it's installed:
+
+[role="term"]
+----
+lttng view
+----
+
+should output something like:
+
+----
+[18:10:27.684304496] (+?.?????????) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "hi there!", my_integer_field = 23 }
+[18:10:27.684338440] (+0.000033944) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "./hello", my_integer_field = 0 }
+[18:10:27.684340692] (+0.000002252) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "world", my_integer_field = 1 }
+[18:10:27.684342616] (+0.000001924) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "and", my_integer_field = 2 }
+[18:10:27.684343518] (+0.000000902) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "beyond", my_integer_field = 3 }
+[18:10:27.684357978] (+0.000014460) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "x^2", my_integer_field = 16 }
+----
+
+When you're done, you may destroy the tracing session, which does _not_
+destroy the generated trace files, leaving them available for further
+analysis:
+
+[role="term"]
+----
+lttng destroy
+----
+
+The next section presents other alternatives to view and analyze your
+LTTng traces.
+
+
+[[viewing-and-analyzing-your-traces]]
+=== Viewing and analyzing your traces
+
+This section describes how to visualize the data gathered after tracing
+the Linux kernel or a user space application.
+
+Many ways exist to read LTTng traces:
+
+* **`babeltrace`** is a command line utility which converts trace formats;
+ it supports the format used by LTTng,
+ CTF, as well as a basic
+ text output which may be ++grep++ed. The `babeltrace` command is
+ part of the
+ http://diamon.org/babeltrace[Babeltrace] project.
+* Babeltrace also includes **Python bindings** so that you may
+ easily open and read an LTTng trace with your own script, benefiting
+ from the power of Python.
+* **http://tracecompass.org/[Trace Compass]**
+ is an Eclipse plugin used to visualize and analyze various types of
+ traces, including LTTng's. It also comes as a standalone application.
+
+LTTng trace files are usually recorded in the dir:{~/lttng-traces} directory.
+Let's now view the trace and perform a basic analysis using
+`babeltrace`.
+
+The simplest way to list all the recorded events of a trace is to pass its
+path to `babeltrace` with no options:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-session
+----
+
+`babeltrace` finds all traces recursively within the given path and
+prints all their events, merging them in order of time.
+
+Listing all the system calls of a Linux kernel trace with their arguments is
+easy with `babeltrace` and `grep`:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session | grep sys_
+----
+
+Counting events is also straightforward:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session | grep sys_read | wc --lines
+----
+
+The text output of `babeltrace` is useful for isolating events by simple
+matching using `grep` and similar utilities. However, more elaborate filters
+such as keeping only events with a field value falling within a specific range
+are not trivial to write using a shell. Moreover, reductions and even the
+most basic computations involving multiple events are virtually impossible
+to implement.
+
+Fortunately, Babeltrace ships with Python 3 bindings which makes it
+really easy to read the events of an LTTng trace sequentially and compute
+the desired information.
+
+Here's a simple example using the Babeltrace Python bindings. The following
+script accepts an LTTng Linux kernel trace path as its first argument and
+prints the short names of the top 5 running processes on CPU 0 during the
+whole trace:
+
+[source,python]
+----
+import sys
+from collections import Counter
+import babeltrace
+
+
+def top5proc():
+ if len(sys.argv) != 2:
+ msg = 'Usage: python {} TRACEPATH'.format(sys.argv[0])
+ raise ValueError(msg)
+
+ # a trace collection holds one to many traces
+ col = babeltrace.TraceCollection()
+
+ # add the trace provided by the user
+ # (LTTng traces always have the 'ctf' format)
+ if col.add_trace(sys.argv[1], 'ctf') is None:
+ raise RuntimeError('Cannot add trace')
+
+ # this counter dict will hold execution times:
+ #
+ # task command name -> total execution time (ns)
+ exec_times = Counter()
+
+ # this holds the last `sched_switch` timestamp
+ last_ts = None
+
+ # iterate events
+ for event in col.events:
+ # keep only `sched_switch` events
+ if event.name != 'sched_switch':
+ continue
+
+ # keep only events which happened on CPU 0
+ if event['cpu_id'] != 0:
+ continue
+
+ # event timestamp
+ cur_ts = event.timestamp
+
+ if last_ts is None:
+ # we start here
+ last_ts = cur_ts
+
+ # previous task command (short) name
+ prev_comm = event['prev_comm']
+
+ # initialize entry in our dict if not yet done
+ if prev_comm not in exec_times:
+ exec_times[prev_comm] = 0
+
+ # compute previous command execution time
+ diff = cur_ts - last_ts
+
+ # update execution time of this command
+ exec_times[prev_comm] += diff
+
+ # update last timestamp
+ last_ts = cur_ts
+
+ # display top 10
+ for name, ns in exec_times.most_common(5):
+ s = ns / 1000000000
+ print('{:20}{} s'.format(name, s))
+
+
+if __name__ == '__main__':
+ top5proc()
+----
+
+Save this script as path:{top5proc.py} and run it with Python 3, providing the
+path to an LTTng Linux kernel trace as the first argument:
+
+[role="term"]
+----
+python3 top5proc.py ~/lttng-sessions/my-session-.../kernel
+----
+
+Make sure the path you provide is the directory containing actual trace
+files (`channel0_0`, `metadata`, and the rest): the `babeltrace` utility
+recurses directories, but the Python bindings do not.
+
+Here's an example of output:
+
+----
+swapper/0 48.607245889 s
+chromium 7.192738188 s
+pavucontrol 0.709894415 s
+Compositor 0.660867933 s
+Xorg.bin 0.616753786 s
+----
+
+Note that `swapper/0` is the "idle" process of CPU 0 on Linux; since we
+weren't using the CPU that much when tracing, its first position in the list
+makes sense.
+
+
+[[understanding-lttng]]
+== Understanding LTTng
+
+If you're going to use LTTng in any serious way, it is fundamental that
+you become familiar with its core concepts. Technical terms like
+_tracing sessions_, _domains_, _channels_ and _events_ are used over
+and over in the <<using-lttng,Using LTTng>> chapter,
+and it is assumed that you understand what they mean when reading it.
+
+LTTng, as you already know, is a _toolkit_. It would be wrong
+to call it a simple _tool_ since it is composed of multiple interacting
+components. This chapter also describes the latter, providing details
+about their respective roles and how they connect together to form
+the current LTTng ecosystem.
+
+
+[[core-concepts]]
+=== Core concepts
+
+This section explains the various elementary concepts a user has to deal
+with when using LTTng. They are:
+
+* <<tracing-session,tracing session>>
+* <<domain,domain>>
+* <<channel,channel>>
+* <<event,event>>
+
+
+[[tracing-session]]
+==== Tracing session
+
+A _tracing session_ is--like any session--a container of
+state. Anything that is done when tracing using LTTng happens in the
+scope of a tracing session. In this regard, it is analogous to a bank
+website's session: you can't interact online with your bank account
+unless you are logged in a session, except for reading a few static
+webpages (LTTng, too, can report some static information that does not
+need a created tracing session).
+
+A tracing session holds the following attributes and objects (some of
+which are described in the following sections):
+
+* a name
+* the tracing state (tracing started or stopped)
+* the trace data output path/URL (local path or sent over the network)
+* a mode (normal, snapshot or live)
+* the snapshot output paths/URLs (if applicable)
+* for each <<domain,domain>>, a list of <<channel,channels>>
+* for each channel:
+** a name
+** the channel state (enabled or disabled)
+** its parameters (event loss mode, sub-buffers size and count,
+ timer periods, output type, trace files size and count, and the rest)
+** a list of added context information
+** a list of <<event,events>>
+* for each event:
+** its state (enabled or disabled)
+** a list of instrumentation points (tracepoints, system calls,
+ dynamic probes, other types of probes)
+** associated log levels
+** a filter expression
+
+All this information is completely isolated between tracing sessions.
+As you can see in the list above, even the tracing state
+is a per-tracing session attribute, so that you may trace your target
+system/application in a given tracing session with a specific
+configuration while another one stays inactive.
+
+[role="img-100"]
+.A _tracing session_ is a container of domains, channels, and events.
+image::concepts.png[]
+
+Conceptually, a tracing session is a per-user object; the
+<<plumbing,Plumbing>> section shows how this is actually
+implemented. Any user may create as many concurrent tracing sessions
+as desired.
+
+[role="img-100"]
+.Each user may create as many tracing sessions as desired.
+image::many-sessions.png[]
+
+The trace data generated in a tracing session may be either saved
+to disk, sent over the network or not saved at all (in which case
+snapshots may still be saved to disk or sent to a remote machine).
+
+
+[[domain]]
+==== Domain
+
+A tracing _domain_ is the official term the LTTng project uses to
+designate a tracer category.
+
+There are currently four known domains:
+
+* Linux kernel
+* user space
+* `java.util.logging` (JUL)
+* log4j
+
+Different tracers expose common features in their own interfaces, but,
+from a user's perspective, you still need to target a specific type of
+tracer to perform some actions. For example, since both kernel and user
+space tracers support named tracepoints (probes manually inserted in
+source code), you need to specify which one is concerned when enabling
+an event because both domains could have existing events with the same
+name.
+
+Some features are not available in all domains. Filtering enabled
+events using custom expressions, for example, is currently not
+supported in the kernel domain, but support could be added in the
+future.
+
+
+[[channel]]
+==== Channel
+
+A _channel_ is a set of events with specific parameters and potential
+added context information. Channels have unique names per domain within
+a tracing session. A given event is always registered to at least one
+channel; having the same enabled event in two channels makes
+this event being recorded twice everytime it occurs.
+
+Channels may be individually enabled or disabled. Occurring events of
+a disabled channel never make it to recorded events.
+
+The fundamental role of a channel is to keep a shared ring buffer, where
+events are eventually recorded by the tracer and consumed by a consumer
+daemon. This internal ring buffer is divided into many sub-buffers of
+equal size.
+
+Channels, when created, may be fine-tuned thanks to a few parameters,
+many of them related to sub-buffers. The following subsections explain
+what those parameters are and in which situations you should manually
+adjust them.
+
+
+[[channel-overwrite-mode-vs-discard-mode]]
+===== Overwrite and discard event loss modes
+
+As previously mentioned, a channel's ring buffer is divided into many
+equally sized sub-buffers.
+
+As events occur, they are serialized as trace data into a specific
+sub-buffer (yellow arc in the following animation) until it is full:
+when this happens, the sub-buffer is marked as consumable (red) and
+another, _empty_ (white) sub-buffer starts receiving the following
+events. The marked sub-buffer is eventually consumed by a consumer
+daemon (returns to white).
+
+[NOTE]
+[role="docsvg-channel-subbuf-anim"]
+====
+{note-no-anim}
+====
+
+In an ideal world, sub-buffers are consumed faster than filled, like it
+is the case above. In the real world, however, all sub-buffers could be
+full at some point, leaving no space to record the following events. By
+design, LTTng is a _non-blocking_ tracer: when no empty sub-buffer
+exists, losing events is acceptable when the alternative would be to
+cause substantial delays in the instrumented application's execution.
+LTTng privileges performance over integrity, aiming at perturbing the
+traced system as little as possible in order to make tracing of subtle
+race conditions and rare interrupt cascades possible.
+
+When it comes to losing events because no empty sub-buffer is available,
+the channel's _event loss mode_ determines what to do amongst:
+
+Discard::
+ Drop the newest events until a sub-buffer is released.
+
+Overwrite::
+ Clear the sub-buffer containing the oldest recorded
+ events and start recording the newest events there. This mode is
+ sometimes called _flight recorder mode_ because it behaves like a
+ flight recorder: always keep a fixed amount of the latest data.
+
+Which mechanism you should choose depends on your context: prioritize
+the newest or the oldest events in the ring buffer?
+
+Beware that, in overwrite mode, a whole sub-buffer is abandoned as soon
+as a new event doesn't find an empty sub-buffer, whereas in discard
+mode, only the event that doesn't fit is discarded.
+
+Also note that a count of lost events is incremented and saved in
+the trace itself when an event is lost in discard mode, whereas no
+information is kept when a sub-buffer gets overwritten before being
+committed.
+
+There are known ways to decrease your probability of losing events. The
+next section shows how tuning the sub-buffers count and size can be
+used to virtually stop losing events.
+
+
+[[channel-subbuf-size-vs-subbuf-count]]
+===== Sub-buffers count and size
+
+For each channel, an LTTng user may set its number of sub-buffers and
+their size.
+
+Note that there is a noticeable tracer's CPU overhead introduced when
+switching sub-buffers (marking a full one as consumable and switching
+to an empty one for the following events to be recorded). Knowing this,
+the following list presents a few practical situations along with how
+to configure sub-buffers for them:
+
+High event throughput::
+ In general, prefer bigger sub-buffers to
+ lower the risk of losing events. Having bigger sub-buffers
+ also ensures a lower sub-buffer switching frequency. The number of
+ sub-buffers is only meaningful if the channel is enabled in
+ overwrite mode: in this case, if a sub-buffer overwrite happens, the
+ other sub-buffers are left unaltered.
+
+Low event throughput::
+ In general, prefer smaller sub-buffers
+ since the risk of losing events is already low. Since events
+ happen less frequently, the sub-buffer switching frequency should
+ remain low and thus the tracer's overhead should not be a problem.
+
+Low memory system::
+ If your target system has a low memory
+ limit, prefer fewer first, then smaller sub-buffers. Even if the
+ system is limited in memory, you want to keep the sub-buffers as
+ big as possible to avoid a high sub-buffer switching frequency.
+
+You should know that LTTng uses CTF as its trace format, which means
+event data is very compact. For example, the average LTTng Linux kernel
+event weights about 32{nbsp}bytes. A sub-buffer size of 1{nbsp}MiB is
+thus considered big.
+
+The previous situations highlight the major trade-off between a few big
+sub-buffers and more, smaller sub-buffers: sub-buffer switching
+frequency vs. how much data is lost in overwrite mode. Assuming a
+constant event throughput and using the overwrite mode, the two
+following configurations have the same ring buffer total size:
+
+[NOTE]
+[role="docsvg-channel-subbuf-size-vs-count-anim"]
+====
+{note-no-anim}
+====
+
+* **2 sub-buffers of 4 MiB each** lead to a very low sub-buffer
+ switching frequency, but if a sub-buffer overwrite happens, half of
+ the recorded events so far (4{nbsp}MiB) are definitely lost.
+* **8 sub-buffers of 1 MiB each** lead to 4{nbsp}times the tracer's
+ overhead as the previous configuration, but if a sub-buffer
+ overwrite happens, only the eighth of events recorded so far are
+ definitely lost.
+
+In discard mode, the sub-buffers count parameter is pointless: use two
+sub-buffers and set their size according to the requirements of your
+situation.
+
+
+[[channel-switch-timer]]
+===== Switch timer
+
+The _switch timer_ period is another important configurable feature of
+channels to ensure periodic sub-buffer flushing.
+
+When the _switch timer_ fires, a sub-buffer switch happens. This timer
+may be used to ensure that event data is consumed and committed to
+trace files periodically in case of a low event throughput:
+
+[NOTE]
+[role="docsvg-channel-switch-timer"]
+====
+{note-no-anim}
+====
+
+It's also convenient when big sub-buffers are used to cope with
+sporadic high event throughput, even if the throughput is normally
+lower.
+
+
+[[channel-buffering-schemes]]
+===== Buffering schemes
+
+In the user space tracing domain, two **buffering schemes** are
+available when creating a channel:
+
+Per-PID buffering::
+ Keep one ring buffer per process.
+
+Per-UID buffering::
+ Keep one ring buffer for all processes of a single user.
+
+The per-PID buffering scheme consumes more memory than the per-UID
+option if more than one process is instrumented for LTTng-UST. However,
+per-PID buffering ensures that one process having a high event
+throughput won't fill all the shared sub-buffers, only its own.
+
+The Linux kernel tracing domain only has one available buffering scheme
+which is to use a single ring buffer for the whole system.
+
+
+[[event]]
+==== Event
+
+An _event_, in LTTng's realm, is a term often used metonymically,
+having multiple definitions depending on the context:
+
+. When tracing, an event is a _point in space-time_. Space, in a
+ tracing context, is the set of all executable positions of a
+ compiled application by a logical processor. When a program is
+ executed by a processor and some instrumentation point, or
+ _probe_, is encountered, an event occurs. This event is accompanied
+ by some contextual payload (values of specific variables at this
+ point of execution) which may or may not be recorded.
+. In the context of a recorded trace file, the term _event_ implies
+ a _recorded event_.
+. When configuring a tracing session, _enabled events_ refer to
+ specific rules which could lead to the transfer of actual
+ occurring events (1) to recorded events (2).
+
+The whole <<core-concepts,Core concepts>> section focuses on the
+third definition. An event is always registered to _one or more_
+channels and may be enabled or disabled at will per channel. A disabled
+event never leads to a recorded event, even if its channel is enabled.
+
+An event (3) is enabled with a few conditions that must _all_ be met
+when an event (1) happens in order to generate a recorded event (2):
+
+. A _probe_ or group of probes in the traced application must be
+ executed.
+. **Optionally**, the probe must have a log level matching a
+ log level range specified when enabling the event.
+. **Optionally**, the occurring event must satisfy a custom
+ expression, or _filter_, specified when enabling the event.
+
+
+[[plumbing]]
+=== Plumbing
+
+The previous section described the concepts at the heart of LTTng.
+This section summarizes LTTng's implementation: how those objects are
+managed by different applications and libraries working together to
+form the toolkit.
+
+
+[[plumbing-overview]]
+==== Overview
+
+As <<installing-lttng,mentioned previously>>, the whole LTTng suite
+is made of the LTTng-tools, LTTng-UST, and
+LTTng-modules packages. Together, they provide different daemons, libraries,
+kernel modules and command line interfaces. The following tree shows
+which usable component belongs to which package:
+
+* **LTTng-tools**:
+** session daemon (`lttng-sessiond`)
+** consumer daemon (`lttng-consumerd`)
+** relay daemon (`lttng-relayd`)
+** tracing control library (`liblttng-ctl`)
+** tracing control command line tool (`lttng`)
+* **LTTng-UST**:
+** user space tracing library (`liblttng-ust`) and its headers
+** preloadable user space tracing helpers
+ (`liblttng-ust-libc-wrapper`, `liblttng-ust-pthread-wrapper`,
+ `liblttng-ust-cyg-profile`, `liblttng-ust-cyg-profile-fast`
+ and `liblttng-ust-dl`)
+** user space tracepoint code generator command line tool
+ (`lttng-gen-tp`)
+** `java.util.logging`/log4j tracepoint providers
+ (`liblttng-ust-jul-jni` and `liblttng-ust-log4j-jni`) and JAR
+ file (path:{liblttng-ust-agent.jar})
+* **LTTng-modules**:
+** LTTng Linux kernel tracer module
+** tracing ring buffer kernel modules
+** many LTTng probe kernel modules
+
+The following diagram shows how the most important LTTng components
+interact. Plain purple arrows represent trace data paths while dashed
+red arrows indicate control communications. The LTTng relay daemon is
+shown running on a remote system, although it could as well run on the
+target (monitored) system.
+
+[role="img-100"]
+.Control and data paths between LTTng components.
+image::plumbing-26.png[]
+
+Each component is described in the following subsections.
+
+
+[[lttng-sessiond]]
+==== Session daemon
+
+At the heart of LTTng's plumbing is the _session daemon_, often called
+by its command name, `lttng-sessiond`.
+
+The session daemon is responsible for managing tracing sessions and
+what they logically contain (channel properties, enabled/disabled
+events, and the rest). By communicating locally with instrumented
+applications (using LTTng-UST) and with the LTTng Linux kernel modules
+(LTTng-modules), it oversees all tracing activities.
+
+One of the many things that `lttng-sessiond` does is to keep
+track of the available event types. User space applications and
+libraries actively connect and register to the session daemon when they
+start. By contrast, `lttng-sessiond` seeks out and loads the appropriate
+LTTng kernel modules as part of its own initialization. Kernel event
+types are _pulled_ by `lttng-sessiond`, whereas user space event types
+are _pushed_ to it by the various user space tracepoint providers.
+
+Using a specific inter-process communication protocol with Linux kernel
+and user space tracers, the session daemon can send channel information
+so that they are initialized, enable/disable specific probes based on
+enabled/disabled events by the user, send event filters information to
+LTTng tracers so that filtering actually happens at the tracer site,
+start/stop tracing a specific application or the Linux kernel, and more.
+
+The session daemon is not useful without some user controlling it,
+because it's only a sophisticated control interchange and thus
+doesn't make any decision on its own. `lttng-sessiond` opens a local
+socket for controlling it, albeit the preferred way to control it is
+using `liblttng-ctl`, an installed C library hiding the communication
+protocol behind an easy-to-use API. The `lttng` tool makes use of
+`liblttng-ctl` to implement a user-friendly command line interface.
+
+`lttng-sessiond` does not receive any trace data from instrumented
+applications; the _consumer daemons_ are the programs responsible for
+collecting trace data using shared ring buffers. However, the session
+daemon is the one that must spawn a consumer daemon and establish
+a control communication with it.
+
+Session daemons run on a per-user basis. Knowing this, multiple
+instances of `lttng-sessiond` may run simultaneously, each belonging
+to a different user and each operating independently of the others.
+Only `root`'s session daemon, however, may control LTTng kernel modules
+(that is, the kernel tracer). With that in mind, if a user has no root
+access on the target system, he cannot trace the system's kernel, but
+should still be able to trace its own instrumented applications.
+
+It has to be noted that, although only `root`'s session daemon may
+control the kernel tracer, the `lttng-sessiond` command has a `--group`
+option which may be used to specify the name of a special user group
+allowed to communicate with `root`'s session daemon and thus record
+kernel traces. By default, this group is named `tracing`.
+
+If not done yet, the `lttng` tool, by default, automatically starts a
+session daemon. `lttng-sessiond` may also be started manually:
+
+[role="term"]
+----
+lttng-sessiond
+----
+
+This starts the session daemon in foreground. Use
+
+[role="term"]
+----
+lttng-sessiond --daemonize
+----
+
+to start it as a true daemon.
+
+To kill the current user's session daemon, `pkill` may be used:
+
+[role="term"]
+----
+pkill lttng-sessiond
+----
+
+The default `SIGTERM` signal terminates it cleanly.
+
+Several other options are available and described in
+man:lttng-sessiond(8) or by running `lttng-sessiond --help`.
+
+
+[[lttng-consumerd]]
+==== Consumer daemon
+
+The _consumer daemon_, or `lttng-consumerd`, is a program sharing some
+ring buffers with user applications or the LTTng kernel modules to
+collect trace data and output it at some place (on disk or sent over
+the network to an LTTng relay daemon).
+
+Consumer daemons are created by a session daemon as soon as events are
+enabled within a tracing session, well before tracing is activated
+for the latter. Entirely managed by session daemons,
+consumer daemons survive session destruction to be reused later,
+should a new tracing session be created. Consumer daemons are always
+owned by the same user as their session daemon. When its owner session
+daemon is killed, the consumer daemon also exits. This is because
+the consumer daemon is always the child process of a session daemon.
+Consumer daemons should never be started manually. For this reason,
+they are not installed in one of the usual locations listed in the
+`PATH` environment variable. `lttng-sessiond` has, however, a
+bunch of options (see man:lttng-sessiond(8)) to
+specify custom consumer daemon paths if, for some reason, a consumer
+daemon other than the default installed one is needed.
+
+There are up to two running consumer daemons per user, whereas only one
+session daemon may run per user. This is because each process has
+independent bitness: if the target system runs a mixture of 32-bit and
+64-bit processes, it is more efficient to have separate corresponding
+32-bit and 64-bit consumer daemons. The `root` user is an exception: it
+may have up to _three_ running consumer daemons: 32-bit and 64-bit
+instances for its user space applications and one more reserved for
+collecting kernel trace data.
+
+As new tracing domains are added to LTTng, the development community's
+intent is to minimize the need for additionnal consumer daemon instances
+dedicated to them. For instance, the `java.util.logging` (JUL) domain
+events are in fact mapped to the user space domain, thus tracing this
+particular domain is handled by existing user space domain consumer
+daemons.
+
+
+[[lttng-relayd]]
+==== Relay daemon
+
+When a tracing session is configured to send its trace data over the
+network, an LTTng _relay daemon_ must be used at the other end to
+receive trace packets and serialize them to trace files. This setup
+makes it possible to trace a target system without ever committing trace
+data to its local storage, a feature which is useful for embedded
+systems, amongst others. The command implementing the relay daemon
+is `lttng-relayd`.
+
+The basic use case of `lttng-relayd` is to transfer trace data received
+over the network to trace files on the local file system. The relay
+daemon must listen on two TCP ports to achieve this: one control port,
+used by the target session daemon, and one data port, used by the
+target consumer daemon. The relay and session daemons agree on common
+default ports when custom ones are not specified.
+
+Since the communication transport protocol for both ports is standard
+TCP, the relay daemon may be started either remotely or locally (on the
+target system).
+
+While two instances of consumer daemons (32-bit and 64-bit) may run
+concurrently for a given user, `lttng-relayd` needs only be of its
+host operating system's bitness.
+
+The other important feature of LTTng's relay daemon is the support of
+_LTTng live_. LTTng live is an application protocol to view events as
+they arrive. The relay daemon still records events in trace files,
+but a _tee_ allows to inspect incoming events.
+
+[role="img-100"]
+.The relay daemon creates a _tee_, forwarding the trace data to both trace files and a live viewer.
+image::lttng-live.png[]
+
+Using LTTng live locally thus requires to run a local relay daemon.
+
+
+[[liblttng-ctl-lttng]]
+==== [[lttng-cli]]Control library and command line interface
+
+The LTTng control library, `liblttng-ctl`, can be used to communicate
+with the session daemon using a C API that hides the underlying
+protocol's details. `liblttng-ctl` is part of LTTng-tools.
+
+`liblttng-ctl` may be used by including its "master" header:
+
+[source,c]
+----
+#include <lttng/lttng.h>
+----
+
+Some objects are referred by name (C string), such as tracing sessions,
+but most of them require creating a handle first using
+`lttng_create_handle()`. The best available developer documentation for
+`liblttng-ctl` is, for the moment, its installed header files as such.
+Every function/structure is thoroughly documented.
+
+The `lttng` program is the _de facto_ standard user interface to
+control LTTng tracing sessions. `lttng` uses `liblttng-ctl` to
+communicate with session daemons behind the scenes.
+Its man page, man:lttng(1), is exhaustive, as well as its command
+line help (+lttng _cmd_ --help+, where +_cmd_+ is the command name).
+
+The <<controlling-tracing,Controlling tracing>> section is a feature
+tour of the `lttng` tool.
+
+
+[[lttng-ust]]
+==== User space tracing library
+
+The user space tracing part of LTTng is possible thanks to the user
+space tracing library, `liblttng-ust`, which is part of the LTTng-UST
+package.
+
+`liblttng-ust` provides header files containing macros used to define
+tracepoints and create tracepoint providers, as well as a shared object
+that must be linked to individual applications to connect to and
+communicate with a session daemon and a consumer daemon as soon as the
+application starts.
+
+The exact mechanism by which an application is registered to the
+session daemon is beyond the scope of this documentation. The only thing
+you need to know is that, since the library constructor does this job
+automatically, tracepoints may be safely inserted anywhere in the source
+code without prior manual initialization of `liblttng-ust`.
+
+The `liblttng-ust`-session daemon collaboration also provides an
+interesting feature: user space events may be enabled _before_
+applications actually start. By doing this and starting tracing before
+launching the instrumented application, you make sure that even the
+earliest occurring events can be recorded.
+
+The <<c-application,C application>> instrumenting guide of the
+<<using-lttng,Using LTTng>> chapter focuses on using `liblttng-ust`:
+instrumenting, building/linking and running a user application.
+
+
+[[lttng-modules]]
+==== LTTng kernel modules
+
+The LTTng Linux kernel modules provide everything needed to trace the
+Linux kernel: various probes, a ring buffer implementation for a
+consumer daemon to read trace data and the tracer itself.
+
+Only in exceptional circumstances should you ever need to load the
+LTTng kernel modules manually: it is normally the responsability of
+`root`'s session daemon to do so. Even if you were to develop your
+own LTTng probe module--for tracing a custom kernel or some kernel
+module (this topic is covered in the
+<<instrumenting-linux-kernel,Linux kernel>> instrumenting guide of
+the <<using-lttng,Using LTTng>> chapter)—you
+should use the `--extra-kmod-probes` option of the session daemon to
+append your probe to the default list. The session and consumer daemons
+of regular users do not interact with the LTTng kernel modules at all.
+
+LTTng kernel modules are installed, by default, in
++/usr/lib/modules/_release_/extra+, where +_release_+ is the
+kernel release (see `uname --kernel-release`).
+
+
+[[using-lttng]]
+== Using LTTng
+
+Using LTTng involves two main activities: **instrumenting** and
+**controlling tracing**.
+
+_<<instrumenting,Instrumenting>>_ is the process of inserting probes
+into some source code. It can be done manually, by writing tracepoint
+calls at specific locations in the source code of the program to trace,
+or more automatically using dynamic probes (address in assembled code,
+symbol name, function entry/return, and others).
+
+It has to be noted that, as an LTTng user, you may not have to worry
+about the instrumentation process. Indeed, you may want to trace a
+program already instrumented. As an example, the Linux kernel is
+thoroughly instrumented, which is why you can trace it without caring
+about adding probes.
+
+_<<controlling-tracing,Controlling tracing>>_ is everything
+that can be done by the LTTng session daemon, which is controlled using
+`liblttng-ctl` or its command line utility, `lttng`: creating tracing
+sessions, listing tracing sessions and events, enabling/disabling
+events, starting/stopping the tracers, taking snapshots, amongst many
+other commands.
+
+This chapter is a complete user guide of both activities,
+with common use cases of LTTng exposed throughout the text. It is
+assumed that you are familiar with LTTng's concepts (events, channels,
+domains, tracing sessions) and that you understand the roles of its
+components (daemons, libraries, command line tools); if not, we invite
+you to read the <<understanding-lttng,Understanding LTTng>> chapter
+before you begin reading this one.
+
+If you're new to LTTng, we suggest that you rather start with the
+<<getting-started,Getting started>> small guide first, then come
+back here to broaden your knowledge.
+
+If you're only interested in tracing the Linux kernel with its current
+instrumentation, you may skip the
+<<instrumenting,Instrumenting>> section.
+
+
+[[instrumenting]]
+=== Instrumenting
+
+There are many examples of tracing and monitoring in our everyday life.
+You have access to real-time and historical weather reports and forecasts
+thanks to weather stations installed around the country. You know your
+possibly hospitalized friends' and family's hearts are safe thanks to
+electrocardiography. You make sure not to drive your car too fast
+and have enough fuel to reach your destination thanks to gauges visible
+on your dashboard.
+
+All the previous examples have something in common: they rely on
+**probes**. Without electrodes attached to the surface of a body's
+skin, cardiac monitoring would be futile.
+
+LTTng, as a tracer, is no different from the real life examples above.
+If you're about to trace a software system or, put in other words, record its
+history of execution, you better have probes in the subject you're
+tracing: the actual software. Various ways were developed to do this.
+The most straightforward one is to manually place probes, called
+_tracepoints_, in the software's source code. The Linux kernel tracing
+domain also allows probes added dynamically.
+
+If you're only interested in tracing the Linux kernel, it may very well
+be that your tracing needs are already appropriately covered by LTTng's
+built-in Linux kernel tracepoints and other probes. Or you may be in
+possession of a user space application which has already been
+instrumented. In such cases, the work resides entirely in the design
+and execution of tracing sessions, allowing you to jump to
+<<controlling-tracing,Controlling tracing>> right now.
+
+This chapter focuses on the following use cases of instrumentation:
+
+* <<c-application,C>> and <<cxx-application,$$C++$$>> applications
+* <<prebuilt-ust-helpers,prebuilt user space tracing helpers>>
+* <<java-application,Java application>>
+* <<instrumenting-linux-kernel,Linux kernel>> module or the
+ kernel itself
+* the <<proc-lttng-logger-abi,path:{/proc/lttng-logger} ABI>>
+
+Some advanced techniques are also presented at the very end of this
+chapter.
+
+
+[[c-application]]
+==== C application
+
+Instrumenting a C (or $$C++$$) application, be it an executable program
+or a library, implies using LTTng-UST, the
+user space tracing component of LTTng. For C/$$C++$$ applications, the
+LTTng-UST package includes a dynamically loaded library
+(`liblttng-ust`), C headers and the `lttng-gen-tp` command line utility.
+
+Since C and $$C++$$ are the base languages of virtually all other
+programming languages
+(Java virtual machine, Python, Perl, PHP and Node.js interpreters, to
+name a few), implementing user space tracing for an unsupported language
+is just a matter of using the LTTng-UST C API at the right places.
+
+The usual work flow to instrument a user space C application with
+LTTng-UST is:
+
+. Define tracepoints (actual probes)
+. Write tracepoint providers
+. Insert tracepoints into target source code
+. Package (build) tracepoint providers
+. Build user application and link it with tracepoint providers
+
+The steps above are discussed in greater detail in the following
+subsections.
+
+
+[[tracepoint-provider]]
+===== Tracepoint provider
+
+Before jumping into defining tracepoints and inserting
+them into the application source code, you must understand what a
+_tracepoint provider_ is.
+
+For the sake of this guide, consider the following two files:
+
+[source,c]
+.path:{tp.h}
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER my_provider
+
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./tp.h"
+
+#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _TP_H
+
+#include <lttng/tracepoint.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_first_tracepoint,
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_other_tracepoint,
+ TP_ARGS(
+ int, my_int
+ ),
+ TP_FIELDS(
+ ctf_integer(int, some_field, my_int)
+ )
+)
+
+#endif /* _TP_H */
+
+#include <lttng/tracepoint-event.h>
+----
+
+[source,c]
+.path:{tp.c}
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+The two files above are defining a _tracepoint provider_. A tracepoint
+provider is some sort of namespace for _tracepoint definitions_. Tracepoint
+definitions are written above with the `TRACEPOINT_EVENT()` macro, and allow
+eventual `tracepoint()` calls respecting their definitions to be inserted
+into the user application's C source code (we explore this in a
+later section).
+
+Many tracepoint definitions may be part of the same tracepoint provider
+and many tracepoint providers may coexist in a user space application. A
+tracepoint provider is packaged either:
+
+* directly into an existing user application's C source file
+* as an object file
+* as a static library
+* as a shared library
+
+The two files above, path:{tp.h} and path:{tp.c}, show a typical template for
+writing a tracepoint provider. LTTng-UST was designed so that two
+tracepoint providers should not be defined in the same header file.
+
+We will now go through the various parts of the above files and
+give them a meaning. As you may have noticed, the LTTng-UST API for
+C/$$C++$$ applications is some preprocessor sorcery. The LTTng-UST macros
+used in your application and those in the LTTng-UST headers are
+combined to produce actual source code needed to make tracing possible
+using LTTng.
+
+Let's start with the header file, path:{tp.h}. It begins with
+
+[source,c]
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER my_provider
+----
+
+`TRACEPOINT_PROVIDER` defines the name of the provider to which the
+following tracepoint definitions belong. It is used internally by
+LTTng-UST headers and _must_ be defined. Since `TRACEPOINT_PROVIDER`
+could have been defined by another header file also included by the same
+C source file, the best practice is to undefine it first.
+
+NOTE: Names in LTTng-UST follow the C
+_identifier_ syntax (starting with a letter and containing either
+letters, numbers or underscores); they are _not_ C strings
+(not surrounded by double quotes). This is because LTTng-UST macros
+use those identifier-like strings to create symbols (named types and
+variables).
+
+The tracepoint provider is a group of tracepoint definitions; its chosen
+name should reflect this. A hierarchy like Java packages is recommended,
+using underscores instead of dots, for example,
+`org_company_project_component`.
+
+Next is `TRACEPOINT_INCLUDE`:
+
+[source,c]
+----
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./tp.h"
+----
+
+This little bit of instrospection is needed by LTTng-UST to include
+your header at various predefined places.
+
+Include guard follows:
+
+[source,c]
+----
+#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _TP_H
+----
+
+Add these precompiler conditionals to ensure the tracepoint event
+generation can include this file more than once.
+
+The `TRACEPOINT_EVENT()` macro is defined in a LTTng-UST header file which
+must be included:
+
+[source,c]
+----
+#include <lttng/tracepoint.h>
+----
+
+This also allows the application to use the `tracepoint()` macro.
+
+Next is a list of `TRACEPOINT_EVENT()` macro calls which create the
+actual tracepoint definitions. We skip this for the moment and
+come back to how to use `TRACEPOINT_EVENT()`
+<<defining-tracepoints,in a later section>>. Just pay attention to
+the first argument: it's always the name of the tracepoint provider
+being defined in this header file.
+
+End of include guard:
+
+[source,c]
+----
+#endif /* _TP_H */
+----
+
+Finally, include `<lttng/tracepoint-event.h>` to expand the macros:
+
+[source,c]
+----
+#include <lttng/tracepoint-event.h>
+----
+
+That's it for path:{tp.h}. Of course, this is only a header file; it must be
+included in some C source file to actually use it. This is the job of
+path:{tp.c}:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+When `TRACEPOINT_CREATE_PROBES` is defined, the macros used in path:{tp.h},
+which is included just after, actually create the source code for
+LTTng-UST probes (global data structures and functions) out of your
+tracepoint definitions. How exactly this is done is out of this text's scope.
+`TRACEPOINT_CREATE_PROBES` is discussed further
+in
+<<building-tracepoint-providers-and-user-application,Building/linking
+tracepoint providers and the user application>>.
+
+You could include other header files like path:{tp.h} here to create the probes
+of different tracepoint providers, for example:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp1.h"
+#include "tp2.h"
+----
+
+The rule is: probes of a given tracepoint provider
+must be created in exactly one source file. This source file could be one
+of your project's; it doesn't have to be on its own like
+path:{tp.c}, although
+<<building-tracepoint-providers-and-user-application,a later section>>
+shows that doing so allows packaging the tracepoint providers
+independently and keep them out of your application, also making it
+possible to reuse them between projects.
+
+The following sections explain how to define tracepoints, how to use the
+`tracepoint()` macro to instrument your user space C application and how
+to build/link tracepoint providers and your application with LTTng-UST
+support.
+
+
+[[lttng-gen-tp]]
+===== Using `lttng-gen-tp`
+
+LTTng-UST ships with `lttng-gen-tp`, a handy command line utility for
+generating most of the stuff discussed above. It takes a _template file_,
+with a name usually ending with the `.tp` extension, containing only
+tracepoint definitions, and outputs a tracepoint provider (either a C
+source file or a precompiled object file) with its header file.
+
+`lttng-gen-tp` should suffice in <<static-linking,static linking>>
+situations. When using it, write a template file containing a list of
+`TRACEPOINT_EVENT()` macro calls. The tool finds the provider names
+used and generate the appropriate files which are going to look a lot
+like path:{tp.h} and path:{tp.c} above.
+
+Just call `lttng-gen-tp` like this:
+
+[role="term"]
+----
+lttng-gen-tp my-template.tp
+----
+
+path:{my-template.c}, path:{my-template.o} and path:{my-template.h}
+are created in the same directory.
+
+You may specify custom C flags passed to the compiler invoked by
+`lttng-gen-tp` using the `CFLAGS` environment variable:
+
+[role="term"]
+----
+CFLAGS=-I/custom/include/path lttng-gen-tp my-template.tp
+----
+
+For more information on `lttng-gen-tp`, see man:lttng-gen-tp(1).
+
+
+[[defining-tracepoints]]
+===== Defining tracepoints
+
+As written in <<tracepoint-provider,Tracepoint provider>>,
+tracepoints are defined using the
+`TRACEPOINT_EVENT()` macro. Each tracepoint, when called using the
+`tracepoint()` macro in the actual application's source code, generates
+a specific event type with its own fields.
+
+Let's have another look at the example above, with a few added comments:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ /* tracepoint provider name */
+ my_provider,
+
+ /* tracepoint/event name */
+ my_first_tracepoint,
+
+ /* list of tracepoint arguments */
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+
+ /* list of fields of eventual event */
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+----
+
+The tracepoint provider name must match the name of the tracepoint
+provider in which this tracepoint is defined
+(see <<tracepoint-provider,Tracepoint provider>>). In other words,
+always use the same string as the value of `TRACEPOINT_PROVIDER` above.
+
+The tracepoint name becomes the event name once events are recorded
+by the LTTng-UST tracer. It must follow the tracepoint provider name
+syntax: start with a letter and contain either letters, numbers or
+underscores. Two tracepoints under the same provider cannot have the
+same name. In other words, you cannot overload a tracepoint like you
+would overload functions and methods in $$C++$$/Java.
+
+NOTE: The concatenation of the tracepoint
+provider name and the tracepoint name cannot exceed 254 characters. If
+it does, the instrumented application compiles and runs, but LTTng
+issues multiple warnings and you could experience serious problems.
+
+The list of tracepoint arguments gives this tracepoint its signature:
+see it like the declaration of a C function. The format of `TP_ARGS()`
+arguments is: C type, then argument name; repeat as needed, up to ten
+times. For example, if we were to replicate the signature of C standard
+library's `fseek()`, the `TP_ARGS()` part would look like:
+
+[source,c]
+----
+ TP_ARGS(
+ FILE*, stream,
+ long int, offset,
+ int, origin
+ ),
+----
+
+Of course, you need to include appropriate header files before
+the `TRACEPOINT_EVENT()` macro calls if any argument has a complex type.
+
+`TP_ARGS()` may not be omitted, but may be empty. `TP_ARGS(void)` is
+also accepted.
+
+The list of fields is where the fun really begins. The fields defined
+in this list are the fields of the events generated by the execution
+of this tracepoint. Each tracepoint field definition has a C
+_argument expression_ which is evaluated when the execution reaches
+the tracepoint. Tracepoint arguments _may be_ used freely in those
+argument expressions, but they _don't_ have to.
+
+There are several types of tracepoint fields available. The macros to
+define them are given and explained in the
+<<liblttng-ust-tp-fields,LTTng-UST library reference>> section.
+
+Field names must follow the standard C identifier syntax: letter, then
+optional sequence of letters, numbers or underscores. Each field must have
+a different name.
+
+Those `ctf_*()` macros are added to the `TP_FIELDS()` part of
+`TRACEPOINT_EVENT()`. Note that they are not delimited by commas.
+`TP_FIELDS()` may be empty, but the `TP_FIELDS(void)` form is _not_
+accepted.
+
+The following snippet shows how argument expressions may be used in
+tracepoint fields and how they may refer freely to tracepoint arguments.
+
+[source,c]
+----
+/* for struct stat */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, my_int_arg,
+ char*, my_str_arg,
+ struct stat*, st
+ ),
+ TP_FIELDS(
+ /* simple integer field with constant value */
+ ctf_integer(
+ int, /* field C type */
+ my_constant_field, /* field name */
+ 23 + 17 /* argument expression */
+ )
+
+ /* my_int_arg tracepoint argument */
+ ctf_integer(
+ int,
+ my_int_arg_field,
+ my_int_arg
+ )
+
+ /* my_int_arg squared */
+ ctf_integer(
+ int,
+ my_int_arg_field2,
+ my_int_arg * my_int_arg
+ )
+
+ /* sum of first 4 characters of my_str_arg */
+ ctf_integer(
+ int,
+ sum4,
+ my_str_arg[0] + my_str_arg[1] +
+ my_str_arg[2] + my_str_arg[3]
+ )
+
+ /* my_str_arg as string field */
+ ctf_string(
+ my_str_arg_field, /* field name */
+ my_str_arg /* argument expression */
+ )
+
+ /* st_size member of st tracepoint argument, hexadecimal */
+ ctf_integer_hex(
+ off_t, /* field C type */
+ size_field, /* field name */
+ st->st_size /* argument expression */
+ )
+
+ /* st_size member of st tracepoint argument, as double */
+ ctf_float(
+ double, /* field C type */
+ size_dbl_field, /* field name */
+ (double) st->st_size /* argument expression */
+ )
+
+ /* half of my_str_arg string as text sequence */
+ ctf_sequence_text(
+ char, /* element C type */
+ half_my_str_arg_field, /* field name */
+ my_str_arg, /* argument expression */
+ size_t, /* length expression C type */
+ strlen(my_str_arg) / 2 /* length expression */
+ )
+ )
+)
+----
+
+As you can see, having a custom argument expression for each field
+makes tracepoints very flexible for tracing a user space C application.
+This tracepoint definition is reused later in this guide, when
+actually using tracepoints in a user space application.
+
+
+[[using-tracepoint-classes]]
+===== Using tracepoint classes
+
+In LTTng-UST, a _tracepoint class_ is a class of tracepoints sharing the
+same field types and names. A _tracepoint instance_ is one instance of
+such a declared tracepoint class, with its own event name and tracepoint
+provider name.
+
+What is documented in <<defining-tracepoints,Defining tracepoints>>
+is actually how to declare a _tracepoint class_ and define a
+_tracepoint instance_ at the same time. Without revealing the internals
+of LTTng-UST too much, it has to be noted that one serialization
+function is created for each tracepoint class. A serialization
+function is responsible for serializing the fields of a tracepoint
+into a sub-buffer when tracing. For various performance reasons, when
+your situation requires multiple tracepoints with different names, but
+with the same fields layout, the best practice is to manually create
+a tracepoint class and instantiate as many tracepoint instances as
+needed. One positive effect of such a design, amongst other advantages,
+is that all tracepoint instances of the same tracepoint class
+reuse the same serialization function, thus reducing cache pollution.
+
+As an example, here are three tracepoint definitions as we know them:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ my_app,
+ get_account,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+----
+
+In this case, three tracepoint classes are created, with one tracepoint
+instance for each of them: `get_account`, `get_settings` and
+`get_transaction`. However, they all share the same field names and
+types. Declaring one tracepoint class and three tracepoint instances of
+the latter is a better design choice:
+
+[source,c]
+----
+/* the tracepoint class */
+TRACEPOINT_EVENT_CLASS(
+ /* tracepoint provider name */
+ my_app,
+
+ /* tracepoint class name */
+ my_class,
+
+ /* arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+
+ /* fields */
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+/* the tracepoint instances */
+TRACEPOINT_EVENT_INSTANCE(
+ /* tracepoint provider name */
+ my_app,
+
+ /* tracepoint class name */
+ my_class,
+
+ /* tracepoint/event name */
+ get_account,
+
+ /* arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+----
+
+Of course, all those names and `TP_ARGS()` invocations are redundant,
+but some C preprocessor magic can solve this:
+
+[source,c]
+----
+#define MY_TRACEPOINT_ARGS \
+ TP_ARGS( \
+ int, userid, \
+ size_t, len \
+ )
+
+TRACEPOINT_EVENT_CLASS(
+ my_app,
+ my_class,
+ MY_TRACEPOINT_ARGS,
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+#define MY_APP_TRACEPOINT_INSTANCE(name) \
+ TRACEPOINT_EVENT_INSTANCE( \
+ my_app, \
+ my_class, \
+ name, \
+ MY_TRACEPOINT_ARGS \
+ )
+
+MY_APP_TRACEPOINT_INSTANCE(get_account)
+MY_APP_TRACEPOINT_INSTANCE(get_settings)
+MY_APP_TRACEPOINT_INSTANCE(get_transaction)
+----
+
+
+[[assigning-log-levels]]
+===== Assigning log levels to tracepoints
+
+Optionally, a log level can be assigned to a defined tracepoint.
+Assigning different levels of importance to tracepoints can be useful;
+when controlling tracing sessions,
+<<controlling-tracing,you can choose>> to only enable tracepoints
+falling into a specific log level range.
+
+Log levels are assigned to defined tracepoints using the
+`TRACEPOINT_LOGLEVEL()` macro. The latter must be used _after_ having
+used `TRACEPOINT_EVENT()` for a given tracepoint. The
+`TRACEPOINT_LOGLEVEL()` macro has the following construct:
+
+[source,c]
+----
+TRACEPOINT_LOGLEVEL(PROVIDER_NAME, TRACEPOINT_NAME, LOG_LEVEL)
+----
+
+where the first two arguments are the same as the first two arguments
+of `TRACEPOINT_EVENT()` and `LOG_LEVEL` is one
+of the values given in the
+<<liblttng-ust-tracepoint-loglevel,LTTng-UST library reference>>
+section.
+
+As an example, let's assign a `TRACE_DEBUG_UNIT` log level to our
+previous tracepoint definition:
+
+[source,c]
+----
+TRACEPOINT_LOGLEVEL(my_provider, my_tracepoint, TRACE_DEBUG_UNIT)
+----
+
+
+[[probing-the-application-source-code]]
+===== Probing the application's source code
+
+Once tracepoints are properly defined within a tracepoint provider,
+they may be inserted into the user application to be instrumented
+using the `tracepoint()` macro. Its first argument is the tracepoint
+provider name and its second is the tracepoint name. The next, optional
+arguments are defined by the `TP_ARGS()` part of the definition of
+the tracepoint to use.
+
+As an example, let us again take the following tracepoint definition:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ /* tracepoint provider name */
+ my_provider,
+
+ /* tracepoint/event name */
+ my_first_tracepoint,
+
+ /* list of tracepoint arguments */
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+
+ /* list of fields of eventual event */
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+----
+
+Assuming this is part of a file named path:{tp.h} which defines the tracepoint
+provider and which is included by path:{tp.c}, here's a complete C application
+calling this tracepoint (multiple times):
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+int main(int argc, char* argv[])
+{
+ int i;
+
+ tracepoint(my_provider, my_first_tracepoint, 23, "Hello, World!");
+
+ for (i = 0; i < argc; ++i) {
+ tracepoint(my_provider, my_first_tracepoint, i, argv[i]);
+ }
+
+ return 0;
+}
+----
+
+For each tracepoint provider, `TRACEPOINT_DEFINE` must be defined into
+exactly one translation unit (C source file) of the user application,
+before including the tracepoint provider header file. In other words,
+for a given tracepoint provider, you cannot define `TRACEPOINT_DEFINE`,
+and then include its header file in two separate C source files of
+the same application. `TRACEPOINT_DEFINE` is discussed further in
+<<building-tracepoint-providers-and-user-application,Building/linking
+tracepoint providers and the user application>>.
+
+As another example, remember this definition we wrote in a previous
+section (comments are stripped):
+
+[source,c]
+----
+/* for struct stat */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, my_int_arg,
+ char*, my_str_arg,
+ struct stat*, st
+ ),
+ TP_FIELDS(
+ ctf_integer(int, my_constant_field, 23 + 17)
+ ctf_integer(int, my_int_arg_field, my_int_arg)
+ ctf_integer(int, my_int_arg_field2, my_int_arg * my_int_arg)
+ ctf_integer(int, sum4_field, my_str_arg[0] + my_str_arg[1] +
+ my_str_arg[2] + my_str_arg[3])
+ ctf_string(my_str_arg_field, my_str_arg)
+ ctf_integer_hex(off_t, size_field, st->st_size)
+ ctf_float(double, size_dbl_field, (double) st->st_size)
+ ctf_sequence_text(char, half_my_str_arg_field, my_str_arg,
+ size_t, strlen(my_str_arg) / 2)
+ )
+)
+----
+
+Here's an example of calling it:
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+int main(void)
+{
+ struct stat s;
+
+ stat("/etc/fstab", &s);
+
+ tracepoint(my_provider, my_tracepoint, 23, "Hello, World!", &s);
+
+ return 0;
+}
+----
+
+When viewing the trace, assuming the file size of path:{/etc/fstab} is
+301{nbsp}bytes, the event generated by the execution of this tracepoint
+should have the following fields, in this order:
+
+----
+my_constant_field 40
+my_int_arg_field 23
+my_int_arg_field2 529
+sum4_field 389
+my_str_arg_field "Hello, World!"
+size_field 0x12d
+size_dbl_field 301.0
+half_my_str_arg_field "Hello,"
+----
+
+
+[[building-tracepoint-providers-and-user-application]]
+===== Building/linking tracepoint providers and the user application
+
+The final step of using LTTng-UST for tracing a user space C application
+(beside running the application) is building and linking tracepoint
+providers and the application itself.
+
+As discussed above, the macros used by the user-written tracepoint provider
+header file are useless until actually used to create probes code
+(global data structures and functions) in a translation unit (C source file).
+This is accomplished by defining `TRACEPOINT_CREATE_PROBES` in a translation
+unit and then including the tracepoint provider header file.
+When `TRACEPOINT_CREATE_PROBES` is defined, macros used and included by
+the tracepoint provider header produce actual source code needed by any
+application using the defined tracepoints. Defining
+`TRACEPOINT_CREATE_PROBES` produces code used when registering
+tracepoint providers when the tracepoint provider package loads.
+
+The other important definition is `TRACEPOINT_DEFINE`. This one creates
+global, per-tracepoint structures referencing the tracepoint providers
+data. Those structures are required by the actual functions inserted
+where `tracepoint()` macros are placed and need to be defined by the
+instrumented application.
+
+Both `TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` need to be defined
+at some places in order to trace a user space C application using LTTng.
+Although explaining their exact mechanism is beyond the scope of this
+document, the reason they both exist separately is to allow the trace
+providers to be packaged as a shared object (dynamically loaded library).
+
+There are two ways to compile and link the tracepoint providers
+with the application: _<<static-linking,statically>>_ or
+_<<dynamic-linking,dynamically>>_. Both methods are covered in the
+following subsections.
+
+
+[[static-linking]]
+===== Static linking the tracepoint providers to the application
+
+With the static linking method, compiled tracepoint providers are copied
+into the target application. There are three ways to do this:
+
+. Use one of your **existing C source files** to create probes.
+. Create probes in a separate C source file and build it as an
+ **object file** to be linked with the application (more decoupled).
+. Create probes in a separate C source file, build it as an
+ object file and archive it to create a **static library**
+ (more decoupled, more portable).
+
+The first approach is to define `TRACEPOINT_CREATE_PROBES` and include
+your tracepoint provider(s) header file(s) directly into an existing C
+source file. Here's an example:
+
+[source,c]
+----
+#include <stdlib.h>
+#include <stdio.h>
+/* ... */
+
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+/* ... */
+
+int my_func(int a, const char* b)
+{
+ /* ... */
+
+ tracepoint(my_provider, my_tracepoint, buf, sz, limit, &tt)
+
+ /* ... */
+}
+
+/* ... */
+----
+
+Again, before including a given tracepoint provider header file,
+`TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` must be defined in
+one, **and only one**, translation unit. Other C source files of the
+same application may include path:{tp.h} to use tracepoints with
+the `tracepoint()` macro, but must not define
+`TRACEPOINT_CREATE_PROBES`/`TRACEPOINT_DEFINE` again.
+
+This translation unit may be built as an object file by making sure to
+add `.` to the include path:
+
+[role="term"]
+----
+gcc -c -I. file.c
+----
+
+The second approach is to isolate the tracepoint provider code into a
+separate object file by using a dedicated C source file to create probes:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+`TRACEPOINT_DEFINE` must be defined by a translation unit of the
+application. Since we're talking about static linking here, it could as
+well be defined directly in the file above, before `#include "tp.h"`:
+
+[source,c]
+----
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+
+#include "tp.h"
+----
+
+This is actually what <<lttng-gen-tp,`lttng-gen-tp`>> does, and is
+the recommended practice.
+
+Build the tracepoint provider:
+
+[role="term"]
+----
+gcc -c -I. tp.c
+----
+
+Finally, the resulting object file may be archived to create a
+more portable tracepoint provider static library:
+
+[role="term"]
+----
+ar rc tp.a tp.o
+----
+
+Using a static library does have the advantage of centralising the
+tracepoint providers objects so they can be shared between multiple
+applications. This way, when the tracepoint provider is modified, the
+source code changes don't have to be patched into each application's source
+code tree. The applications need to be relinked after each change, but need
+not to be otherwise recompiled (unless the tracepoint provider's API
+changes).
+
+Regardless of which method you choose, you end up with an object file
+(potentially archived) containing the trace providers assembled code.
+To link this code with the rest of your application, you must also link
+with `liblttng-ust` and `libdl`:
+
+[role="term"]
+----
+gcc -o app tp.o other.o files.o of.o your.o app.o -llttng-ust -ldl
+----
+
+or
+
+[role="term"]
+----
+gcc -o app tp.a other.o files.o of.o your.o app.o -llttng-ust -ldl
+----
+
+If you're using a BSD
+system, replace `-ldl` with `-lc`:
+
+[role="term"]
+----
+gcc -o app tp.a other.o files.o of.o your.o app.o -llttng-ust -lc
+----
+
+The application can be started as usual, for example:
+
+[role="term"]
+----
+./app
+----
+
+The `lttng` command line tool can be used to
+<<controlling-tracing,control tracing>>.
+
+
+[[dynamic-linking]]
+===== Dynamic linking the tracepoint providers to the application
+
+The second approach to package the tracepoint providers is to use
+dynamic linking: the library and its member functions are explicitly
+sought, loaded and unloaded at runtime using `libdl`.
+
+It has to be noted that, for a variety of reasons, the created shared
+library is be dynamically _loaded_, as opposed to dynamically
+_linked_. The tracepoint provider shared object is, however, linked
+with `liblttng-ust`, so that `liblttng-ust` is guaranteed to be loaded
+as soon as the tracepoint provider is. If the tracepoint provider is
+not loaded, since the application itself is not linked with
+`liblttng-ust`, the latter is not loaded at all and the tracepoint calls
+become inert.
+
+The process to create the tracepoint provider shared object is pretty
+much the same as the static library method, except that:
+
+* since the tracepoint provider is not part of the application
+ anymore, `TRACEPOINT_DEFINE` _must_ be defined, for each tracepoint
+ provider, in exactly one translation unit (C source file) of the
+ _application_;
+* `TRACEPOINT_PROBE_DYNAMIC_LINKAGE` must be defined next to
+ `TRACEPOINT_DEFINE`.
+
+Regarding `TRACEPOINT_DEFINE` and `TRACEPOINT_PROBE_DYNAMIC_LINKAGE`,
+the recommended practice is to use a separate C source file in your
+application to define them, then include the tracepoint provider
+header files afterwards. For example:
+
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+
+/* include the header files of one or more tracepoint providers below */
+#include "tp1.h"
+#include "tp2.h"
+#include "tp3.h"
+----
+
+`TRACEPOINT_PROBE_DYNAMIC_LINKAGE` makes the macros included afterwards
+(by including the tracepoint provider header, which itself includes
+LTTng-UST headers) aware that the tracepoint provider is to be loaded
+dynamically and not part of the application's executable.
+
+The tracepoint provider object file used to create the shared library
+is built like it is using the static library method, only with the
+`-fpic` option added:
+
+[role="term"]
+----
+gcc -c -fpic -I. tp.c
+----
+
+It is then linked as a shared library like this:
+
+[role="term"]
+----
+gcc -shared -Wl,--no-as-needed -o tp.so -llttng-ust tp.o
+----
+
+As previously stated, this tracepoint provider shared object isn't
+linked with the user application: it's loaded manually. This is
+why the application is built with no mention of this tracepoint
+provider, but still needs `libdl`:
+
+[role="term"]
+----
+gcc -o app other.o files.o of.o your.o app.o -ldl
+----
+
+Now, to make LTTng-UST tracing available to the application, the
+`LD_PRELOAD` environment variable is used to preload the tracepoint
+provider shared library _before_ the application actually starts:
+
+[role="term"]
+----
+LD_PRELOAD=/path/to/tp.so ./app
+----
+
+[NOTE]
+====
+It is not safe to use
+`dlclose()` on a tracepoint provider shared object that
+is being actively used for tracing, due to a lack of reference
+counting from LTTng-UST to the shared object.
+
+For example, statically linking a tracepoint provider to a
+shared object which is to be dynamically loaded by an application
+(a plugin, for example) is not safe: the shared object, which
+contains the tracepoint provider, could be dynamically closed
+(`dlclose()`) at any time by the application.
+
+To instrument a shared object, either:
+
+* Statically link the tracepoint provider to the _application_, or
+* Build the tracepoint provider as a shared object (following
+ the procedure shown in this section), and preload it when
+ tracing is needed using the `LD_PRELOAD`
+ environment variable.
+====
+
+Your application will still work without this preloading, albeit without
+LTTng-UST tracing support:
+
+[role="term"]
+----
+./app
+----
+
+
+[[using-lttng-ust-with-daemons]]
+===== Using LTTng-UST with daemons
+
+Some extra care is needed when using `liblttng-ust` with daemon
+applications that call `fork()`, `clone()` or BSD's `rfork()` without
+a following `exec()` family system call. The `liblttng-ust-fork`
+library must be preloaded for the application.
+
+Example:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-fork.so ./app
+----
+
+Or, if you're using a tracepoint provider shared library:
+
+[role="term"]
+----
+LD_PRELOAD="liblttng-ust-fork.so /path/to/tp.so" ./app
+----
+
+
+[[lttng-ust-pkg-config]]
+===== Using pkg-config
+
+On some distributions, LTTng-UST is shipped with a pkg-config metadata
+file, so that you may use the `pkg-config` tool:
+
+[role="term"]
+----
+pkg-config --libs lttng-ust
+----
+
+This prints `-llttng-ust -ldl` on Linux systems.
+
+You may also check the LTTng-UST version using `pkg-config`:
+
+[role="term"]
+----
+pkg-config --modversion lttng-ust
+----
+
+For more information about pkg-config, see
+http://linux.die.net/man/1/pkg-config[its manpage].
+
+
+[role="since-2.5"]
+[[tracef]]
+===== Using `tracef()`
+
+`tracef()` is a small LTTng-UST API to avoid defining your own
+tracepoints and tracepoint providers. The signature of `tracef()` is
+the same as `printf()`'s.
+
+The `tracef()` utility function was developed to make user space tracing
+super simple, albeit with notable disadvantages compared to custom,
+full-fledged tracepoint providers:
+
+* All generated events have the same provider/event names, respectively
+ `lttng_ust_tracef` and `event`.
+* There's no static type checking.
+* The only event field you actually get, named `msg`, is a string
+ potentially containing the values you passed to the function
+ using your own format. This also means that you cannot use filtering
+ using a custom expression at runtime because there are no isolated
+ fields.
+* Since `tracef()` uses C standard library's `vasprintf()` function
+ in the background to format the strings at runtime, its
+ expected performance is lower than using custom tracepoint providers
+ with typed fields, which do not require a conversion to a string.
+
+Thus, `tracef()` is useful for quick prototyping and debugging, but
+should not be considered for any permanent/serious application
+instrumentation.
+
+To use `tracef()`, first include `<lttng/tracef.h>` in the C source file
+where you need to insert probes:
+
+[source,c]
+----
+#include <lttng/tracef.h>
+----
+
+Use `tracef()` like you would use `printf()` in your source code, for
+example:
+
+[source,c]
+----
+ /* ... */
+
+ tracef("my message, my integer: %d", my_integer);
+
+ /* ... */
+----
+
+Link your application with `liblttng-ust`:
+
+[role="term"]
+----
+gcc -o app app.c -llttng-ust
+----
+
+Execute the application as usual:
+
+[role="term"]
+----
+./app
+----
+
+Voilà! Use the `lttng` command line tool to
+<<controlling-tracing,control tracing>>. You can enable `tracef()`
+events like this:
+
+[role="term"]
+----
+lttng enable-event --userspace 'lttng_ust_tracef:*'
+----
+
+
+[[lttng-ust-environment-variables-compiler-flags]]
+===== LTTng-UST environment variables and special compilation flags
+
+A few special environment variables and compile flags may affect the
+behavior of LTTng-UST.
+
+LTTng-UST's debugging can be activated by setting the environment
+variable `LTTNG_UST_DEBUG` to `1` when launching the application. It
+can also be enabled at compile time by defining `LTTNG_UST_DEBUG` when
+compiling LTTng-UST (using the `-DLTTNG_UST_DEBUG` compiler option).
+
+The environment variable `LTTNG_UST_REGISTER_TIMEOUT` can be used to
+specify how long the application should wait for the
+<<lttng-sessiond,session daemon>>'s _registration done_ command
+before proceeding to execute the main program. The timeout value is
+specified in milliseconds. 0 means _don't wait_. -1 means
+_wait forever_. Setting this environment variable to 0 is recommended
+for applications with time contraints on the process startup time.
+
+The default value of `LTTNG_UST_REGISTER_TIMEOUT` (when not defined)
+is **3000{nbsp}ms**.
+
+The compilation definition `LTTNG_UST_DEBUG_VALGRIND` should be enabled
+at build time (`-DLTTNG_UST_DEBUG_VALGRIND`) to allow `liblttng-ust`
+to be used with http://valgrind.org/[Valgrind].
+The side effect of defining `LTTNG_UST_DEBUG_VALGRIND` is that per-CPU
+buffering is disabled.
+
+
+[[cxx-application]]
+==== $$C++$$ application
+
+Because of $$C++$$'s cross-compatibility with the C language, $$C++$$
+applications can be readily instrumented with the LTTng-UST C API.
+
+Follow the <<c-application,C application>> user guide above. It
+should be noted that, in this case, tracepoint providers should have
+the typical `.cpp`, `.cxx` or `.cc` extension and be built with `g++`
+instead of `gcc`. This is the easiest way of avoiding linking errors
+due to symbol name mangling incompatibilities between both languages.
+
+
+[[prebuilt-ust-helpers]]
+==== Prebuilt user space tracing helpers
+
+The LTTng-UST package provides a few helpers that one may find
+useful in some situations. They all work the same way: you must
+preload the appropriate shared object before running the user
+application (using the `LD_PRELOAD` environment variable).
+
+The shared objects are normally found in dir:{/usr/lib}.
+
+The current installed helpers are:
+
+path:{liblttng-ust-libc-wrapper.so} and path:{liblttng-ust-pthread-wrapper.so}::
+ <<liblttng-ust-libc-pthread-wrapper,C{nbsp}standard library
+ and POSIX threads tracing>>.
+
+path:{liblttng-ust-cyg-profile.so} and path:{liblttng-ust-cyg-profile-fast.so}::
+ <<liblttng-ust-cyg-profile,Function tracing>>.
+
+path:{liblttng-ust-dl.so}::
+ <<liblttng-ust-dl,Dynamic linker tracing>>.
+
+The following subsections document what helpers instrument exactly
+and how to use them.
+
+
+[role="since-2.3"]
+[[liblttng-ust-libc-pthread-wrapper]]
+===== C standard library and POSIX threads tracing
+
+path:{liblttng-ust-libc-wrapper.so} and path:{liblttng-ust-pthread-wrapper.so}
+can add instrumentation to respectively some C standard library and
+POSIX threads functions.
+
+The following functions are traceable by path:{liblttng-ust-libc-wrapper.so}:
+
+[role="growable"]
+.Functions instrumented by path:{liblttng-ust-libc-wrapper.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.6+|`ust_libc` |`malloc` |`malloc()`
+ |`calloc` |`calloc()`
+ |`realloc` |`realloc()`
+ |`free` |`free()`
+ |`memalign` |`memalign()`
+ |`posix_memalign` |`posix_memalign()`
+|====
+
+The following functions are traceable by
+path:{liblttng-ust-pthread-wrapper.so}:
+
+[role="growable"]
+.Functions instrumented by path:{liblttng-ust-pthread-wrapper.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.4+|`ust_pthread` |`pthread_mutex_lock_req` |`pthread_mutex_lock()` (request time)
+ |`pthread_mutex_lock_acq` |`pthread_mutex_lock()` (acquire time)
+ |`pthread_mutex_trylock` |`pthread_mutex_trylock()`
+ |`pthread_mutex_unlock` |`pthread_mutex_unlock()`
+|====
+
+All tracepoints have fields corresponding to the arguments of the
+function they instrument.
+
+To use one or the other with any user application, independently of
+how the latter is built, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
+----
+
+or
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-pthread-wrapper.so my-app
+----
+
+To use both, do:
+
+[role="term"]
+----
+LD_PRELOAD="liblttng-ust-libc-wrapper.so liblttng-ust-pthread-wrapper.so" my-app
+----
+
+When the shared object is preloaded, it effectively replaces the
+functions listed in the above tables by wrappers which add tracepoints
+and call the replaced functions.
+
+Of course, like any other tracepoint, the ones above need to be enabled
+in order for LTTng-UST to generate events. This is done using the
+`lttng` command line tool
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[[liblttng-ust-cyg-profile]]
+===== Function tracing
+
+Function tracing is the recording of which functions are entered and
+left during the execution of an application. Like with any LTTng event,
+the precise time at which this happens is also kept.
+
+GCC and clang have an option named
+https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/Code-Gen-Options.html[`-finstrument-functions`]
+which generates instrumentation calls for entry and exit to functions.
+The LTTng-UST function tracing helpers, path:{liblttng-ust-cyg-profile.so}
+and path:{liblttng-ust-cyg-profile-fast.so}, take advantage of this feature
+to add instrumentation to the two generated functions (which contain
+`cyg_profile` in their names, hence the shared object's name).
+
+In order to use LTTng-UST function tracing, the translation units to
+instrument must be built using the `-finstrument-functions` compiler
+flag.
+
+LTTng-UST function tracing comes in two flavors, each providing
+different trade-offs: path:{liblttng-ust-cyg-profile-fast.so} and
+path:{liblttng-ust-cyg-profile.so}.
+
+**path:{liblttng-ust-cyg-profile-fast.so}** is a lightweight variant that
+should only be used where it can be _guaranteed_ that the complete event
+stream is recorded without any missing events. Any kind of duplicate
+information is left out. This version registers the following
+tracepoints:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-cyg-profile-fast.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`lttng_ust_cyg_profile_fast`
+
+|`func_entry`
+a|Function entry
+
+`addr`::
+ Address of called function.
+
+|`func_exit`
+|Function exit
+|====
+
+Assuming no event is lost, having only the function addresses on entry
+is enough for creating a call graph (remember that a recorded event
+always contains the ID of the CPU that generated it). A tool like
+https://sourceware.org/binutils/docs/binutils/addr2line.html[`addr2line`]
+may be used to convert function addresses back to source files names
+and line numbers.
+
+The other helper,
+**path:{liblttng-ust-cyg-profile.so}**,
+is a more robust variant which also works for use cases where
+events might get discarded or not recorded from application startup.
+In these cases, the trace analyzer needs extra information to be
+able to reconstruct the program flow. This version registers the
+following tracepoints:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-cyg-profile.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`lttng_ust_cyg_profile`
+
+|`func_entry`
+a|Function entry
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+
+|`func_exit`
+a|Function exit
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+|====
+
+To use one or the other variant with any user application, assuming at
+least one translation unit of the latter is compiled with the
+`-finstrument-functions` option, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-cyg-profile-fast.so my-app
+----
+
+or
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-cyg-profile.so my-app
+----
+
+It might be necessary to limit the number of source files where
+`-finstrument-functions` is used to prevent excessive amount of trace
+data to be generated at runtime.
+
+TIP: When using GCC, at least, you can use
+ the `-finstrument-functions-exclude-function-list`
+ option to avoid instrumenting entries and exits of specific
+ symbol names.
+
+All events generated from LTTng-UST function tracing are provided on
+log level `TRACE_DEBUG_FUNCTION`, which is useful to easily enable
+function tracing events in your tracing session using the
+`--loglevel-only` option of `lttng enable-event`
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[role="since-2.4"]
+[[liblttng-ust-dl]]
+===== Dynamic linker tracing
+
+This LTTng-UST helper causes all calls to `dlopen()` and `dlclose()`
+in the target application to be traced with LTTng.
+
+The helper's shared object, path:{liblttng-ust-dl.so}, registers the
+following tracepoints when preloaded:
+
+[role="growable",options="header,autowidth"]
+.Functions instrumented by path:{liblttng-ust-dl.so}
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`ust_baddr`
+
+|`push`
+a|`dlopen()` call
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+
+`sopath`::
+ File system path to the loaded shared object.
+
+`size`::
+ File size of the the loaded shared object.
+
+`mtime`::
+ Last modification time (seconds since Epoch time) of the loaded shared
+ object.
+
+|`pop`
+a|Function exit
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+|====
+
+To use this LTTng-UST helper with any user application, independently of
+how the latter is built, do:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-dl.so my-app
+----
+
+Of course, like any other tracepoint, the ones above need to be enabled
+in order for LTTng-UST to generate events. This is done using the
+`lttng` command line tool
+(see <<controlling-tracing,Controlling tracing>>).
+
+
+[role="since-2.4"]
+[[java-application]]
+==== Java application
+
+LTTng-UST provides a _logging_ back-end for Java applications using either
+http://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html[`java.util.logging`]
+(JUL) or
+http://logging.apache.org/log4j/1.2/[Apache log4j 1.2]
+This back-end is called the _LTTng-UST Java agent_, and it is responsible
+for the communications with an LTTng session daemon.
+
+From the user's point of view, once the LTTng-UST Java agent has been
+initialized, JUL and log4j loggers may be created and used as usual.
+The agent adds its own handler to the _root logger_, so that all
+loggers may generate LTTng events with no effort.
+
+Common JUL/log4j features are supported using the `lttng` tool
+(see <<controlling-tracing,Controlling tracing>>):
+
+* listing all logger names
+* enabling/disabling events per logger name
+* JUL/log4j log levels
+
+
+[role="since-2.1"]
+[[jul]]
+===== `java.util.logging`
+
+Here's an example of tracing a Java application which is using
+**`java.util.logging`**:
+
+[source,java]
+----
+import java.util.logging.Logger;
+import org.lttng.ust.agent.LTTngAgent;
+
+public class Test
+{
+ private static final int answer = 42;
+
+ public static void main(String[] argv) throws Exception
+ {
+ // create a logger
+ Logger logger = Logger.getLogger("jello");
+
+ // call this as soon as possible (before logging)
+ LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+
+ // log at will!
+ logger.info("some info");
+ logger.warning("some warning");
+ Thread.sleep(500);
+ logger.finer("finer information; the answer is " + answer);
+ Thread.sleep(123);
+ logger.severe("error!");
+
+ // not mandatory, but cleaner
+ lttngAgent.dispose();
+ }
+}
+----
+
+The LTTng-UST Java agent is packaged in a JAR file named
+`liblttng-ust-agent.jar` It is typically located in
+dir:{/usr/lib/lttng/java}. To compile the snippet above
+(saved as `Test.java`), do:
+
+[role="term"]
+----
+javac -cp /usr/lib/lttng/java/liblttng-ust-agent.jar Test.java
+----
+
+You can run the resulting compiled class like this:
+
+[role="term"]
+----
+java -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:. Test
+----
+
+NOTE: http://openjdk.java.net/[OpenJDK] 7 is used for development and
+continuous integration, thus this version is directly supported.
+However, the LTTng-UST Java agent has also been tested with OpenJDK 6.
+
+
+[role="since-2.6"]
+[[log4j]]
+===== Apache log4j 1.2
+
+LTTng features an Apache log4j 1.2 agent, which means your existing
+Java applications using log4j 1.2 for logging can record events to
+LTTng traces with just a minor source code modification.
+
+NOTE: This version of LTTng does not support Log4j 2.
+
+Here's an example:
+
+[source,java]
+----
+import org.apache.log4j.Logger;
+import org.apache.log4j.BasicConfigurator;
+import org.lttng.ust.agent.LTTngAgent;
+
+public class Test
+{
+ private static final int answer = 42;
+
+ public static void main(String[] argv) throws Exception
+ {
+ // create and configure a logger
+ Logger logger = Logger.getLogger(Test.class);
+ BasicConfigurator.configure();
+
+ // call this as soon as possible (before logging)
+ LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+
+ // log at will!
+ logger.info("some info");
+ logger.warn("some warning");
+ Thread.sleep(500);
+ logger.debug("debug information; the answer is " + answer);
+ Thread.sleep(123);
+ logger.error("error!");
+ logger.fatal("fatal error!");
+
+ // not mandatory, but cleaner
+ lttngAgent.dispose();
+ }
+}
+----
+
+To compile the snippet above, do:
+
+[role="term"]
+----
+javac -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:$LOG4JCP Test.java
+----
+
+where `$LOG4JCP` is the log4j 1.2 JAR file path.
+
+You can run the resulting compiled class like this:
+
+[role="term"]
+----
+java -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:$LOG4JCP:. Test
+----
+
+
+[[instrumenting-linux-kernel]]
+==== Linux kernel
+
+The Linux kernel can be instrumented for LTTng tracing, either its core
+source code or a kernel module. It has to be noted that Linux is
+readily traceable using LTTng since many parts of its source code are
+already instrumented: this is the job of the upstream
+http://git.lttng.org/?p=lttng-modules.git[LTTng-modules]
+package. This section presents how to add LTTng instrumentation where it
+does not currently exist and how to instrument custom kernel modules.
+
+All LTTng instrumentation in the Linux kernel is based on an existing
+infrastructure which bears the name of its main macro, `TRACE_EVENT()`.
+This macro is used to define tracepoints,
+each tracepoint having a name, usually with the
++__subsys_____name__+ format,
++_subsys_+ being the subsystem name and
++_name_+ the specific event name.
+
+Tracepoints defined with `TRACE_EVENT()` may be inserted anywhere in
+the Linux kernel source code, after what callbacks, called _probes_,
+may be registered to execute some action when a tracepoint is
+executed. This mechanism is directly used by ftrace and perf,
+but cannot be used as is by LTTng: an adaptation layer is added to
+satisfy LTTng's specific needs.
+
+With that in mind, this documentation does not cover the `TRACE_EVENT()`
+format and how to use it, but it is mandatory to understand it and use
+it to instrument Linux for LTTng. A series of
+LWN articles explain
+`TRACE_EVENT()` in details:
+http://lwn.net/Articles/379903/[part 1],
+http://lwn.net/Articles/381064/[part 2], and
+http://lwn.net/Articles/383362/[part 3].
+Once you master `TRACE_EVENT()` enough for your use case, continue
+reading this section so that you can add the LTTng adaptation layer of
+instrumentation.
+
+This section first discusses the general method of instrumenting the
+Linux kernel for LTTng. This method is then reused for the specific
+case of instrumenting a kernel module.
+
+
+[[instrumenting-linux-kernel-itself]]
+===== Instrumenting the Linux kernel for LTTng
+
+The following subsections explain strictly how to add custom LTTng
+instrumentation to the Linux kernel. They do not explain how the
+macros actually work and the internal mechanics of the tracer.
+
+You should have a Linux kernel source code tree to work with.
+Throughout this section, all file paths are relative to the root of
+this tree unless otherwise stated.
+
+You need a copy of the LTTng-modules Git repository:
+
+[role="term"]
+----
+git clone git://git.lttng.org/lttng-modules.git
+----
+
+The steps to add custom LTTng instrumentation to a Linux kernel
+involves defining and using the mainline `TRACE_EVENT()` tracepoints
+first, then writing and using the LTTng adaptation layer.
+
+
+[[mainline-trace-event]]
+===== Defining/using tracepoints with mainline `TRACE_EVENT()` infrastructure
+
+The first step is to define tracepoints using the mainline Linux
+`TRACE_EVENT()` macro and insert tracepoints where you want them.
+Your tracepoint definitions reside in a header file in
+dir:{include/trace/events}. If you're adding tracepoints to an existing
+subsystem, edit its appropriate header file.
+
+As an example, the following header file (let's call it
+dir:{include/trace/events/hello.h}) defines one tracepoint using
+`TRACE_EVENT()`:
+
+[source,c]
+----
+/* subsystem name is "hello" */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hello
+
+#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HELLO_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(
+ /* "hello" is the subsystem name, "world" is the event name */
+ hello_world,
+
+ /* tracepoint function prototype */
+ TP_PROTO(int foo, const char* bar),
+
+ /* arguments for this tracepoint */
+ TP_ARGS(foo, bar),
+
+ /* LTTng doesn't need those */
+ TP_STRUCT__entry(),
+ TP_fast_assign(),
+ TP_printk("", 0)
+);
+
+#endif
+
+/* this part must be outside protection */
+#include <trace/define_trace.h>
+----
+
+Notice that we don't use any of the last three arguments: they
+are left empty here because LTTng doesn't need them. You would only fill
+`TP_STRUCT__entry()`, `TP_fast_assign()` and `TP_printk()` if you were
+to also use this tracepoint for ftrace/perf.
+
+Once this is done, you may place calls to `trace_hello_world()`
+wherever you want in the Linux source code. As an example, let us place
+such a tracepoint in the `usb_probe_device()` static function
+(path:{drivers/usb/core/driver.c}):
+
+[source,c]
+----
+/* called from driver core with dev locked */
+static int usb_probe_device(struct device *dev)
+{
+ struct usb_device_driver *udriver = to_usb_device_driver(dev->driver);
+ struct usb_device *udev = to_usb_device(dev);
+ int error = 0;
+
+ trace_hello_world(udev->devnum, udev->product);
+
+ /* ... */
+}
+----
+
+This tracepoint should fire every time a USB device is plugged in.
+
+At the top of path:{driver.c}, we need to include our actual tracepoint
+definition and, in this case (one place per subsystem), define
+`CREATE_TRACE_POINTS`, which creates our tracepoint:
+
+[source,c]
+----
+/* ... */
+
+#include "usb.h"
+
+#define CREATE_TRACE_POINTS
+#include <trace/events/hello.h>
+
+/* ... */
+----
+
+Build your custom Linux kernel. In order to use LTTng, make sure the
+following kernel configuration options are enabled:
+
+* `CONFIG_MODULES` (loadable module support)
+* `CONFIG_KALLSYMS` (load all symbols for debugging/kksymoops)
+* `CONFIG_HIGH_RES_TIMERS` (high resolution timer support)
+* `CONFIG_TRACEPOINTS` (kernel tracepoint instrumentation)
+
+Boot the custom kernel. The directory
+dir:{/sys/kernel/debug/tracing/events/hello} should exist if everything
+went right, with a dir:{hello_world} subdirectory.
+
+
+[[lttng-adaptation-layer]]
+===== Adding the LTTng adaptation layer
+
+The steps to write the LTTng adaptation layer are, in your
+LTTng-modules copy's source code tree:
+
+. In dir:{instrumentation/events/lttng-module},
+ add a header +__subsys__.h+ for your custom
+ subsystem +__subsys__+ and write your
+ tracepoint definitions using LTTng-modules macros in it.
+ Those macros look like the mainline kernel equivalents,
+ but they present subtle, yet important differences.
+. In dir:{probes}, create the C source file of the LTTng probe kernel
+ module for your subsystem. It should be named
+ +lttng-probe-__subsys__.c+.
+. Edit path:{probes/Makefile} so that the LTTng-modules project
+ builds your custom LTTng probe kernel module.
+. Build and install LTTng kernel modules.
+
+Following our `hello_world` event example, here's the content of
+path:{instrumentation/events/lttng-module/hello.h}:
+
+[source,c]
+----
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM hello
+
+#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_HELLO_H
+
+#include "../../../probes/lttng-tracepoint-event.h"
+#include <linux/tracepoint.h>
+
+LTTNG_TRACEPOINT_EVENT(
+ /* format identical to mainline version for those */
+ hello_world,
+ TP_PROTO(int foo, const char* bar),
+ TP_ARGS(foo, bar),
+
+ /* possible differences */
+ TP_STRUCT__entry(
+ __field(int, my_int)
+ __field(char, char0)
+ __field(char, char1)
+ __string(product, bar)
+ ),
+
+ /* notice the use of tp_assign()/tp_strcpy() and no semicolons */
+ TP_fast_assign(
+ tp_assign(my_int, foo)
+ tp_assign(char0, bar[0])
+ tp_assign(char1, bar[1])
+ tp_strcpy(product, bar)
+ ),
+
+ /* This one is actually not used by LTTng either, but must be
+ * present for the moment.
+ */
+ TP_printk("", 0)
+
+/* no semicolon after this either */
+)
+
+#endif
+
+/* other difference: do NOT include <trace/define_trace.h> */
+#include "../../../probes/define_trace.h"
+----
+
+Some possible entries for `TP_STRUCT__entry()` and `TP_fast_assign()`,
+in the case of LTTng-modules, are shown in the
+<<lttng-modules-ref,LTTng-modules reference>> section.
+
+The best way to learn how to use the above macros is to inspect
+existing LTTng tracepoint definitions in
+dir:{instrumentation/events/lttng-module} header files. Compare
+them with the Linux kernel mainline versions in
+dir:{include/trace/events}.
+
+The next step is writing the LTTng probe kernel module C source file.
+This one is named +lttng-probe-__subsys__.c+
+in dir:{probes}. You may always use the following template:
+
+[source,c]
+----
+#include <linux/module.h>
+#include "../lttng-tracer.h"
+
+/* Build time verification of mismatch between mainline TRACE_EVENT()
+ * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
+ */
+#include <trace/events/hello.h>
+
+/* create LTTng tracepoint probes */
+#define LTTNG_PACKAGE_BUILD
+#define CREATE_TRACE_POINTS
+#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module
+
+#include "../instrumentation/events/lttng-module/hello.h"
+
+MODULE_LICENSE("GPL and additional rights");
+MODULE_AUTHOR("Your name <your-email>");
+MODULE_DESCRIPTION("LTTng hello probes");
+MODULE_VERSION(__stringify(LTTNG_MODULES_MAJOR_VERSION) "."
+ __stringify(LTTNG_MODULES_MINOR_VERSION) "."
+ __stringify(LTTNG_MODULES_PATCHLEVEL_VERSION)
+ LTTNG_MODULES_EXTRAVERSION);
+----
+
+Just replace `hello` with your subsystem name. In this example,
+`<trace/events/hello.h>`, which is the original mainline tracepoint
+definition header, is included for verification purposes: the
+LTTng-modules build system is able to emit an error at build time when
+the arguments of the mainline `TRACE_EVENT()` definitions do not match
+the ones of the LTTng-modules adaptation layer
+(`LTTNG_TRACEPOINT_EVENT()`).
+
+Edit path:{probes/Makefile} and add your new kernel module object
+next to existing ones:
+
+[source,make]
+----
+# ...
+
+obj-m += lttng-probe-module.o
+obj-m += lttng-probe-power.o
+
+obj-m += lttng-probe-hello.o
+
+# ...
+----
+
+Time to build! Point to your custom Linux kernel source tree using
+the `KERNELDIR` variable:
+
+[role="term"]
+----
+make KERNELDIR=/path/to/custom/linux
+----
+
+Finally, install modules:
+
+[role="term"]
+----
+sudo make modules_install
+----
+
+
+[[instrumenting-linux-kernel-tracing]]
+===== Tracing
+
+The <<controlling-tracing,Controlling tracing>> section explains
+how to use the `lttng` tool to create and control tracing sessions.
+Although the `lttng` tool loads the appropriate _known_ LTTng kernel
+modules when needed (by launching `root`'s session daemon), it won't
+load your custom `lttng-probe-hello` module by default. You need to
+manually start an LTTng session daemon as `root` and use the
+`--extra-kmod-probes` option to append your custom probe module to the
+default list:
+
+[role="term"]
+----
+sudo pkill -u root lttng-sessiond
+sudo lttng-sessiond --extra-kmod-probes=hello
+----
+
+The first command makes sure any existing instance is killed. If
+you're not interested in using the default probes, or if you only
+want to use a few of them, you could use `--kmod-probes` instead,
+which specifies an absolute list:
+
+[role="term"]
+----
+sudo lttng-sessiond --kmod-probes=hello,ext4,net,block,signal,sched
+----
+
+Confirm the custom probe module is loaded:
+
+[role="term"]
+----
+lsmod | grep lttng_probe_hello
+----
+
+The `hello_world` event should appear in the list when doing
+
+[role="term"]
+----
+lttng list --kernel | grep hello
+----
+
+You may now create an LTTng tracing session, enable the `hello_world`
+kernel event (and others if you wish) and start tracing:
+
+[role="term"]
+----
+sudo lttng create my-session
+sudo lttng enable-event --kernel hello_world
+sudo lttng start
+----
+
+Plug a few USB devices, then stop tracing and inspect the trace (if
+http://diamon.org/babeltrace[Babeltrace]
+is installed):
+
+[role="term"]
+----
+sudo lttng stop
+sudo lttng view
+----
+
+Here's a sample output:
+
+----
+[15:30:34.835895035] (+?.?????????) hostname hello_world: { cpu_id = 1 }, { my_int = 8, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
+[15:30:42.262781421] (+7.426886386) hostname hello_world: { cpu_id = 1 }, { my_int = 9, char0 = 80, char1 = 97, product = "Patriot Memory" }
+[15:30:48.175621778] (+5.912840357) hostname hello_world: { cpu_id = 1 }, { my_int = 10, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
+----
+
+Two USB flash drives were used for this test.
+
+You may change your LTTng custom probe, rebuild it and reload it at
+any time when not tracing. Make sure you remove the old module
+(either by killing the root LTTng session daemon which loaded the
+module in the first place, or by using `modprobe --remove` directly)
+before loading the updated one.
+
+
+[[instrumenting-out-of-tree-linux-kernel]]
+===== Advanced: Instrumenting an out-of-tree Linux kernel module for LTTng
+
+Instrumenting a custom Linux kernel module for LTTng follows the exact
+same steps as
+<<instrumenting-linux-kernel-itself,adding instrumentation
+to the Linux kernel itself>>,
+the only difference being that your mainline tracepoint definition
+header doesn't reside in the mainline source tree, but in your
+kernel module source tree.
+
+The only reference to this mainline header is in the LTTng custom
+probe's source code (path:{probes/lttng-probe-hello.c} in our example),
+for build time verification:
+
+[source,c]
+----
+/* ... */
+
+/* Build time verification of mismatch between mainline TRACE_EVENT()
+ * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
+ */
+#include <trace/events/hello.h>
+
+/* ... */
+----
+
+The preferred, flexible way to include your module's mainline
+tracepoint definition header is to put it in a specific directory
+relative to your module's root (`tracepoints`, for example) and include it
+relative to your module's root directory in the LTTng custom probe's
+source:
+
+[source,c]
+----
+#include <tracepoints/hello.h>
+----
+
+You may then build LTTng-modules by adding your module's root
+directory as an include path to the extra C flags:
+
+[role="term"]
+----
+make ccflags-y=-I/path/to/kernel/module KERNELDIR=/path/to/custom/linux
+----
+
+Using `ccflags-y` allows you to move your kernel module to another
+directory and rebuild the LTTng-modules project with no change to
+source files.
+
+
+[role="since-2.5"]
+[[proc-lttng-logger-abi]]
+==== LTTng logger ABI
+
+The `lttng-tracer` Linux kernel module, installed by the LTTng-modules
+package, creates a special LTTng logger ABI file path:{/proc/lttng-logger}
+when loaded. Writing text data to this file generates an LTTng kernel
+domain event named `lttng_logger`.
+
+Unlike other kernel domain events, `lttng_logger` may be enabled by
+any user, not only root users or members of the tracing group.
+
+To use the LTTng logger ABI, simply write a string to
+path:{/proc/lttng-logger}:
+
+[role="term"]
+----
+echo -n 'Hello, World!' > /proc/lttng-logger
+----
+
+The `msg` field of the `lttng_logger` event contains the recorded
+message.
+
+NOTE: Messages are split in chunks of 1024{nbsp}bytes.
+
+The LTTng logger ABI is a quick and easy way to trace some events from
+user space through the kernel tracer. However, it is much more basic
+than LTTng-UST: it's slower (involves system call round-trip to the
+kernel and only supports logging strings). The LTTng logger ABI is
+particularly useful for recording logs as LTTng traces from shell
+scripts, potentially combining them with other Linux kernel/user space
+events.
+
+
+[[instrumenting-32-bit-app-on-64-bit-system]]
+==== Advanced: Instrumenting a 32-bit application on a 64-bit system
+
+[[advanced-instrumenting-techniques]]In order to trace a 32-bit
+application running on a 64-bit system,
+LTTng must use a dedicated 32-bit
+<<lttng-consumerd,consumer daemon>>. This section discusses how to
+build that daemon (which is _not_ part of the default 64-bit LTTng
+build) and the LTTng 32-bit tracing libraries, and how to instrument
+a 32-bit application in that context.
+
+Make sure you install all 32-bit versions of LTTng dependencies.
+Their names can be found in the `README.md` files of each LTTng package
+source. How to find and install them depends on your target's
+Linux distribution. `gcc-multilib` is a common package name for the
+multilib version of GCC, which you also need.
+
+The following packages will be built for 32-bit support on a 64-bit
+system: http://urcu.so/[Userspace RCU],
+LTTng-UST and LTTng-tools.
+
+
+[[building-32-bit-userspace-rcu]]
+===== Building 32-bit Userspace RCU
+
+Follow this:
+
+[role="term"]
+----
+git clone git://git.urcu.so/urcu.git
+cd urcu
+./bootstrap
+./configure --libdir=/usr/lib32 CFLAGS=-m32
+make
+sudo make install
+sudo ldconfig
+----
+
+The `-m32` C compiler flag creates 32-bit object files and `--libdir`
+indicates where to install the resulting libraries.
+
+
+[[building-32-bit-lttng-ust]]
+===== Building 32-bit LTTng-UST
+
+Follow this:
+
+[role="term"]
+----
+git clone http://git.lttng.org/lttng-ust.git
+cd lttng-ust
+./bootstrap
+./configure --prefix=/usr \
+ --libdir=/usr/lib32 \
+ CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS=-L/usr/lib32
+make
+sudo make install
+sudo ldconfig
+----
+
+`-L/usr/lib32` is required for the build to find the 32-bit versions
+of Userspace RCU and other dependencies.
+
+[NOTE]
+====
+Depending on your Linux distribution,
+32-bit libraries could be installed at a different location than
+dir:{/usr/lib32}. For example, Debian is known to install
+some 32-bit libraries in dir:{/usr/lib/i386-linux-gnu}.
+
+In this case, make sure to set `LDFLAGS` to all the
+relevant 32-bit library paths, for example,
+`LDFLAGS="-L/usr/lib32 -L/usr/lib/i386-linux-gnu"`.
+====
+
+NOTE: You may add options to path:{./configure} if you need them, e.g., for
+Java and SystemTap support. Look at `./configure --help` for more
+information.
+
+
+[[building-32-bit-lttng-tools]]
+===== Building 32-bit LTTng-tools
+
+Since the host is a 64-bit system, most 32-bit binaries and libraries of
+LTTng-tools are not needed; the host uses their 64-bit counterparts.
+The required step here is building and installing a 32-bit consumer
+daemon.
+
+Follow this:
+
+[role="term"]
+----
+git clone http://git.lttng.org/lttng-tools.git
+cd lttng-ust
+./bootstrap
+./configure --prefix=/usr \
+ --libdir=/usr/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS=-L/usr/lib32
+make
+cd src/bin/lttng-consumerd
+sudo make install
+sudo ldconfig
+----
+
+The above commands build all the LTTng-tools project as 32-bit
+applications, but only installs the 32-bit consumer daemon.
+
+
+[[building-64-bit-lttng-tools]]
+===== Building 64-bit LTTng-tools
+
+Finally, you need to build a 64-bit version of LTTng-tools which is
+aware of the 32-bit consumer daemon previously built and installed:
+
+[role="term"]
+----
+make clean
+./bootstrap
+./configure --prefix=/usr \
+ --with-consumerd32-libdir=/usr/lib32 \
+ --with-consumerd32-bin=/usr/lib32/lttng/libexec/lttng-consumerd
+make
+sudo make install
+sudo ldconfig
+----
+
+Henceforth, the 64-bit session daemon automatically finds the
+32-bit consumer daemon if required.
+
+
+[[building-instrumented-32-bit-c-application]]
+===== Building an instrumented 32-bit C application
+
+Let us reuse the _Hello world_ example of
+<<tracing-your-own-user-application,Tracing your own user application>>
+(<<getting-started,Getting started>> chapter).
+
+The instrumentation process is unaltered.
+
+First, a typical 64-bit build (assuming you're running a 64-bit system):
+
+[role="term"]
+----
+gcc -o hello64 -I. hello.c hello-tp.c -ldl -llttng-ust
+----
+
+Now, a 32-bit build:
+
+[role="term"]
+----
+gcc -o hello32 -I. -m32 hello.c hello-tp.c -L/usr/lib32 \
+ -ldl -llttng-ust -Wl,-rpath,/usr/lib32
+----
+
+The `-rpath` option, passed to the linker, makes the dynamic loader
+check for libraries in dir:{/usr/lib32} before looking in its default paths,
+where it should find the 32-bit version of `liblttng-ust`.
+
+
+[[running-32-bit-and-64-bit-c-applications]]
+===== Running 32-bit and 64-bit versions of an instrumented C application
+
+Now, both 32-bit and 64-bit versions of the _Hello world_ example above
+can be traced in the same tracing session. Use the `lttng` tool as usual
+to create a tracing session and start tracing:
+
+[role="term"]
+----
+lttng create session-3264
+lttng enable-event -u -a
+./hello32
+./hello64
+lttng stop
+----
+
+Use `lttng view` to verify both processes were
+successfully traced.
+
+
+[[controlling-tracing]]
+=== Controlling tracing
+
+Once you're in possession of a software that is properly
+<<instrumenting,instrumented>> for LTTng tracing, be it thanks to
+the built-in LTTng probes for the Linux kernel, a custom user
+application or a custom Linux kernel, all that is left is actually
+tracing it. As a user, you control LTTng tracing using a single command
+line interface: the `lttng` tool. This tool uses `liblttng-ctl` behind
+the scene to connect to and communicate with session daemons. LTTng
+session daemons may either be started manually (`lttng-sessiond`) or
+automatically by the `lttng` command when needed. Trace data may
+be forwarded to the network and used elsewhere using an LTTng relay
+daemon (`lttng-relayd`).
+
+The manpages of `lttng`, `lttng-sessiond` and `lttng-relayd` are pretty
+complete, thus this section is not an online copy of the latter (we
+leave this contents for the
+<<online-lttng-manpages,Online LTTng manpages>> section).
+This section is rather a tour of LTTng
+features through practical examples and tips.
+
+If not already done, make sure you understand the core concepts
+and how LTTng components connect together by reading the
+<<understanding-lttng,Understanding LTTng>> chapter; this section
+assumes you are familiar with them.
+
+
+[[creating-destroying-tracing-sessions]]
+==== Creating and destroying tracing sessions
+
+Whatever you want to do with `lttng`, it has to happen inside a
+**tracing session**, created beforehand. A session, in general, is a
+per-user container of state. A tracing session is no different; it
+keeps a specific state of stuff like:
+
+* session name
+* enabled/disabled channels with associated parameters
+* enabled/disabled events with associated log levels and filters
+* context information added to channels
+* tracing activity (started or stopped)
+
+and more.
+
+A single user may have many active tracing sessions. LTTng session
+daemons are the ultimate owners and managers of tracing sessions. For
+user space tracing, each user has its own session daemon. Since Linux
+kernel tracing requires root privileges, only `root`'s session daemon
+may enable and trace kernel events. However, `lttng` has a `--group`
+option (which is passed to `lttng-sessiond` when starting it) to
+specify the name of a _tracing group_ which selected users may be part
+of to be allowed to communicate with `root`'s session daemon. By
+default, the tracing group name is `tracing`.
+
+To create a tracing session, do:
+
+[role="term"]
+----
+lttng create my-session
+----
+
+This creates a new tracing session named `my-session` and make it
+the current one. If you don't specify a name (running only
+`lttng create`), your tracing session is named `auto` followed by the
+current date and time. Traces
+are written in +\~/lttng-traces/__session__-+ followed
+by the tracing session's creation date/time by default, where
++__session__+ is the tracing session name. To save them
+at a different location, use the `--output` option:
+
+[role="term"]
+----
+lttng create --output /tmp/some-directory my-session
+----
+
+You may create as many tracing sessions as you wish:
+
+[role="term"]
+----
+lttng create other-session
+lttng create yet-another-session
+----
+
+You may view all existing tracing sessions using the `list` command:
+
+[role="term"]
+----
+lttng list
+----
+
+The state of a _current tracing session_ is kept in path:{~/.lttngrc}. Each
+invocation of `lttng` reads this file to set its current tracing
+session name so that you don't have to specify a session name for each
+command. You could edit this file manually, but the preferred way to
+set the current tracing session is to use the `set-session` command:
+
+[role="term"]
+----
+lttng set-session other-session
+----
+
+Most `lttng` commands accept a `--session` option to specify the name
+of the target tracing session.
+
+Any existing tracing session may be destroyed using the `destroy`
+command:
+
+[role="term"]
+----
+lttng destroy my-session
+----
+
+Providing no argument to `lttng destroy` destroys the current
+tracing session. Destroying a tracing session stops any tracing
+running within the latter. Destroying a tracing session frees resources
+acquired by the session daemon and tracer side, making sure to flush
+all trace data.
+
+You can't do much with LTTng using only the `create`, `set-session`
+and `destroy` commands of `lttng`, but it is essential to know them in
+order to control LTTng tracing, which always happen within the scope of
+a tracing session.
+
+
+[[enabling-disabling-events]]
+==== Enabling and disabling events
+
+Inside a tracing session, individual events may be enabled or disabled
+so that tracing them may or may not generate trace data.
+
+We sometimes use the term _event_ metonymically throughout this text to
+refer to a specific condition, or _rule_, that could lead, when
+satisfied, to an actual occurring event (a point at a specific position
+in source code/binary program, logical processor and time capturing
+some payload) being recorded as trace data. This specific condition is
+composed of:
+
+. A **domain** (kernel, user space, `java.util.logging`, or log4j)
+ (required).
+. One or many **instrumentation points** in source code or binary
+ program (tracepoint name, address, symbol name, function name,
+ logger name, amongst other types of probes) to be executed (required).
+. A **log level** (each instrumentation point declares its own log
+ level) or log level range to match (optional; only valid for user
+ space domain).
+. A **custom user expression**, or **filter**, that must evaluate to
+ _true_ when a tracepoint is executed (optional; only valid for user
+ space domain).
+
+All conditions are specified using arguments passed to the
+`enable-event` command of the `lttng` tool.
+
+Condition 1 is specified using either `--kernel`/`-k` (kernel),
+`--userspace`/`-u` (user space), `--jul`/`-j`
+(JUL), or `--log4j`/`-l` (log4j).
+Exactly one of those four arguments must be specified.
+
+Condition 2 is specified using one of:
+
+`--tracepoint`::
+ Tracepoint.
+
+`--probe`::
+ Dynamic probe (address, symbol name or combination
+ of both in binary program; only valid for kernel domain).
+
+`--function`::
+ function entry/exit (address, symbol name or
+ combination of both in binary program; only valid for kernel domain).
+
+`--syscall`::
+ System call entry/exit (only valid for kernel domain).
+
+When none of the above is specified, `enable-event` defaults to
+using `--tracepoint`.
+
+Condition 3 is specified using one of:
+
+`--loglevel`::
+ Log level range from the specified level to the most severe
+ level.
+
+`--loglevel-only`::
+ Specific log level.
+
+See `lttng enable-event --help` for the complete list of log level
+names.
+
+Condition 4 is specified using the `--filter` option. This filter is
+a C-like expression, potentially reading real-time values of event
+fields, that has to evaluate to _true_ for the condition to be satisfied.
+Event fields are read using plain identifiers while context fields
+must be prefixed with `$ctx.`. See `lttng enable-event --help` for
+all usage details.
+
+The aforementioned arguments are combined to create and enable events.
+Each unique combination of arguments leads to a different
+_enabled event_. The log level and filter arguments are optional, their
+default values being respectively all log levels and a filter which
+always returns _true_.
+
+Here are a few examples (you must
+<<creating-destroying-tracing-sessions,create a tracing session>>
+first):
+
+[role="term"]
+----
+lttng enable-event -u --tracepoint my_app:hello_world
+lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_WARNING
+lttng enable-event -u --tracepoint 'my_other_app:*'
+lttng enable-event -u --tracepoint my_app:foo_bar \
+ --filter 'some_field <= 23 && !other_field'
+lttng enable-event -k --tracepoint sched_switch
+lttng enable-event -k --tracepoint gpio_value
+lttng enable-event -k --function usb_probe_device usb_probe_device
+lttng enable-event -k --syscall --all
+----
+
+The wildcard symbol, `*`, matches _anything_ and may only be used at
+the end of the string when specifying a _tracepoint_. Make sure to
+use it between single quotes in your favorite shell to avoid
+undesired shell expansion.
+
+System call events can be enabled individually, too:
+
+[role="term"]
+----
+lttng enable-event -k --syscall open
+lttng enable-event -k --syscall read
+lttng enable-event -k --syscall fork,chdir,pipe
+----
+
+The complete list of available system call events can be
+obtained using
+
+[role="term"]
+----
+lttng list --kernel --syscall
+----
+
+You can see a list of events (enabled or disabled) using
+
+[role="term"]
+----
+lttng list some-session
+----
+
+where `some-session` is the name of the desired tracing session.
+
+What you're actually doing when enabling events with specific conditions
+is creating a **whitelist** of traceable events for a given channel.
+Thus, the following case presents redundancy:
+
+[role="term"]
+----
+lttng enable-event -u --tracepoint my_app:hello_you
+lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_DEBUG
+----
+
+The second command, matching a log level range, is useless since the first
+command enables all tracepoints matching the same name,
+`my_app:hello_you`.
+
+Disabling an event is simpler: you only need to provide the event
+name to the `disable-event` command:
+
+[role="term"]
+----
+lttng disable-event --userspace my_app:hello_you
+----
+
+This name has to match a name previously given to `enable-event` (it
+has to be listed in the output of `lttng list some-session`).
+The `*` wildcard is supported, as long as you also used it in a
+previous `enable-event` invocation.
+
+Disabling an event does not add it to some blacklist: it simply removes
+it from its channel's whitelist. This is why you cannot disable an event
+which wasn't previously enabled.
+
+A disabled event doesn't generate any trace data, even if all its
+specified conditions are met.
+
+Events may be enabled and disabled at will, either when LTTng tracers
+are active or not. Events may be enabled before a user space application
+is even started.
+
+
+[[basic-tracing-session-control]]
+==== Basic tracing session control
+
+Once you have
+<<creating-destroying-tracing-sessions,created a tracing session>>
+and <<enabling-disabling-events,enabled one or more events>>,
+you may activate the LTTng tracers for the current tracing session at
+any time:
+
+[role="term"]
+----
+lttng start
+----
+
+Subsequently, you may stop the tracers:
+
+[role="term"]
+----
+lttng stop
+----
+
+LTTng is very flexible: user space applications may be launched before
+or after the tracers are started. Events are only recorded if they
+are properly enabled and if they occur while tracers are active.
+
+A tracing session name may be passed to both the `start` and `stop`
+commands to start/stop tracing a session other than the current one.
+
+
+[[enabling-disabling-channels]]
+==== Enabling and disabling channels
+
+<<event,As mentioned>> in the
+<<understanding-lttng,Understanding LTTng>> chapter, enabled
+events are contained in a specific channel, itself contained in a
+specific tracing session. A channel is a group of events with
+tunable parameters (event loss mode, sub-buffer size, number of
+sub-buffers, trace file sizes and count, to name a few). A given channel
+may only be responsible for enabled events belonging to one domain:
+either kernel or user space.
+
+If you only used the `create`, `enable-event` and `start`/`stop`
+commands of the `lttng` tool so far, one or two channels were
+automatically created for you (one for the kernel domain and/or one
+for the user space domain). The default channels are both named
+`channel0`; channels from different domains may have the same name.
+
+The current channels of a given tracing session can be viewed with
+
+[role="term"]
+----
+lttng list some-session
+----
+
+where `some-session` is the name of the desired tracing session.
+
+To create and enable a channel, use the `enable-channel` command:
+
+[role="term"]
+----
+lttng enable-channel --kernel my-channel
+----
+
+This creates a kernel domain channel named `my-channel` with
+default parameters in the current tracing session.
+
+[NOTE]
+====
+Because of a current limitation, all
+channels must be _created_ prior to beginning tracing in a
+given tracing session, that is before the first time you do
+`lttng start`.
+
+Since a channel is automatically created by
+`enable-event` only for the specified domain, you cannot,
+for example, enable a kernel domain event, start tracing and then
+enable a user space domain event because no user space channel
+exists yet and it's too late to create one.
+
+For this reason, make sure to configure your channels properly
+before starting the tracers for the first time!
+====
+
+Here's another example:
+
+[role="term"]
+----
+lttng enable-channel --userspace --session other-session --overwrite \
+ --tracefile-size 1048576 1mib-channel
+----
+
+This creates a user space domain channel named `1mib-channel` in
+the tracing session named `other-session` that loses new events by
+overwriting previously recorded events (instead of the default mode of
+discarding newer ones) and saves trace files with a maximum size of
+1{nbsp}MiB each.
+
+Note that channels may also be created using the `--channel` option of
+the `enable-event` command when the provided channel name doesn't exist
+for the specified domain:
+
+[role="term"]
+----
+lttng enable-event --kernel --channel some-channel sched_switch
+----
+
+If no kernel domain channel named `some-channel` existed before calling
+the above command, it would be created with default parameters.
+
+You may enable the same event in two different channels:
+
+[role="term"]
+----
+lttng enable-event --userspace --channel my-channel app:tp
+lttng enable-event --userspace --channel other-channel app:tp
+----
+
+If both channels are enabled, the occurring `app:tp` event
+generates two recorded events, one for each channel.
+
+Disabling a channel is done with the `disable-event` command:
+
+[role="term"]
+----
+lttng disable-event --kernel some-channel
+----
+
+The state of a channel precedes the individual states of events within
+it: events belonging to a disabled channel, even if they are
+enabled, won't be recorded.
+
+
+
+[[fine-tuning-channels]]
+===== Fine-tuning channels
+
+There are various parameters that may be fine-tuned with the
+`enable-channel` command. The latter are well documented in
+man:lttng(1) and in the <<channel,Channel>> section of the
+<<understanding-lttng,Understanding LTTng>> chapter. For basic
+tracing needs, their default values should be just fine, but here are a
+few examples to break the ice.
+
+As the frequency of recorded events increases--either because the
+event throughput is actually higher or because you enabled more events
+than usual—__event loss__ might be experienced. Since LTTng never
+waits, by design, for sub-buffer space availability (non-blocking
+tracer), when a sub-buffer is full and no empty sub-buffers are left,
+there are two possible outcomes: either the new events that do not fit
+are rejected, or they start replacing the oldest recorded events.
+The choice of which algorithm to use is a per-channel parameter, the
+default being discarding the newest events until there is some space
+left. If your situation always needs the latest events at the expense
+of writing over the oldest ones, create a channel with the `--overwrite`
+option:
+
+[role="term"]
+----
+lttng enable-channel --kernel --overwrite my-channel
+----
+
+When an event is lost, it means no space was available in any
+sub-buffer to accommodate it. Thus, if you want to cope with sporadic
+high event throughput situations and avoid losing events, you need to
+allocate more room for storing them in memory. This can be done by
+either increasing the size of sub-buffers or by adding sub-buffers.
+The following example creates a user space domain channel with
+16{nbsp}sub-buffers of 512{nbsp}kiB each:
+
+[role="term"]
+----
+lttng enable-channel --userspace --num-subbuf 16 --subbuf-size 512k big-channel
+----
+
+Both values need to be powers of two, otherwise they are rounded up
+to the next one.
+
+Two other interesting available parameters of `enable-channel` are
+`--tracefile-size` and `--tracefile-count`, which respectively limit
+the size of each trace file and the their count for a given channel.
+When the number of written trace files reaches its limit for a given
+channel-CPU pair, the next trace file overwrites the very first
+one. The following example creates a kernel domain channel with a
+maximum of three trace files of 1{nbsp}MiB each:
+
+[role="term"]
+----
+lttng enable-channel --kernel --tracefile-size 1M --tracefile-count 3 my-channel
+----
+
+An efficient way to make sure lots of events are generated is enabling
+all kernel events in this channel and starting the tracer:
+
+[role="term"]
+----
+lttng enable-event --kernel --all --channel my-channel
+lttng start
+----
+
+After a few seconds, look at trace files in your tracing session
+output directory. For two CPUs, it should look like:
+
+----
+my-channel_0_0 my-channel_1_0
+my-channel_0_1 my-channel_1_1
+my-channel_0_2 my-channel_1_2
+----
+
+Amongst the files above, you might see one in each group with a size
+lower than 1{nbsp}MiB: they are the files currently being written.
+
+Since all those small files are valid LTTng trace files, LTTng trace
+viewers may read them. It is the viewer's responsibility to properly
+merge the streams so as to present an ordered list to the user.
+http://diamon.org/babeltrace[Babeltrace]
+merges LTTng trace files correctly and is fast at doing it.
+
+
+[[adding-context]]
+==== Adding some context to channels
+
+If you read all the sections of
+<<controlling-tracing,Controlling tracing>> so far, you should be
+able to create tracing sessions, create and enable channels and events
+within them and start/stop the LTTng tracers. Event fields recorded in
+trace files provide important information about occurring events, but
+sometimes external context may help you solve a problem faster. This
+section discusses how to add context information to events of a
+specific channel using the `lttng` tool.
+
+There are various available context values which can accompany events
+recorded by LTTng, for example:
+
+* **process information**:
+** identifier (PID)
+** name
+** priority
+** scheduling priority (niceness)
+** thread identifier (TID)
+* the **hostname** of the system on which the event occurred
+* plenty of **performance counters** using perf, for example:
+** CPU cycles, stalled cycles, idle cycles, and the other cycle types
+** cache misses
+** branch instructions, misses, loads
+** CPU faults
+
+The full list is available in the output of `lttng add-context --help`.
+Some of them are reserved for a specific domain (kernel or
+user space) while others are available for both.
+
+To add context information to one or all channels of a given tracing
+session, use the `add-context` command:
+
+[role="term"]
+----
+lttng add-context --userspace --type vpid --type perf:thread:cpu-cycles
+----
+
+The above example adds the virtual process identifier and per-thread
+CPU cycles count values to all recorded user space domain events of the
+current tracing session. Use the `--channel` option to select a specific
+channel:
+
+[role="term"]
+----
+lttng add-context --kernel --channel my-channel --type tid
+----
+
+adds the thread identifier value to all recorded kernel domain events
+in the channel `my-channel` of the current tracing session.
+
+Beware that context information cannot be removed from channels once
+it's added for a given tracing session.
+
+
+[role="since-2.5"]
+[[saving-loading-tracing-session]]
+==== Saving and loading tracing session configurations
+
+Configuring a tracing session may be long: creating and enabling
+channels with specific parameters, enabling kernel and user space
+domain events with specific log levels and filters, and adding context
+to some channels are just a few of the many possible operations using
+the `lttng` command line tool. If you're going to use LTTng to solve real
+world problems, chances are you're going to have to record events using
+the same tracing session setup over and over, modifying a few variables
+each time in your instrumented program or environment. To avoid
+constant tracing session reconfiguration, the `lttng` tool is able to
+save and load tracing session configurations to/from XML files.
+
+To save a given tracing session configuration, do:
+
+[role="term"]
+----
+lttng save my-session
+----
+
+where `my-session` is the name of the tracing session to save. Tracing
+session configurations are saved to dir:{~/.lttng/sessions} by default;
+use the `--output-path` option to change this destination directory.
+
+All configuration parameters are saved:
+
+* tracing session name
+* trace data output path
+* channels with their state and all their parameters
+* context information added to channels
+* events with their state, log level and filter
+* tracing activity (started or stopped)
+
+To load a tracing session, simply do:
+
+[role="term"]
+----
+lttng load my-session
+----
+
+or, if you used a custom path:
+
+[role="term"]
+----
+lttng load --input-path /path/to/my-session.lttng
+----
+
+Your saved tracing session is restored as if you just configured
+it manually.
+
+
+[[sending-trace-data-over-the-network]]
+==== Sending trace data over the network
+
+The possibility of sending trace data over the network comes as a
+built-in feature of LTTng-tools. For this to be possible, an LTTng
+_relay daemon_ must be executed and listening on the machine where
+trace data is to be received, and the user must create a tracing
+session using appropriate options to forward trace data to the remote
+relay daemon.
+
+The relay daemon listens on two different TCP ports: one for control
+information and the other for actual trace data.
+
+Starting the relay daemon on the remote machine is easy:
+
+[role="term"]
+----
+lttng-relayd
+----
+
+This makes it listen to its default ports: 5342 for control and
+5343 for trace data. The `--control-port` and `--data-port` options may
+be used to specify different ports.
+
+Traces written by `lttng-relayd` are written to
++\~/lttng-traces/__hostname__/__session__+ by
+default, where +__hostname__+ is the host name of the
+traced (monitored) system and +__session__+ is the
+tracing session name. Use the `--output` option to write trace data
+outside dir:{~/lttng-traces}.
+
+On the sending side, a tracing session must be created using the
+`lttng` tool with the `--set-url` option to connect to the distant
+relay daemon:
+
+[role="term"]
+----
+lttng create my-session --set-url net://distant-host
+----
+
+The URL format is described in the output of `lttng create --help`.
+The above example uses the default ports; the `--ctrl-url` and
+`--data-url` options may be used to set the control and data URLs
+individually.
+
+Once this basic setup is completed and the connection is established,
+you may use the `lttng` tool on the target machine as usual; everything
+you do is transparently forwarded to the remote machine if needed.
+For example, a parameter changing the maximum size of trace files
+only has an effect on the distant relay daemon actually writing
+the trace.
+
+
+[role="since-2.4"]
+[[lttng-live]]
+==== Viewing events as they arrive
+
+We have seen how trace files may be produced by LTTng out of generated
+application and Linux kernel events. We have seen that those trace files
+may be either recorded locally by consumer daemons or remotely using
+a relay daemon. And we have seen that the maximum size and count of
+trace files is configurable for each channel. With all those features,
+it's still not possible to read a trace file as it is being written
+because it could be incomplete and appear corrupted to the viewer.
+There is a way to view events as they arrive, however: using
+_LTTng live_.
+
+LTTng live is implemented, in LTTng, solely on the relay daemon side.
+As trace data is sent over the network to a relay daemon by a (possibly
+remote) consumer daemon, a _tee_ is created: trace data is recorded to
+trace files _as well as_ being transmitted to a connected live viewer:
+
+[role="img-90"]
+.The relay daemon creates a _tee_, forwarding the trace data to both trace files and a live viewer.
+image::lttng-live.png[]
+
+In order to use this feature, a tracing session must created in live
+mode on the target system:
+
+[role="term"]
+----
+lttng create --live
+----
+
+An optional parameter may be passed to `--live` to set the period
+(in microseconds) between flushes to the network
+(1{nbsp}second is the default). With:
+
+[role="term"]
+----
+lttng create --live 100000
+----
+
+the daemons flush their data every 100{nbsp}ms.
+
+If no network output is specified to the `create` command, a local
+relay daemon is spawned. In this very common case, viewing a live
+trace is easy: enable events and start tracing as usual, then use
+`lttng view` to start the default live viewer:
+
+[role="term"]
+----
+lttng view
+----
+
+The correct arguments are passed to the live viewer so that it
+may connect to the local relay daemon and start reading live events.
+
+You may also wish to use a live viewer not running on the target
+system. In this case, you should specify a network output when using
+the `create` command (`--set-url` or `--ctrl-url`/`--data-url` options).
+A distant LTTng relay daemon should also be started to receive control
+and trace data. By default, `lttng-relayd` listens on 127.0.0.1:5344
+for an LTTng live connection. Otherwise, the desired URL may be
+specified using its `--live-port` option.
+
+The
+http://diamon.org/babeltrace[`babeltrace`]
+viewer supports LTTng live as one of its input formats. `babeltrace` is
+the default viewer when using `lttng view`. To use it manually, first
+list active tracing sessions by doing the following (assuming the relay
+daemon to connect to runs on the same host):
+
+[role="term"]
+----
+babeltrace --input-format lttng-live net://localhost
+----
+
+Then, choose a tracing session and start viewing events as they arrive
+using LTTng live:
+
+[role="term"]
+----
+babeltrace --input-format lttng-live net://localhost/host/hostname/my-session
+----
+
+
+[role="since-2.3"]
+[[taking-a-snapshot]]
+==== Taking a snapshot
+
+The normal behavior of LTTng is to record trace data as trace files.
+This is ideal for keeping a long history of events that occurred on
+the target system and applications, but may be too much data in some
+situations. For example, you may wish to trace your application
+continuously until some critical situation happens, in which case you
+would only need the latest few recorded events to perform the desired
+analysis, not multi-gigabyte trace files.
+
+LTTng has an interesting feature called _snapshots_. When creating
+a tracing session in snapshot mode, no trace files are written; the
+tracers' sub-buffers are constantly overwriting the oldest recorded
+events with the newest. At any time, either when the tracers are started
+or stopped, you may take a snapshot of those sub-buffers.
+
+There is no difference between the format of a normal trace file and the
+format of a snapshot: viewers of LTTng traces also support LTTng
+snapshots. By default, snapshots are written to disk, but they may also
+be sent over the network.
+
+To create a tracing session in snapshot mode, do:
+
+[role="term"]
+----
+lttng create --snapshot my-snapshot-session
+----
+
+Next, enable channels, events and add context to channels as usual.
+Once a tracing session is created in snapshot mode, channels are
+forced to use the
+<<channel-overwrite-mode-vs-discard-mode,overwrite>> mode
+(`--overwrite` option of the `enable-channel` command; also called
+_flight recorder mode_) and have an `mmap()` channel type
+(`--output mmap`).
+
+Start tracing. When you're ready to take a snapshot, do:
+
+[role="term"]
+----
+lttng snapshot record --name my-snapshot
+----
+
+This records a snapshot named `my-snapshot` of all channels of
+all domains of the current tracing session. By default, snapshots files
+are recorded in the path returned by `lttng snapshot list-output`. You
+may change this path or decide to send snapshots over the network
+using either:
+
+. an output path/URL specified when creating the tracing session
+ (`lttng create`)
+. an added snapshot output path/URL using
+ `lttng snapshot add-output`
+. an output path/URL provided directly to the
+ `lttng snapshot record` command
+
+Method 3 overrides method 2 which overrides method 1. When specifying
+a URL, a relay daemon must be listening on some machine (see
+<<sending-trace-data-over-the-network,Sending trace data over the network>>).
+
+If you need to make absolutely sure that the output file won't be
+larger than a certain limit, you can set a maximum snapshot size when
+taking it with the `--max-size` option:
+
+[role="term"]
+----
+lttng snapshot record --name my-snapshot --max-size 2M
+----
+
+Older recorded events are discarded in order to respect this
+maximum size.
+
+
+[role="since-2.6"]
+[[mi]]
+==== Machine interface
+
+The `lttng` tool aims at providing a command output as human-readable as
+possible. While this output is easy to parse by a human being, machines
+have a hard time.
+
+This is why the `lttng` tool provides the general `--mi` option, which
+must specify a machine interface output format. As of the latest
+LTTng stable release, only the `xml` format is supported. A schema
+definition (XSD) is made
+https://github.com/lttng/lttng-tools/blob/master/src/common/mi_lttng.xsd[available]
+to ease the integration with external tools as much as possible.
+
+The `--mi` option can be used in conjunction with all `lttng` commands.
+Here are some examples:
+
+[role="term"]
+----
+lttng --mi xml create some-session
+lttng --mi xml list some-session
+lttng --mi xml list --kernel
+lttng --mi xml enable-event --kernel --syscall open
+lttng --mi xml start
+----
+
+
+[[reference]]
+== Reference
+
+This chapter presents various references for LTTng packages such as links
+to online manpages, tables needed by the rest of the text, descriptions
+of library functions, and more.
+
+
+[[online-lttng-manpages]]
+=== Online LTTng manpages
+
+LTTng packages currently install the following link:/man[man pages],
+available online using the links below:
+
+* **LTTng-tools**
+** man:lttng(1)
+** man:lttng-sessiond(8)
+** man:lttng-relayd(8)
+* **LTTng-UST**
+** man:lttng-gen-tp(1)
+** man:lttng-ust(3)
+** man:lttng-ust-cyg-profile(3)
+** man:lttng-ust-dl(3)
+
+
+[[lttng-ust-ref]]
+=== LTTng-UST
+
+This section presents references of the LTTng-UST package.
+
+
+[[liblttng-ust]]
+==== LTTng-UST library (+liblttng‑ust+)
+
+The LTTng-UST library, or `liblttng-ust`, is the main shared object
+against which user applications are linked to make LTTng user space
+tracing possible.
+
+The <<c-application,C application>> guide shows the complete
+process to instrument, build and run a C/$$C++$$ application using
+LTTng-UST, while this section contains a few important tables.
+
+
+[[liblttng-ust-tp-fields]]
+===== Tracepoint fields macros (for `TP_FIELDS()`)
+
+The available macros to define tracepoint fields, which should be listed
+within `TP_FIELDS()` in `TRACEPOINT_EVENT()`, are:
+
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+.Available macros to define LTTng-UST tracepoint fields
+|====
+|Macro |Description and parameters
+
+|
++ctf_integer(__t__, __n__, __e__)+
+
++ctf_integer_nowrite(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `long`, `size_t`, ...).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_hex(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network(__t__, __n__, __e__)+
+|
+Integer in network byte order (big endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network_hex(__t__, __n__, __e__)+
+|
+Integer in network byte order, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_float(__t__, __n__, __e__)+
+
++ctf_float_nowrite(__t__, __n__, __e__)+
+|
+Floating point number.
+
++__t__+::
+ Floating point number C type (`float` or `double`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_string(__n__, __e__)+
+
++ctf_string_nowrite(__n__, __e__)+
+|
+Null-terminated string; undefined behavior if +__e__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_array(__t__, __n__, __e__, __s__)+
+
++ctf_array_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array of integers
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_array_text(__t__, __n__, __e__, __s__)+
+
++ctf_array_text_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array, printed as text.
+
+The string does not need to be null-terminated.
+
++__t__+::
+ Array element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_sequence(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers.
+
+The type of +__E__+ needs to be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|
++ctf_sequence_text(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array, displayed as text.
+
+The string does not need to be null-terminated.
+
+The type of +__E__+ needs to be unsigned.
+
+The behaviour is undefined if +__e__+ is `NULL`.
+
++__t__+::
+ Sequence element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+|====
+
+The `_nowrite` versions omit themselves from the session trace, but are
+otherwise identical. This means the `_nowrite` fields won't be written
+in the recorded trace. Their primary purpose is to make some
+of the event context available to the
+<<enabling-disabling-events,event filters>> without having to
+commit the data to sub-buffers.
+
+
+[[liblttng-ust-tracepoint-loglevel]]
+===== Tracepoint log levels (for `TRACEPOINT_LOGLEVEL()`)
+
+The following table shows the available log level values for the
+`TRACEPOINT_LOGLEVEL()` macro:
+
+`TRACE_EMERG`::
+ System is unusable.
+
+`TRACE_ALERT`::
+ Action must be taken immediately.
+
+`TRACE_CRIT`::
+ Critical conditions.
+
+`TRACE_ERR`::
+ Error conditions.
+
+`TRACE_WARNING`::
+ Warning conditions.
+
+`TRACE_NOTICE`::
+ Normal, but significant, condition.
+
+`TRACE_INFO`::
+ Informational message.
+
+`TRACE_DEBUG_SYSTEM`::
+ Debug information with system-level scope (set of programs).
+
+`TRACE_DEBUG_PROGRAM`::
+ Debug information with program-level scope (set of processes).
+
+`TRACE_DEBUG_PROCESS`::
+ Debug information with process-level scope (set of modules).
+
+`TRACE_DEBUG_MODULE`::
+ Debug information with module (executable/library) scope (set of units).
+
+`TRACE_DEBUG_UNIT`::
+ Debug information with compilation unit scope (set of functions).
+
+`TRACE_DEBUG_FUNCTION`::
+ Debug information with function-level scope.
+
+`TRACE_DEBUG_LINE`::
+ Debug information with line-level scope (TRACEPOINT_EVENT default).
+
+`TRACE_DEBUG`::
+ Debug-level message.
+
+Log levels `TRACE_EMERG` through `TRACE_INFO` and `TRACE_DEBUG` match
+http://man7.org/linux/man-pages/man3/syslog.3.html[syslog]
+level semantics. Log levels `TRACE_DEBUG_SYSTEM` through `TRACE_DEBUG`
+offer more fine-grained selection of debug information.
+
+
+[[lttng-modules-ref]]
+=== LTTng-modules
+
+This section presents references of the LTTng-modules package.
+
+
+[[lttng-modules-tp-struct-entry]]
+==== Tracepoint fields macros (for `TP_STRUCT__entry()`)
+
+This table describes possible entries for the `TP_STRUCT__entry()` part
+of `LTTNG_TRACEPOINT_EVENT()`:
+
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+.Available entries for `TP_STRUCT__entry()` (in `LTTNG_TRACEPOINT_EVENT()`)
+|====
+|Macro |Description and parameters
+
+|+\__field(__t__, __n__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `unsigned char`, `size_t`, ...).
+
++__n__+::
+ Field name.
+
+|+\__field_hex(__t__, __n__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_oct(__t__, __n__)+
+|
+Standard integer, displayed in base 8.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_network(__t__, __n__)+
+|
+Integer in network byte order (big endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__field_network_hex(__t__, __n__)+
+|
+Integer in network byte order (big endian), displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
+|+\__array(__t__, __n__, __s__)+
+|
+Statically-sized array, elements displayed in base 10.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Number of elements.
+
+|+\__array_hex(__t__, __n__, __s__)+
+|
+Statically-sized array, elements displayed in base 16.
+
++__t__+::
+ array element C type.
++__n__+::
+ field name.
++__s__+::
+ number of elements.
+
+|+\__array_text(__t__, __n__, __s__)+
+|
+Statically-sized array, displayed as text.
+
++__t__+::
+ Array element C type (always char).
+
++__n__+::
+ Field name.
+
++__s__+::
+ Number of elements.
+
+|+\__dynamic_array(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed in base 10.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__dynamic_array_hex(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed in base 16.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__dynamic_array_text(__t__, __n__, __s__)+
+|
+Dynamically-sized array, displayed as text.
+
++__t__+::
+ Array element C type (always char).
+
++__n__+::
+ Field name.
+
++__s__+::
+ Length C expression.
+
+|+\__string(n, __s__)+
+|
+Null-terminated string.
+
+The behaviour is undefined behavior if +__s__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__s__+::
+ String source (pointer).
+|====
+
+The above macros should cover the majority of cases. For advanced items,
+see path:{probes/lttng-events.h}.
+
+
+[[lttng-modules-tp-fast-assign]]
+==== Tracepoint assignment macros (for `TP_fast_assign()`)
+
+This table describes possible entries for the `TP_fast_assign()` part
+of `LTTNG_TRACEPOINT_EVENT()`:
+
+[role="growable func-desc",cols="asciidoc,asciidoc"]
+.Available entries for `TP_fast_assign()` (in `LTTNG_TRACEPOINT_EVENT()`)
+|====
+|Macro |Description and parameters
+
+|+tp_assign(__d__, __s__)+
+|
+Assignment of C expression +__s__+ to tracepoint field +__d__+.
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
+|+tp_memcpy(__d__, __s__, __l__)+
+|
+Memory copy of +__l__+ bytes from +__s__+ to tracepoint field
++__d__+ (use with array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_memcpy_from_user(__d__, __s__, __l__)+
+|
+Memory copy of +__l__+ bytes from user space +__s__+ to tracepoint
+field +__d__+ (use with array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_memcpy_dyn(__d__, __s__)+
+|
+Memory copy of dynamically-sized array from +__s__+ to tracepoint field
++__d__+.
+
+The number of bytes is known from the field's length expression
+(use with dynamically-sized array fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+
++__l__+::
+ Number of bytes to copy.
+
+|+tp_strcpy(__d__, __s__)+
+|
+String copy of +__s__+ to tracepoint field +__d__+ (use with string
+fields).
+
++__d__+::
+ Name of destination tracepoint field.
+
++__s__+::
+ Source C expression (may refer to tracepoint arguments).
+|====
--- /dev/null
+The LTTng Documentation
+=======================
+Philippe Proulx <pproulx@efficios.com>
+v2.7, 25 October 2016
+
+
+include::../common/copyright.txt[]
+
+
+include::../common/welcome.txt[]
+
+
+include::../common/audience.txt[]
+
+
+[[chapters]]
+=== What's in this documentation?
+
+The LTTng Documentation is divided into the following sections:
+
+* **<<nuts-and-bolts,Nuts and bolts>>** explains the
+ rudiments of software tracing and the rationale behind the
+ LTTng project.
++
+You can skip this section if you’re familiar with software tracing and
+with the LTTng project.
+
+* **<<installing-lttng,Installation>>** describes the steps to
+ install the LTTng packages on common Linux distributions and from
+ their sources.
++
+You can skip this section if you already properly installed LTTng on
+your target system.
+
+* **<<getting-started,Quick start>>** is a concise guide to
+ getting started quickly with LTTng kernel and user space tracing.
++
+We recommend this section if you're new to LTTng or to software tracing
+in general.
++
+You can skip this section if you're not new to LTTng.
+
+* **<<core-concepts,Core concepts>>** explains the concepts at
+ the heart of LTTng.
++
+It's a good idea to become familiar with the core concepts
+before attempting to use the toolkit.
+
+* **<<plumbing,Components of LTTng>>** describes the various components
+ of the LTTng machinery, like the daemons, the libraries, and the
+ command-line interface.
+* **<<instrumenting,Instrumentation>>** shows different ways to
+ instrument user applications and the Linux kernel.
++
+Instrumenting source code is essential to provide a meaningful
+source of events.
++
+You can skip this section if you do not have a programming background.
+
+* **<<controlling-tracing,Tracing control>>** is divided into topics
+ which demonstrate how to use the vast array of features that
+ LTTng{nbsp}{revision} offers.
+* **<<reference,Reference>>** contains reference links and tables.
+* **<<glossary,Glossary>>** is a specialized dictionary of terms related
+ to LTTng or to the field of software tracing.
+
+
+include::../common/convention.txt[]
+
+
+include::../common/acknowledgements.txt[]
+
+
+[[whats-new]]
+== What's new in LTTng {revision}?
+
+* **Tracing control**:
+** Dynamic filter support for <<event,event rules>> in the Linux kernel
+ <<domain,tracing domain>>. For example:
++
+--
+[role="term"]
+----
+lttng enable-event --kernel irq_handler_entry --filter='irq == 28'
+----
+--
+
+** Wildcard support in the instrumentation point name of an event rule
+ in the Linux kernel tracing domain. For example:
++
+--
+[role="term"]
+----
+lttng enable-event --kernel 'sched_*'
+----
+--
+
+** New `lttng track` and `lttng untrack` commands to make
+ <<pid-tracking,PID tracking>> super-fast for both the Linux kernel
+ and the user space tracing domains.
++
+When LTTng _tracks_ one or more PIDs, only the processes having those PIDs
+can emit events for a given tracing session.
+
+** New `--shm-path` option of the `lttng create` command to specify the
+ path where LTTng creates the shared memory holding the ring buffers.
++
+This feature is useful when used with persistent memory file systems to
+extract the latest recorded trace data in the event of a crash requiring
+a reboot.
++
+The new man:lttng-crash(1) command-line utility can extract trace data
+from such a file (see <<persistent-memory-file-systems,Record trace data
+on persistent memory file systems>>).
+
+* **User space tracing**:
+** New <<python-application,LTTng-UST Python agent>> which makes it easy
+ to trace existing Python applications that are using the standard
+ https://docs.python.org/3/howto/logging.html[`logging` package].
++
+This agent is compatible with both the Python 2 and Python 3 languages.
+
+** New <<tracelog,`tracelog()`>> facility to ease the migration from
+ logging to tracing.
++
+`tracelog()` is similar to <<tracef,`tracef()`>>,
+but it accepts an additional log level parameter.
+
+** Plugin support in LTTng-UST to provide a custom clock source and to
+ retrieve the current CPU number.
++
+This feature exists for very advanced use cases.
++
+See the
+https://github.com/lttng/lttng-ust/tree/stable-{revision}/doc/examples/clock-override[clock-override]
+and
+https://github.com/lttng/lttng-ust/tree/stable-{revision}/doc/examples/getcpu-override[getcpu-override]
+examples for more details.
+
+Moreover, LTTng{nbsp}{revision} boasts great stability, benifiting from
+piles of bug fixes and more-than-welcome internal refactorings.
+
+To learn more about the new features of LTTng{nbsp}{revision}, see
+https://lttng.org/blog/2015/10/14/lttng-2.7-released/[the release announcement].
+
+
+[[nuts-and-bolts]]
+== Nuts and bolts
+
+What is LTTng? As its name suggests, the _Linux Trace Toolkit: next
+generation_ is a modern toolkit for tracing Linux systems and
+applications. So your first question might be:
+**what is tracing?**
+
+
+[[what-is-tracing]]
+=== What is tracing?
+
+As the history of software engineering progressed and led to what
+we now take for granted--complex, numerous and
+interdependent software applications running in parallel on
+sophisticated operating systems like Linux--the authors of such
+components, software developers, began feeling a natural
+urge to have tools that would ensure the robustness and good performance
+of their masterpieces.
+
+One major achievement in this field is, inarguably, the
+https://www.gnu.org/software/gdb/[GNU debugger (GDB)],
+an essential tool for developers to find and fix bugs. But even the best
+debugger won't help make your software run faster, and nowadays, faster
+software means either more work done by the same hardware, or cheaper
+hardware for the same work.
+
+A _profiler_ is often the tool of choice to identify performance
+bottlenecks. Profiling is suitable to identify _where_ performance is
+lost in a given software. The profiler outputs a profile, a statistical
+summary of observed events, which you may use to discover which
+functions took the most time to execute. However, a profiler won't
+report _why_ some identified functions are the bottleneck. Bottlenecks
+might only occur when specific conditions are met, conditions that are
+sometimes impossible to capture by a statistical profiler, or impossible
+to reproduce with an application altered by the overhead of an
+event-based profiler. For a thorough investigation of software
+performance issues, a history of execution is essential, with the
+recorded values of variables and context fields you choose, and
+with as little influence as possible on the instrumented software. This
+is where tracing comes in handy.
+
+_Tracing_ is a technique used to understand what goes on in a running
+software system. The software used for tracing is called a _tracer_,
+which is conceptually similar to a tape recorder. When recording,
+specific instrumentation points placed in the software source code
+generate events that are saved on a giant tape: a _trace_ file. You
+can trace user applications and the operating system at the same time,
+opening the possibility of resolving a wide range of problems that would
+otherwise be extremely challenging.
+
+Tracing is often compared to _logging_. However, tracers and loggers are
+two different tools, serving two different purposes. Tracers are
+designed to record much lower-level events that occur much more
+frequently than log messages, often in the range of thousands per
+second, with very little execution overhead. Logging is more appropriate
+for a very high-level analysis of less frequent events: user accesses,
+exceptional conditions (errors and warnings, for example), database
+transactions, instant messaging communications, and such. Simply put,
+logging is one of the many use cases that can be satisfied with tracing.
+
+The list of recorded events inside a trace file can be read manually
+like a log file for the maximum level of detail, but it is generally
+much more interesting to perform application-specific analyses to
+produce reduced statistics and graphs that are useful to resolve a
+given problem. Trace viewers and analyzers are specialized tools
+designed to do this.
+
+In the end, this is what LTTng is: a powerful, open source set of
+tools to trace the Linux kernel and user applications at the same time.
+LTTng is composed of several components actively maintained and
+developed by its link:/community/#where[community].
+
+
+[[lttng-alternatives]]
+=== Alternatives to noch:{LTTng}
+
+Excluding proprietary solutions, a few competing software tracers
+exist for Linux:
+
+* https://github.com/dtrace4linux/linux[dtrace4linux] is a port of
+ Sun Microsystems's DTrace to Linux. The cmd:dtrace tool interprets
+ user scripts and is responsible for loading code into the
+ Linux kernel for further execution and collecting the outputted data.
+* https://en.wikipedia.org/wiki/Berkeley_Packet_Filter[eBPF] is a
+ subsystem in the Linux kernel in which a virtual machine can execute
+ programs passed from the user space to the kernel. You can attach
+ such programs to tracepoints and KProbes thanks to a system call, and
+ they can output data to the user space when executed thanks to
+ different mechanisms (pipe, VM register values, and eBPF maps, to name
+ a few).
+* https://www.kernel.org/doc/Documentation/trace/ftrace.txt[ftrace]
+ is the de facto function tracer of the Linux kernel. Its user
+ interface is a set of special files in sysfs.
+* https://perf.wiki.kernel.org/[perf] is
+ a performance analyzing tool for Linux which supports hardware
+ performance counters, tracepoints, as well as other counters and
+ types of probes. perf's controlling utility is the cmd:perf command
+ line/curses tool.
+* http://linux.die.net/man/1/strace[strace]
+ is a command-line utility which records system calls made by a
+ user process, as well as signal deliveries and changes of process
+ state. strace makes use of https://en.wikipedia.org/wiki/Ptrace[ptrace]
+ to fulfill its function.
+* http://www.sysdig.org/[sysdig], like SystemTap, uses scripts to
+ analyze Linux kernel events. You write scripts, or _chisels_ in
+ sysdig's jargon, in Lua and sysdig executes them while the system is
+ being traced or afterwards. sysdig's interface is the cmd:sysdig
+ command-line tool as well as the curses-based cmd:csysdig tool.
+* https://sourceware.org/systemtap/[SystemTap] is a Linux kernel and
+ user space tracer which uses custom user scripts to produce plain text
+ traces. SystemTap converts the scripts to the C language, and then
+ compiles them as Linux kernel modules which are loaded to produce
+ trace data. SystemTap's primary user interface is the cmd:stap
+ command-line tool.
+
+The main distinctive features of LTTng is that it produces correlated
+kernel and user space traces, as well as doing so with the lowest
+overhead amongst other solutions. It produces trace files in the
+http://diamon.org/ctf[CTF] format, a file format optimized
+for the production and analyses of multi-gigabyte data.
+
+LTTng is the result of more than 10 years of active open source
+development by a community of passionate developers.
+LTTng{nbsp}{revision} is currently available on major desktop and server
+Linux distributions.
+
+The main interface for tracing control is a single command-line tool
+named cmd:lttng. The latter can create several tracing sessions, enable
+and disable events on the fly, filter events efficiently with custom
+user expressions, start and stop tracing, and much more. LTTng can
+record the traces on the file system or send them over the network, and
+keep them totally or partially. You can view the traces once tracing
+becomes inactive or in real-time.
+
+<<installing-lttng,Install LTTng now>> and
+<<getting-started,start tracing>>!
+
+
+[[installing-lttng]]
+== Installation
+
+**LTTng** is a set of software <<plumbing,components>> which interact to
+<<instrumenting,instrument>> the Linux kernel and user applications, and
+to <<controlling-tracing,control tracing>> (start and stop
+tracing, enable and disable event rules, and the rest). Those
+components are bundled into the following packages:
+
+* **LTTng-tools**: Libraries and command-line interface to
+ control tracing.
+* **LTTng-modules**: Linux kernel modules to instrument and
+ trace the kernel.
+* **LTTng-UST**: Libraries and Java/Python packages to instrument and
+ trace user applications.
+
+Most distributions mark the LTTng-modules and LTTng-UST packages as
+optional when installing LTTng-tools (which is always required). In the
+following sections, we always provide the steps to install all three,
+but note that:
+
+* You only need to install LTTng-modules if you intend to trace the
+ Linux kernel.
+* You only need to install LTTng-UST if you intend to trace user
+ applications.
+
+[role="growable"]
+.Availability of LTTng{nbsp}{revision} for major Linux distributions.
+|====
+|Distribution |Available in releases |Alternatives
+
+|Ubuntu
+|<<ubuntu,Ubuntu{nbsp}16.04 _Xenial Xerus_>>
+|LTTng{nbsp}2.8 for Ubuntu{nbsp}16.10 _Yakkety Yak_.
+
+LTTng{nbsp}{revision} for Ubuntu{nbsp}12.04 _Precise Pangolin_,
+Ubuntu{nbsp}14.04 _Trusty Tahr_, and Ubuntu{nbsp}16.04 _Xenial Xerus_:
+<<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Ubuntu releases.
+
+|Fedora
+|_Not available_
+|LTTng{nbsp}{revision} for Fedora{nbsp}25 and Fedora{nbsp}26 (not
+released yet).
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Fedora releases.
+
+|Debian
+|_Not available_
+|LTTng{nbsp}2.8 for Debian "stretch" (testing).
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Debian releases.
+
+|openSUSE
+|<<opensuse,openSUSE Leap{nbsp}42.1>>
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other openSUSE releases.
+
+|Arch Linux
+|_Not available_
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>>.
+
+|Alpine Linux
+|_Not available_
+|LTTng{nbsp}2.8 for Alpine Linux "edge".
+
+LTTng{nbsp}2.8 for Alpine Linux{nbsp}3.5 (not released yet).
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Alpine Linux releases.
+
+|RHEL and SLES
+|See http://packages.efficios.com/[EfficiOS Enterprise Packages].
+|
+
+|Buildroot
+|<<"buildroot","Buildroot{nbsp}2016.02, Buildroot{nbsp}2016.05,
+and Buildroot{nbsp}2016.08">>
+|LTTng{nbsp}2.8 for Buildroot{nbsp}2016.11 (not released yet).
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Buildroot releases.
+
+|OpenEmbedded and Yocto
+|<<oe-yocto,`openembedded-core` layer from 1{nbsp}December 2016 until
+3{nbsp}September 2016>>
+|LTTng{nbsp}2.8 for OpenEmbedded since 3{nbsp}September 2016.
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other OpenEmbedded releases.
+|====
+
+
+[[ubuntu]]
+=== [[ubuntu-official-repositories]]Ubuntu
+
+LTTng{nbsp}{revision} is available on Ubuntu 16.04 _Xenial Xerus_. For
+previous releases of Ubuntu, <<ubuntu-ppa,use the LTTng
+Stable{nbsp}{revision} PPA>>.
+
+To install LTTng{nbsp}{revision} on Ubuntu{nbsp}16.04 _Xenial Xerus_:
+
+. Install the main LTTng{nbsp}{revision} packages:
++
+--
+[role="term"]
+----
+sudo apt-get install lttng-tools
+sudo apt-get install lttng-modules-dkms
+sudo apt-get install liblttng-ust-dev
+----
+--
+
+. **If you need to instrument and trace
+ <<java-application,Java applications>>**, install the LTTng-UST
+ Java agent:
++
+--
+[role="term"]
+----
+sudo apt-get install liblttng-ust-agent-java
+----
+--
+
+. **If you need to instrument and trace
+ <<python-application,Python applications>>**, install the
+ LTTng-UST Python agent:
++
+--
+[role="term"]
+----
+sudo apt-get install python3-lttngust
+----
+--
+
+
+[[ubuntu-ppa]]
+==== noch:{LTTng} Stable {revision} PPA
+
+The
+https://launchpad.net/~lttng/+archive/ubuntu/stable-{revision}[LTTng Stable{nbsp}{revision} PPA]
+offers the latest stable LTTng{nbsp}{revision} packages for:
+
+* Ubuntu{nbsp}12.04 _Precise Pangolin_
+* Ubuntu{nbsp}14.04 _Trusty Tahr_
+* Ubuntu{nbsp}16.04 _Xenial Xerus_
+
+To install LTTng{nbsp}{revision} from the LTTng Stable{nbsp}{revision}
+PPA:
+
+. Add the LTTng Stable{nbsp}{revision} PPA repository and update the
+ list of packages:
++
+--
+[role="term"]
+----
+sudo apt-add-repository ppa:lttng/stable-2.7
+sudo apt-get update
+----
+--
+
+. Install the main LTTng{nbsp}{revision} packages:
++
+--
+[role="term"]
+----
+sudo apt-get install lttng-tools
+sudo apt-get install lttng-modules-dkms
+sudo apt-get install liblttng-ust-dev
+----
+--
+
+. **If you need to instrument and trace
+ <<java-application,Java applications>>**, install the LTTng-UST
+ Java agent:
++
+--
+[role="term"]
+----
+sudo apt-get install liblttng-ust-agent-java
+----
+--
+
+. **If you need to instrument and trace
+ <<python-application,Python applications>>**, install the
+ LTTng-UST Python agent:
++
+--
+[role="term"]
+----
+sudo apt-get install python3-lttngust
+----
+--
+
+
+[[opensuse]]
+=== noch:{openSUSE}/RPM
+
+To install LTTng{nbsp}{revision} on openSUSE Leap{nbsp}42.1:
+
+* Install the main LTTng{nbsp}{revision} packages:
++
+--
+[role="term"]
+----
+sudo zypper install lttng-tools
+sudo zypper install lttng-modules
+sudo zypper install lttng-ust-devel
+----
+--
+
+[IMPORTANT]
+.Java and Python application instrumentation and tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
+the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options to the `configure` script, depending
+on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} from source and pass the
+`--enable-python-agent` option to the `configure` script.
+====
+
+
+[[buildroot]]
+=== Buildroot
+
+To install LTTng{nbsp}{revision} on Buildroot{nbsp}2016.02,
+Buildroot{nbsp}2016.05, or Buildroot{nbsp}2016.08:
+
+. Launch the Buildroot configuration tool:
++
+--
+[role="term"]
+----
+make menuconfig
+----
+--
+
+. In **Kernel**, check **Linux kernel**.
+. In **Toolchain**, check **Enable WCHAR support**.
+. In **Target packages**{nbsp}→ **Debugging, profiling and benchmark**,
+ check **lttng-modules** and **lttng-tools**.
+. In **Target packages**{nbsp}→ **Libraries**{nbsp}→
+ **Other**, check **lttng-libust**.
+
+
+[[oe-yocto]]
+=== OpenEmbedded and Yocto
+
+LTTng{nbsp}{revision} recipes are available in the
+http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/[`openembedded-core`]
+layer of OpenEmbedded since 1{nbsp}December 2016 until
+3{nbsp}September 2016 under the following names:
+
+* `lttng-tools`
+* `lttng-modules`
+* `lttng-ust`
+
+With BitBake, the simplest way to include LTTng recipes in your target
+image is to add them to `IMAGE_INSTALL_append` in path:{conf/local.conf}:
+
+----
+IMAGE_INSTALL_append = " lttng-tools lttng-modules lttng-ust"
+----
+
+If you use Hob:
+
+. Select a machine and an image recipe.
+. Click **Edit image recipe**.
+. Under the **All recipes** tab, search for **lttng**.
+. Check the desired LTTng recipes.
+
+[IMPORTANT]
+.Java and Python application instrumentation and tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
+the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options to the `configure` script, depending
+on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} from source and pass the
+`--enable-python-agent` option to the `configure` script.
+====
+
+
+[[enterprise-distributions]]
+=== RHEL, SUSE, and other enterprise distributions
+
+To install LTTng on enterprise Linux distributions, such as Red Hat
+Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SUSE), please
+see http://packages.efficios.com/[EfficiOS Enterprise Packages].
+
+
+[[building-from-source]]
+=== Build from source
+
+To build and install LTTng{nbsp}{revision} from source:
+
+. Using your distribution's package manager, or from source, install
+ the following dependencies of LTTng-tools and LTTng-UST:
++
+--
+* https://sourceforge.net/projects/libuuid/[libuuid]
+* http://directory.fsf.org/wiki/Popt[popt]
+* http://liburcu.org/[Userspace RCU]
+* http://www.xmlsoft.org/[libxml2]
+--
+
+. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.7.tar.bz2 &&
+tar -xf lttng-modules-latest-2.7.tar.bz2 &&
+cd lttng-modules-2.7.* &&
+make &&
+sudo make modules_install &&
+sudo depmod -a
+----
+--
+
+. Download, build, and install the latest LTTng-UST{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.7.tar.bz2 &&
+tar -xf lttng-ust-latest-2.7.tar.bz2 &&
+cd lttng-ust-2.7.* &&
+./configure &&
+make &&
+sudo make install &&
+sudo ldconfig
+----
+--
++
+--
+[IMPORTANT]
+.Java and Python application tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>>, pass the `--enable-java-agent-jul`,
+`--enable-java-agent-log4j`, or `--enable-java-agent-all` options to the
+`configure` script, depending on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>>, pass the `--enable-python-agent` option to the
+`configure` script. You can set the `PYTHON` environment variable to the
+path to the Python interpreter for which to install the LTTng-UST Python
+agent package.
+====
+--
++
+--
+[NOTE]
+====
+By default, LTTng-UST libraries are installed to
+dir:{/usr/local/lib}, which is the de facto directory in which to
+keep self-compiled and third-party libraries.
+
+When <<building-tracepoint-providers-and-user-application,linking an
+instrumented user application with `liblttng-ust`>>:
+
+* Append `/usr/local/lib` to the env:LD_LIBRARY_PATH environment
+ variable.
+* Pass the `-L/usr/local/lib` and `-Wl,-rpath,/usr/local/lib` options to
+ man:gcc(1), man:g++(1), or man:clang(1).
+====
+--
+
+. Download, build, and install the latest LTTng-tools{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.7.tar.bz2 &&
+tar -xf lttng-tools-latest-2.7.tar.bz2 &&
+cd lttng-tools-2.7.* &&
+./configure &&
+make &&
+sudo make install &&
+sudo ldconfig
+----
+--
+
+TIP: The https://github.com/eepp/vlttng[vlttng tool] can do all the
+previous steps automatically for a given version of LTTng and confine
+the installed files in a specific directory. This can be useful to test
+LTTng without installing it on your system.
+
+
+[[getting-started]]
+== Quick start
+
+This is a short guide to get started quickly with LTTng kernel and user
+space tracing.
+
+Before you follow this guide, make sure to <<installing-lttng,install>>
+LTTng.
+
+This tutorial walks you through the steps to:
+
+. <<tracing-the-linux-kernel,Trace the Linux kernel>>.
+. <<tracing-your-own-user-application,Trace a user application>> written
+ in C.
+. <<viewing-and-analyzing-your-traces,View and analyze the
+ recorded events>>.
+
+
+[[tracing-the-linux-kernel]]
+=== Trace the Linux kernel
+
+The following command lines start with cmd:sudo because you need root
+privileges to trace the Linux kernel. You can avoid using cmd:sudo if
+your Unix user is a member of the <<lttng-sessiond,tracing group>>.
+
+. Create a <<tracing-session,tracing session>>:
++
+--
+[role="term"]
+----
+sudo lttng create my-kernel-session
+----
+--
+
+. List the available kernel tracepoints and system calls:
++
+--
+[role="term"]
+----
+lttng list --kernel
+----
+--
+
+. Create an <<event,event rule>> which matches the desired event names,
+ for example `sched_switch` and `sched_process_fork`:
++
+--
+[role="term"]
+----
+sudo lttng enable-event --kernel sched_switch,sched_process_fork
+----
+--
++
+You can also create an event rule which _matches_ all the Linux kernel
+tracepoints (this will generate a lot of data when tracing):
++
+--
+[role="term"]
+----
+sudo lttng enable-event --kernel --all
+----
+--
+
+. Start tracing:
++
+--
+[role="term"]
+----
+sudo lttng start
+----
+--
+
+. Do some operation on your system for a few seconds. For example,
+ load a website, or list the files of a directory.
+. Stop tracing and destroy the tracing session:
++
+--
+[role="term"]
+----
+sudo lttng stop
+sudo lttng destroy
+----
+--
++
+The `destroy` command does not destroy the trace data; it only destroys
+the state of the tracing session.
+
+By default, LTTng saves the traces in
++$LTTNG_HOME/lttng-traces/__name__-__date__-__time__+,
+where +__name__+ is the tracing session name. Note that the
+env:LTTNG_HOME environment variable defaults to `$HOME` if not set.
+
+See <<viewing-and-analyzing-your-traces,View and analyze the
+recorded events>> to view the recorded events.
+
+
+[[tracing-your-own-user-application]]
+=== Trace a user application
+
+This section steps you through a simple example to trace a
+_Hello world_ program written in C.
+
+To create the traceable user application:
+
+. Create the tracepoint provider header file, which defines the
+ tracepoints and the events they can generate:
++
+--
+[source,c]
+.path:{hello-tp.h}
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER hello_world
+
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./hello-tp.h"
+
+#if !defined(_HELLO_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _HELLO_TP_H
+
+#include <lttng/tracepoint.h>
+
+TRACEPOINT_EVENT(
+ hello_world,
+ my_first_tracepoint,
+ TP_ARGS(
+ int, my_integer_arg,
+ char*, my_string_arg
+ ),
+ TP_FIELDS(
+ ctf_string(my_string_field, my_string_arg)
+ ctf_integer(int, my_integer_field, my_integer_arg)
+ )
+)
+
+#endif /* _HELLO_TP_H */
+
+#include <lttng/tracepoint-event.h>
+----
+--
+
+. Create the tracepoint provider package source file:
++
+--
+[source,c]
+.path:{hello-tp.c}
+----
+#define TRACEPOINT_CREATE_PROBES
+#define TRACEPOINT_DEFINE
+
+#include "hello-tp.h"
+----
+--
+
+. Build the tracepoint provider package:
++
+--
+[role="term"]
+----
+gcc -c -I. hello-tp.c
+----
+--
+
+. Create the _Hello World_ application source file:
++
+--
+[source,c]
+.path:{hello.c}
+----
+#include <stdio.h>
+#include "hello-tp.h"
+
+int main(int argc, char *argv[])
+{
+ int x;
+
+ puts("Hello, World!\nPress Enter to continue...");
+
+ /*
+ * The following getchar() call is only placed here for the purpose
+ * of this demonstration, to pause the application in order for
+ * you to have time to list its tracepoints. It is not
+ * needed otherwise.
+ */
+ getchar();
+
+ /*
+ * A tracepoint() call.
+ *
+ * Arguments, as defined in hello-tp.h:
+ *
+ * 1. Tracepoint provider name (required)
+ * 2. Tracepoint name (required)
+ * 3. my_integer_arg (first user-defined argument)
+ * 4. my_string_arg (second user-defined argument)
+ *
+ * Notice the tracepoint provider and tracepoint names are
+ * NOT strings: they are in fact parts of variables that the
+ * macros in hello-tp.h create.
+ */
+ tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");
+
+ for (x = 0; x < argc; ++x) {
+ tracepoint(hello_world, my_first_tracepoint, x, argv[x]);
+ }
+
+ puts("Quitting now!");
+ tracepoint(hello_world, my_first_tracepoint, x * x, "x^2");
+
+ return 0;
+}
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -c hello.c
+----
+--
+
+. Link the application with the tracepoint provider package,
+ `liblttng-ust`, and `libdl`:
++
+--
+[role="term"]
+----
+gcc -o hello hello.o hello-tp.o -llttng-ust -ldl
+----
+--
+
+Here's the whole build process:
+
+[role="img-100"]
+.User space tracing tutorial's build steps.
+image::ust-flow.png[]
+
+To trace the user application:
+
+. Run the application with a few arguments:
++
+--
+[role="term"]
+----
+./hello world and beyond
+----
+--
++
+You see:
++
+--
+----
+Hello, World!
+Press Enter to continue...
+----
+--
+
+. Start an LTTng <<lttng-sessiond,session daemon>>:
++
+--
+[role="term"]
+----
+lttng-sessiond --daemonize
+----
+--
++
+Note that a session daemon might already be running, for example as
+a service that the distribution's service manager started.
+
+. List the available user space tracepoints:
++
+--
+[role="term"]
+----
+lttng list --userspace
+----
+--
++
+You see the `hello_world:my_first_tracepoint` tracepoint listed
+under the `./hello` process.
+
+. Create a <<tracing-session,tracing session>>:
++
+--
+[role="term"]
+----
+lttng create my-user-space-session
+----
+--
+
+. Create an <<event,event rule>> which matches the
+ `hello_world:my_first_tracepoint` event name:
++
+--
+[role="term"]
+----
+lttng enable-event --userspace hello_world:my_first_tracepoint
+----
+--
+
+. Start tracing:
++
+--
+[role="term"]
+----
+lttng start
+----
+--
+
+. Go back to the running `hello` application and press Enter. The
+ program executes all `tracepoint()` instrumentation points and exits.
+. Stop tracing and destroy the tracing session:
++
+--
+[role="term"]
+----
+sudo lttng stop
+sudo lttng destroy
+----
+--
++
+The `destroy` command does not destroy the trace data; it only destroys
+the state of the tracing session.
+
+By default, LTTng saves the traces in
++$LTTNG_HOME/lttng-traces/__name__-__date__-__time__+,
+where +__name__+ is the tracing session name. Note that the
+env:LTTNG_HOME environment variable defaults to `$HOME` if not set.
+
+See <<viewing-and-analyzing-your-traces,View and analyze the
+recorded events>> to view the recorded events.
+
+
+[[viewing-and-analyzing-your-traces]]
+=== View and analyze the recorded events
+
+Once you have completed the <<tracing-the-linux-kernel,Trace the Linux
+kernel>> and <<tracing-your-own-user-application,Trace a user
+application>> tutorials, you can inspect the recorded events.
+
+Many tools are available to read LTTng traces:
+
+* **cmd:babeltrace** is a command-line utility which converts trace
+ formats; it supports the format that LTTng produces, CTF, as well as a
+ basic text output which can be ++grep++ed. The cmd:babeltrace command
+ is part of the http://diamon.org/babeltrace[Babeltrace] project.
+* Babeltrace also includes
+ **https://www.python.org/[Python] bindings** so
+ that you can easily open and read an LTTng trace with your own script,
+ benefiting from the power of Python.
+* http://tracecompass.org/[**Trace Compass**]
+ is a graphical user interface for viewing and analyzing any type of
+ logs or traces, including LTTng's.
+* https://github.com/lttng/lttng-analyses[**LTTng analyses**] is a
+ project which includes many high-level analyses of LTTng kernel
+ traces, like scheduling statistics, interrupt frequency distribution,
+ top CPU usage, and more.
+
+NOTE: This section assumes that the traces recorded during the previous
+tutorials were saved to their default location, in the
+dir:{$LTTNG_HOME/lttng-traces} directory. Note that the env:LTTNG_HOME
+environment variable defaults to `$HOME` if not set.
+
+
+[[viewing-and-analyzing-your-traces-bt]]
+==== Use the cmd:babeltrace command-line tool
+
+The simplest way to list all the recorded events of a trace is to pass
+its path to cmd:babeltrace with no options:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-user-space-session*
+----
+
+cmd:babeltrace finds all traces recursively within the given path and
+prints all their events, merging them in chronological order.
+
+You can pipe the output of cmd:babeltrace into a tool like man:grep(1) for
+further filtering:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session* | grep sys_
+----
+
+You can pipe the output of cmd:babeltrace into a tool like man:wc(1) to
+count the recorded events:
+
+[role="term"]
+----
+babeltrace ~/lttng-traces/my-kernel-session* | grep sys_read | wc --lines
+----
+
+
+[[viewing-and-analyzing-your-traces-bt-python]]
+==== Use the Babeltrace Python bindings
+
+The <<viewing-and-analyzing-your-traces-bt,text output of cmd:babeltrace>>
+is useful to isolate events by simple matching using man:grep(1) and
+similar utilities. However, more elaborate filters, such as keeping only
+event records with a field value falling within a specific range, are
+not trivial to write using a shell. Moreover, reductions and even the
+most basic computations involving multiple event records are virtually
+impossible to implement.
+
+Fortunately, Babeltrace ships with Python 3 bindings which makes it easy
+to read the event records of an LTTng trace sequentially and compute the
+desired information.
+
+The following script accepts an LTTng Linux kernel trace path as its
+first argument and prints the short names of the top 5 running processes
+on CPU 0 during the whole trace:
+
+[source,python]
+.path:{top5proc.py}
+----
+from collections import Counter
+import babeltrace
+import sys
+
+
+def top5proc():
+ if len(sys.argv) != 2:
+ msg = 'Usage: python3 {} TRACEPATH'.format(sys.argv[0])
+ print(msg, file=sys.stderr)
+ return False
+
+ # A trace collection contains one or more traces
+ col = babeltrace.TraceCollection()
+
+ # Add the trace provided by the user (LTTng traces always have
+ # the 'ctf' format)
+ if col.add_trace(sys.argv[1], 'ctf') is None:
+ raise RuntimeError('Cannot add trace')
+
+ # This counter dict contains execution times:
+ #
+ # task command name -> total execution time (ns)
+ exec_times = Counter()
+
+ # This contains the last `sched_switch` timestamp
+ last_ts = None
+
+ # Iterate on events
+ for event in col.events:
+ # Keep only `sched_switch` events
+ if event.name != 'sched_switch':
+ continue
+
+ # Keep only events which happened on CPU 0
+ if event['cpu_id'] != 0:
+ continue
+
+ # Event timestamp
+ cur_ts = event.timestamp
+
+ if last_ts is None:
+ # We start here
+ last_ts = cur_ts
+
+ # Previous task command (short) name
+ prev_comm = event['prev_comm']
+
+ # Initialize entry in our dict if not yet done
+ if prev_comm not in exec_times:
+ exec_times[prev_comm] = 0
+
+ # Compute previous command execution time
+ diff = cur_ts - last_ts
+
+ # Update execution time of this command
+ exec_times[prev_comm] += diff
+
+ # Update last timestamp
+ last_ts = cur_ts
+
+ # Display top 5
+ for name, ns in exec_times.most_common(5):
+ s = ns / 1000000000
+ print('{:20}{} s'.format(name, s))
+
+ return True
+
+
+if __name__ == '__main__':
+ sys.exit(0 if top5proc() else 1)
+----
+
+Run this script:
+
+[role="term"]
+----
+python3 top5proc.py ~/lttng-traces/my-kernel-session*/kernel
+----
+
+Output example:
+
+----
+swapper/0 48.607245889 s
+chromium 7.192738188 s
+pavucontrol 0.709894415 s
+Compositor 0.660867933 s
+Xorg.bin 0.616753786 s
+----
+
+Note that `swapper/0` is the "idle" process of CPU 0 on Linux; since we
+weren't using the CPU that much when tracing, its first position in the
+list makes sense.
+
+
+[[core-concepts]]
+== [[understanding-lttng]]Core concepts
+
+From a user's perspective, the LTTng system is built on a few concepts,
+or objects, on which the <<lttng-cli,cmd:lttng command-line tool>>
+operates by sending commands to the <<lttng-sessiond,session daemon>>.
+Understanding how those objects relate to eachother is key in mastering
+the toolkit.
+
+The core concepts are:
+
+* <<tracing-session,Tracing session>>
+* <<domain,Tracing domain>>
+* <<channel,Channel and ring buffer>>
+* <<"event","Instrumentation point, event rule, event, and event record">>
+
+
+[[tracing-session]]
+=== Tracing session
+
+A _tracing session_ is a stateful dialogue between you and
+a <<lttng-sessiond,session daemon>>. You can
+<<creating-destroying-tracing-sessions,create a new tracing
+session>> with the `lttng create` command.
+
+Anything that you do when you control LTTng tracers happens within a
+tracing session. In particular, a tracing session:
+
+* Has its own name.
+* Has its own set of trace files.
+* Has its own state of activity (started or stopped).
+* Has its own <<tracing-session-mode,mode>> (local, network streaming,
+ snapshot, or live).
+* Has its own <<channel,channels>> which have their own
+ <<event,event rules>>.
+
+[role="img-100"]
+.A _tracing session_ contains <<channel,channels>> that are members of <<domain,tracing domains>> and contain <<event,event rules>>.
+image::concepts.png[]
+
+Those attributes and objects are completely isolated between different
+tracing sessions.
+
+A tracing session is analogous to a cash machine session:
+the operations you do on the banking system through the cash machine do
+not alter the data of other users of the same system. In the case of
+the cash machine, a session lasts as long as your bank card is inside.
+In the case of LTTng, a tracing session lasts from the `lttng create`
+command to the `lttng destroy` command.
+
+[role="img-100"]
+.Each Unix user has its own set of tracing sessions.
+image::many-sessions.png[]
+
+
+[[tracing-session-mode]]
+==== Tracing session mode
+
+LTTng can send the generated trace data to different locations. The
+_tracing session mode_ dictates where to send it. The following modes
+are available in LTTng{nbsp}{revision}:
+
+Local mode::
+ LTTng writes the traces to the file system of the machine being traced
+ (target system).
+
+Network streaming mode::
+ LTTng sends the traces over the network to a
+ <<lttng-relayd,relay daemon>> running on a remote system.
+
+Snapshot mode::
+ LTTng does not write the traces by default. Instead, you can request
+ LTTng to <<taking-a-snapshot,take a snapshot>>, that is, a copy of the
+ current tracing buffers, and to write it to the target's file system
+ or to send it over the network to a <<lttng-relayd,relay daemon>>
+ running on a remote system.
+
+Live mode::
+ This mode is similar to the network streaming mode, but a live
+ trace viewer can connect to the distant relay daemon to
+ <<lttng-live,view event records as LTTng generates them>> by
+ the tracers.
+
+
+[[domain]]
+=== Tracing domain
+
+A _tracing domain_ is a namespace for event sources. A tracing domain
+has its own properties and features.
+
+There are currently five available tracing domains:
+
+* Linux kernel
+* User space
+* `java.util.logging` (JUL)
+* log4j
+* Python
+
+You must specify a tracing domain when using some commands to avoid
+ambiguity. For example, since all the domains support named tracepoints
+as event sources (instrumentation points that you manually insert in the
+source code), you need to specify a tracing domain when
+<<enabling-disabling-events,creating an event rule>> because all the
+tracing domains could have tracepoints with the same names.
+
+Some features are reserved to specific tracing domains. Dynamic function
+entry and return instrumentation points, for example, are currently only
+supported in the Linux kernel tracing domain, but support for other
+tracing domains could be added in the future.
+
+You can create <<channel,channels>> in the Linux kernel and user space
+tracing domains. The other tracing domains have a single default
+channel.
+
+
+[[channel]]
+=== Channel and ring buffer
+
+A _channel_ is an object which is responsible for a set of ring buffers.
+Each ring buffer is divided into multiple sub-buffers. When an LTTng
+tracer emits an event, it can record it to one or more
+sub-buffers. The attributes of a channel determine what to do when
+there's no space left for a new event record because all sub-buffers
+are full, where to send a full sub-buffer, and other behaviours.
+
+A channel is always associated to a <<domain,tracing domain>>. The
+`java.util.logging` (JUL), log4j, and Python tracing domains each have
+a default channel which you cannot configure.
+
+A channel also owns <<event,event rules>>. When an LTTng tracer emits
+an event, it records it to the sub-buffers of all
+the enabled channels with a satisfied event rule, as long as those
+channels are part of active <<tracing-session,tracing sessions>>.
+
+
+[[channel-buffering-schemes]]
+==== Per-user vs. per-process buffering schemes
+
+A channel has at least one ring buffer _per CPU_. LTTng always
+records an event to the ring buffer associated to the CPU on which it
+occurred.
+
+Two _buffering schemes_ are available when you
+<<enabling-disabling-channels,create a channel>> in the
+user space <<domain,tracing domain>>:
+
+Per-user buffering::
+ Allocate one set of ring buffers--one per CPU--shared by all the
+ instrumented processes of each Unix user.
++
+--
+[role="img-100"]
+.Per-user buffering scheme.
+image::per-user-buffering.png[]
+--
+
+Per-process buffering::
+ Allocate one set of ring buffers--one per CPU--for each
+ instrumented process.
++
+--
+[role="img-100"]
+.Per-process buffering scheme.
+image::per-process-buffering.png[]
+--
++
+The per-process buffering scheme tends to consume more memory than the
+per-user option because systems generally have more instrumented
+processes than Unix users running instrumented processes. However, the
+per-process buffering scheme ensures that one process having a high
+event throughput won't fill all the shared sub-buffers of the same
+user, only its own.
+
+The Linux kernel tracing domain has only one available buffering scheme
+which is to allocate a single set of ring buffers for the whole system.
+This scheme is similar to the per-user option, but with a single, global
+user "running" the kernel.
+
+
+[[channel-overwrite-mode-vs-discard-mode]]
+==== Overwrite vs. discard event loss modes
+
+When an event occurs, LTTng records it to a specific sub-buffer (yellow
+arc in the following animation) of a specific channel's ring buffer.
+When there's no space left in a sub-buffer, the tracer marks it as
+consumable (red) and another, empty sub-buffer starts receiving the
+following event records. A <<lttng-consumerd,consumer daemon>>
+eventually consumes the marked sub-buffer (returns to white).
+
+[NOTE]
+[role="docsvg-channel-subbuf-anim"]
+====
+{note-no-anim}
+====
+
+In an ideal world, sub-buffers are consumed faster than they are filled,
+as is the case in the previous animation. In the real world,
+however, all sub-buffers can be full at some point, leaving no space to
+record the following events.
+
+By design, LTTng is a _non-blocking_ tracer: when no empty sub-buffer is
+available, it is acceptable to lose event records when the alternative
+would be to cause substantial delays in the instrumented application's
+execution. LTTng privileges performance over integrity; it aims at
+perturbing the traced system as little as possible in order to make
+tracing of subtle race conditions and rare interrupt cascades possible.
+
+When it comes to losing event records because no empty sub-buffer is
+available, the channel's _event loss mode_ determines what to do. The
+available event loss modes are:
+
+Discard mode::
+ Drop the newest event records until a the tracer
+ releases a sub-buffer.
+
+Overwrite mode::
+ Clear the sub-buffer containing the oldest event records and start
+ writing the newest event records there.
++
+This mode is sometimes called _flight recorder mode_ because it's
+similar to a
+https://en.wikipedia.org/wiki/Flight_recorder[flight recorder]:
+always keep a fixed amount of the latest data.
+
+Which mechanism you should choose depends on your context: prioritize
+the newest or the oldest event records in the ring buffer?
+
+Beware that, in overwrite mode, the tracer abandons a whole sub-buffer
+as soon as a there's no space left for a new event record, whereas in
+discard mode, the tracer only discards the event record that doesn't
+fit.
+
+In discard mode, LTTng increments a count of lost event records when
+an event record is lost and saves this count to the trace. In
+overwrite mode, LTTng keeps no information when it overwrites a
+sub-buffer before consuming it.
+
+There are a few ways to decrease your probability of losing event
+records.
+<<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>> shows
+how you can fine-une the sub-buffer count and size of a channel to
+virtually stop losing event records, though at the cost of greater
+memory usage.
+
+
+[[channel-subbuf-size-vs-subbuf-count]]
+==== Sub-buffer count and size
+
+When you <<enabling-disabling-channels,create a channel>>, you can
+set its number of sub-buffers and their size.
+
+Note that there is noticeable CPU overhead introduced when
+switching sub-buffers (marking a full one as consumable and switching
+to an empty one for the following events to be recorded). Knowing this,
+the following list presents a few practical situations along with how
+to configure the sub-buffer count and size for them:
+
+* **High event throughput**: In general, prefer bigger sub-buffers to
+ lower the risk of losing event records.
++
+Having bigger sub-buffers also ensures a lower sub-buffer switching
+frequency.
++
+The number of sub-buffers is only meaningful if you create the channel
+in overwrite mode: in this case, if a sub-buffer overwrite happens, the
+other sub-buffers are left unaltered.
+
+* **Low event throughput**: In general, prefer smaller sub-buffers
+ since the risk of losing event records is low.
++
+Because events occur less frequently, the sub-buffer switching frequency
+should remain low and thus the tracer's overhead should not be a
+problem.
+
+* **Low memory system**: If your target system has a low memory
+ limit, prefer fewer first, then smaller sub-buffers.
++
+Even if the system is limited in memory, you want to keep the
+sub-buffers as big as possible to avoid a high sub-buffer switching
+frequency.
+
+Note that LTTng uses http://diamon.org/ctf/[CTF] as its trace format,
+which means event data is very compact. For example, the average
+LTTng kernel event record weights about 32{nbsp}bytes. Thus, a
+sub-buffer size of 1{nbsp}MiB is considered big.
+
+The previous situations highlight the major trade-off between a few big
+sub-buffers and more, smaller sub-buffers: sub-buffer switching
+frequency vs. how much data is lost in overwrite mode. Assuming a
+constant event throughput and using the overwrite mode, the two
+following configurations have the same ring buffer total size:
+
+[NOTE]
+[role="docsvg-channel-subbuf-size-vs-count-anim"]
+====
+{note-no-anim}
+====
+
+* **2 sub-buffers of 4{nbsp}MiB each**: Expect a very low sub-buffer
+ switching frequency, but if a sub-buffer overwrite happens, half of
+ the event records so far (4{nbsp}MiB) are definitely lost.
+* **8 sub-buffers of 1{nbsp}MiB each**: Expect 4{nbsp}times the tracer's
+ overhead as the previous configuration, but if a sub-buffer
+ overwrite happens, only the eighth of event records so far are
+ definitely lost.
+
+In discard mode, the sub-buffers count parameter is pointless: use two
+sub-buffers and set their size according to the requirements of your
+situation.
+
+
+[[channel-switch-timer]]
+==== Switch timer period
+
+The _switch timer period_ is an important configurable attribute of
+a channel to ensure periodic sub-buffer flushing.
+
+When the _switch timer_ expires, a sub-buffer switch happens. You can
+set the switch timer period attribute when you
+<<enabling-disabling-channels,create a channel>> to ensure that event
+data is consumed and committed to trace files or to a distant relay
+daemon periodically in case of a low event throughput.
+
+[NOTE]
+[role="docsvg-channel-switch-timer"]
+====
+{note-no-anim}
+====
+
+This attribute is also convenient when you use big sub-buffers to cope
+with a sporadic high event throughput, even if the throughput is
+normally low.
+
+
+[[channel-read-timer]]
+==== Read timer period
+
+By default, the LTTng tracers use a notification mechanism to signal a
+full sub-buffer so that a consumer daemon can consume it. When such
+notifications must be avoided, for example in real-time applications,
+you can use the channel's _read timer_ instead. When the read timer
+fires, the <<lttng-consumerd,consumer daemon>> checks for full,
+consumable sub-buffers.
+
+
+[[tracefile-rotation]]
+==== Trace file count and size
+
+By default, trace files can grow as large as needed. You can set the
+maximum size of each trace file that a channel writes when you
+<<enabling-disabling-channels,create a channel>>. When the size of
+a trace file reaches the channel's fixed maximum size, LTTng creates
+another file to contain the next event records. LTTng appends a file
+count to each trace file name in this case.
+
+If you set the trace file size attribute when you create a channel, the
+maximum number of trace files that LTTng creates is _unlimited_ by
+default. To limit them, you can also set a maximum number of trace
+files. When the number of trace files reaches the channel's fixed
+maximum count, the oldest trace file is overwritten. This mechanism is
+called _trace file rotation_.
+
+
+[[event]]
+=== Instrumentation point, event rule, event, and event record
+
+An _event rule_ is a set of conditions which must be **all** satisfied
+for LTTng to record an occuring event.
+
+You set the conditions when you <<enabling-disabling-events,create
+an event rule>>.
+
+You always attach an event rule to <<channel,channel>> when you create
+it.
+
+When an event passes the conditions of an event rule, LTTng records it
+in one of the attached channel's sub-buffers.
+
+The available conditions, as of LTTng{nbsp}{revision}, are:
+
+* The event rule _is enabled_.
+* The instrumentation point's type _is{nbsp}T_.
+* The instrumentation point's name (sometimes called _event name_)
+ _matches{nbsp}N_, but _is not{nbsp}E_.
+* The instrumentation point's log level _is as severe as{nbsp}L_, or
+ _is exactly{nbsp}L_.
+* The fields of the event's payload _satisfy_ a filter
+ expression{nbsp}__F__.
+
+As you can see, all the conditions but the dynamic filter are related to
+the event rule's status or to the instrumentation point, not to the
+occurring events. This is why, without a filter, checking if an event
+passes an event rule is not a dynamic task: when you create or modify an
+event rule, all the tracers of its tracing domain enable or disable the
+instrumentation points themselves once. This is possible because the
+attributes of an instrumentation point (type, name, and log level) are
+defined statically. In other words, without a dynamic filter, the tracer
+_does not evaluate_ the arguments of an instrumentation point unless it
+matches an enabled event rule.
+
+Note that, for LTTng to record an event, the <<channel,channel>> to
+which a matching event rule is attached must also be enabled, and the
+tracing session owning this channel must be active.
+
+[role="img-100"]
+.Logical path from an instrumentation point to an event record.
+image::event-rule.png[]
+
+.Event, event record, or event rule?
+****
+With so many similar terms, it's easy to get confused.
+
+An **event** is the consequence of the execution of an _instrumentation
+point_, like a tracepoint that you manually place in some source code,
+or a Linux kernel KProbe. An event is said to _occur_ at a specific
+time. Different actions can be taken upon the occurance of an event,
+like record the event's payload to a buffer.
+
+An **event record** is the representation of an event in a sub-buffer. A
+tracer is responsible for capturing the payload of an event, current
+context variables, the event's ID, and the event's timestamp. LTTng
+can append this sub-buffer to a trace file.
+
+An **event rule** is a set of conditions which must all be satisfied for
+LTTng to record an occuring event. Events still occur without
+satisfying event rules, but LTTng does not record them.
+****
+
+
+[[plumbing]]
+== Components of noch:{LTTng}
+
+The second _T_ in _LTTng_ stands for _toolkit_: it would be wrong
+to call LTTng a simple _tool_ since it is composed of multiple
+interacting components. This section describes those components,
+explains their respective roles, and shows how they connect together to
+form the LTTng ecosystem.
+
+The following diagram shows how the most important components of LTTng
+interact with user applications, the Linux kernel, and you:
+
+[role="img-100"]
+.Control and trace data paths between LTTng components.
+image::plumbing.png[]
+
+The LTTng project incorporates:
+
+* **LTTng-tools**: Libraries and command-line interface to
+ control tracing sessions.
+** <<lttng-sessiond,Session daemon>> (man:lttng-sessiond(8)).
+** <<lttng-consumerd,Consumer daemon>> (man:lttng-consumerd(8)).
+** <<lttng-relayd,Relay daemon>> (man:lttng-relayd(8)).
+** <<liblttng-ctl-lttng,Tracing control library>> (`liblttng-ctl`).
+** <<lttng-cli,Tracing control command-line tool>> (man:lttng(1)).
+* **LTTng-UST**: Libraries and Java/Python packages to trace user
+ applications.
+** <<lttng-ust,User space tracing library>> (`liblttng-ust`) and its
+ headers to instrument and trace any native user application.
+** <<prebuilt-ust-helpers,Preloadable user space tracing helpers>>:
+*** `liblttng-ust-libc-wrapper`
+*** `liblttng-ust-pthread-wrapper`
+*** `liblttng-ust-cyg-profile`
+*** `liblttng-ust-cyg-profile-fast`
+*** `liblttng-ust-dl`
+** User space tracepoint provider source files generator command-line
+ tool (man:lttng-gen-tp(1)).
+** <<lttng-ust-agents,LTTng-UST Java agent>> to instrument and trace
+ Java applications using `java.util.logging` or
+ Apache log4j 1.2 logging.
+** <<lttng-ust-agents,LTTng-UST Python agent>> to instrument
+ Python applications using the standard `logging` package.
+* **LTTng-modules**: <<lttng-modules,Linux kernel modules>> to trace
+ the kernel.
+** LTTng kernel tracer module.
+** Tracing ring buffer kernel modules.
+** Probe kernel modules.
+** LTTng logger kernel module.
+
+
+[[lttng-cli]]
+=== Tracing control command-line interface
+
+[role="img-100"]
+.The tracing control command-line interface.
+image::plumbing-lttng-cli.png[]
+
+The _man:lttng(1) command-line tool_ is the standard user interface to
+control LTTng <<tracing-session,tracing sessions>>. The cmd:lttng tool
+is part of LTTng-tools.
+
+The cmd:lttng tool is linked with
+<<liblttng-ctl-lttng,`liblttng-ctl`>> to communicate with
+one or more <<lttng-sessiond,session daemons>> behind the scenes.
+
+The cmd:lttng tool has a Git-like interface:
+
+[role="term"]
+----
+lttng <general options> <command> <command options>
+----
+
+The <<controlling-tracing,Tracing control>> section explores the
+available features of LTTng using the cmd:lttng tool.
+
+
+[[liblttng-ctl-lttng]]
+=== Tracing control library
+
+[role="img-100"]
+.The tracing control library.
+image::plumbing-liblttng-ctl.png[]
+
+The _LTTng control library_, `liblttng-ctl`, is used to communicate
+with a <<lttng-sessiond,session daemon>> using a C API that hides the
+underlying protocol's details. `liblttng-ctl` is part of LTTng-tools.
+
+The <<lttng-cli,cmd:lttng command-line tool>>
+is linked with `liblttng-ctl`.
+
+You can use `liblttng-ctl` in C or $$C++$$ source code by including its
+"master" header:
+
+[source,c]
+----
+#include <lttng/lttng.h>
+----
+
+Some objects are referenced by name (C string), such as tracing
+sessions, but most of them require to create a handle first using
+`lttng_create_handle()`.
+
+The best available developer documentation for `liblttng-ctl` is, as of
+LTTng{nbsp}{revision}, its installed header files. Every function and
+structure is thoroughly documented.
+
+
+[[lttng-ust]]
+=== User space tracing library
+
+[role="img-100"]
+.The user space tracing library.
+image::plumbing-liblttng-ust.png[]
+
+The _user space tracing library_, `liblttng-ust` (see man:lttng-ust(3)),
+is the LTTng user space tracer. It receives commands from a
+<<lttng-sessiond,session daemon>>, for example to
+enable and disable specific instrumentation points, and writes event
+records to ring buffers shared with a
+<<lttng-consumerd,consumer daemon>>.
+`liblttng-ust` is part of LTTng-UST.
+
+Public C header files are installed beside `liblttng-ust` to
+instrument any <<c-application,C or $$C++$$ application>>.
+
+<<lttng-ust-agents,LTTng-UST agents>>, which are regular Java and Python
+packages, use their own library providing tracepoints which is
+linked with `liblttng-ust`.
+
+An application or library does not have to initialize `liblttng-ust`
+manually: its constructor does the necessary tasks to properly register
+to a session daemon. The initialization phase also enables the
+instrumentation points matching the <<event,event rules>> that you
+already created.
+
+
+[[lttng-ust-agents]]
+=== User space tracing agents
+
+[role="img-100"]
+.The user space tracing agents.
+image::plumbing-lttng-ust-agents.png[]
+
+The _LTTng-UST Java and Python agents_ are regular Java and Python
+packages which add LTTng tracing capabilities to the
+native logging frameworks. The LTTng-UST agents are part of LTTng-UST.
+
+In the case of Java, the
+https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[`java.util.logging`
+core logging facilities] and
+https://logging.apache.org/log4j/1.2/[Apache log4j 1.2] are supported.
+Note that Apache Log4{nbsp}2 is not supported.
+
+In the case of Python, the standard
+https://docs.python.org/3/library/logging.html[`logging`] package
+is supported. Both Python 2 and Python 3 modules can import the
+LTTng-UST Python agent package.
+
+The applications using the LTTng-UST agents are in the
+`java.util.logging` (JUL),
+log4j, and Python <<domain,tracing domains>>.
+
+Both agents use the same mechanism to trace the log statements. When an
+agent is initialized, it creates a log handler that attaches to the root
+logger. The agent also registers to a <<lttng-sessiond,session daemon>>.
+When the application executes a log statement, it is passed to the
+agent's log handler by the root logger. The agent's log handler calls a
+native function in a tracepoint provider package shared library linked
+with <<lttng-ust,`liblttng-ust`>>, passing the formatted log message and
+other fields, like its logger name and its log level. This native
+function contains a user space instrumentation point, hence tracing the
+log statement.
+
+The log level condition of an
+<<event,event rule>> is considered when tracing
+a Java or a Python application, and it's compatible with the standard
+JUL, log4j, and Python log levels.
+
+
+[[lttng-modules]]
+=== LTTng kernel modules
+
+[role="img-100"]
+.The LTTng kernel modules.
+image::plumbing-lttng-modules.png[]
+
+The _LTTng kernel modules_ are a set of Linux kernel modules
+which implement the kernel tracer of the LTTng project. The LTTng
+kernel modules are part of LTTng-modules.
+
+The LTTng kernel modules include:
+
+* A set of _probe_ modules.
++
+Each module attaches to a specific subsystem
+of the Linux kernel using its tracepoint instrument points. There are
+also modules to attach to the entry and return points of the Linux
+system call functions.
+
+* _Ring buffer_ modules.
++
+A ring buffer implementation is provided as kernel modules. The LTTng
+kernel tracer writes to the ring buffer; a
+<<lttng-consumerd,consumer daemon>> reads from the ring buffer.
+
+* The _LTTng kernel tracer_ module.
+* The _LTTng logger_ module.
++
+The LTTng logger module implements the special path:{/proc/lttng-logger}
+file so that any executable can generate LTTng events by opening and
+writing to this file.
++
+See <<proc-lttng-logger-abi,LTTng logger>>.
+
+Generally, you do not have to load the LTTng kernel modules manually
+(using man:modprobe(8), for example): a root <<lttng-sessiond,session
+daemon>> loads the necessary modules when starting. If you have extra
+probe modules, you can specify to load them to the session daemon on
+the command line.
+
+The LTTng kernel modules are installed in
++/usr/lib/modules/__release__/extra+ by default, where +__release__+ is
+the kernel release (see `uname --kernel-release`).
+
+
+[[lttng-sessiond]]
+=== Session daemon
+
+[role="img-100"]
+.The session daemon.
+image::plumbing-sessiond.png[]
+
+The _session daemon_, man:lttng-sessiond(8), is a daemon responsible for
+managing tracing sessions and for controlling the various components of
+LTTng. The session daemon is part of LTTng-tools.
+
+The session daemon sends control requests to and receives control
+responses from:
+
+* The <<lttng-ust,user space tracing library>>.
++
+Any instance of the user space tracing library first registers to
+a session daemon. Then, the session daemon can send requests to
+this instance, such as:
++
+--
+** Get the list of tracepoints.
+** Share an <<event,event rule>> so that the user space tracing library
+ can enable or disable tracepoints. Amongst the possible conditions
+ of an event rule is a filter expression which `liblttng-ust` evalutes
+ when an event occurs.
+** Share <<channel,channel>> attributes and ring buffer locations.
+--
++
+The session daemon and the user space tracing library use a Unix
+domain socket for their communication.
+
+* The <<lttng-ust-agents,user space tracing agents>>.
++
+Any instance of a user space tracing agent first registers to
+a session daemon. Then, the session daemon can send requests to
+this instance, such as:
++
+--
+** Get the list of loggers.
+** Enable or disable a specific logger.
+--
++
+The session daemon and the user space tracing agent use a TCP connection
+for their communication.
+
+* The <<lttng-modules,LTTng kernel tracer>>.
+* The <<lttng-consumerd,consumer daemon>>.
++
+The session daemon sends requests to the consumer daemon to instruct
+it where to send the trace data streams, amongst other information.
+
+* The <<lttng-relayd,relay daemon>>.
+
+The session daemon receives commands from the
+<<liblttng-ctl-lttng,tracing control library>>.
+
+The root session daemon loads the appropriate
+<<lttng-modules,LTTng kernel modules>> on startup. It also spawns
+a <<lttng-consumerd,consumer daemon>> as soon as you create
+an <<event,event rule>>.
+
+The session daemon does not send and receive trace data: this is the
+role of the <<lttng-consumerd,consumer daemon>> and
+<<lttng-relayd,relay daemon>>. It does, however, generate the
+http://diamon.org/ctf/[CTF] metadata stream.
+
+Each Unix user can have its own session daemon instance. The
+tracing sessions managed by different session daemons are completely
+independent.
+
+The root user's session daemon is the only one which is
+allowed to control the LTTng kernel tracer, and its spawned consumer
+daemon is the only one which is allowed to consume trace data from the
+LTTng kernel tracer. Note, however, that any Unix user which is a member
+of the <<tracing-group,tracing group>> is allowed
+to create <<channel,channels>> in the
+Linux kernel <<domain,tracing domain>>, and thus to trace the Linux
+kernel.
+
+The <<lttng-cli,cmd:lttng command-line tool>> automatically starts a
+session daemon when using its `create` command if none is currently
+running. You can also start the session daemon manually.
+
+
+[[lttng-consumerd]]
+=== Consumer daemon
+
+[role="img-100"]
+.The consumer daemon.
+image::plumbing-consumerd.png[]
+
+The _consumer daemon_, man:lttng-consumerd(8), is a daemon which shares
+ring buffers with user applications or with the LTTng kernel modules to
+collect trace data and send it to some location (on disk or to a
+<<lttng-relayd,relay daemon>> over the network). The consumer daemon
+is part of LTTng-tools.
+
+You do not start a consumer daemon manually: a consumer daemon is always
+spawned by a <<lttng-sessiond,session daemon>> as soon as you create an
+<<event,event rule>>, that is, before you start tracing. When you kill
+its owner session daemon, the consumer daemon also exits because it is
+the session daemon's child process. Command-line options of
+man:lttng-sessiond(8) target the consumer daemon process.
+
+There are up to two running consumer daemons per Unix user, whereas only
+one session daemon can run per user. This is because each process can be
+either 32-bit or 64-bit: if the target system runs a mixture of 32-bit
+and 64-bit processes, it is more efficient to have separate
+corresponding 32-bit and 64-bit consumer daemons. The root user is an
+exception: it can have up to _three_ running consumer daemons: 32-bit
+and 64-bit instances for its user applications, and one more
+reserved for collecting kernel trace data.
+
+
+[[lttng-relayd]]
+=== Relay daemon
+
+[role="img-100"]
+.The relay daemon.
+image::plumbing-relayd.png[]
+
+The _relay daemon_, man:lttng-relayd(8), is a daemon acting as a bridge
+between remote session and consumer daemons, local trace files, and a
+remote live trace viewer. The relay daemon is part of LTTng-tools.
+
+The main purpose of the relay daemon is to implement a receiver of
+<<sending-trace-data-over-the-network,trace data over the network>>.
+This is useful when the target system does not have much file system
+space to record trace files locally.
+
+The relay daemon is also a server to which a
+<<lttng-live,live trace viewer>> can
+connect. The live trace viewer sends requests to the relay daemon to
+receive trace data as the target system emits events. The
+communication protocol is named _LTTng live_; it is used over TCP
+connections.
+
+Note that you can start the relay daemon on the target system directly.
+This is the setup of choice when the use case is to view events as
+the target system emits them without the need of a remote system.
+
+
+[[instrumenting]]
+== [[using-lttng]]Instrumentation
+
+There are many examples of tracing and monitoring in our everyday life:
+
+* You have access to real-time and historical weather reports and
+ forecasts thanks to weather stations installed around the country.
+* You know your heart is safe thanks to an electrocardiogram.
+* You make sure not to drive your car too fast and to have enough fuel
+ to reach your destination thanks to gauges visible on your dashboard.
+
+All the previous examples have something in common: they rely on
+**instruments**. Without the electrodes attached to the surface of your
+body's skin, cardiac monitoring is futile.
+
+LTTng, as a tracer, is no different from those real life examples. If
+you're about to trace a software system or, in other words, record its
+history of execution, you better have **instrumentation points** in the
+subject you're tracing, that is, the actual software.
+
+Various ways were developed to instrument a piece of software for LTTng
+tracing. The most straightforward one is to manually place
+instrumentation points, called _tracepoints_, in the software's source
+code. It is also possible to add instrumentation points dynamically in
+the Linux kernel <<domain,tracing domain>>.
+
+If you're only interested in tracing the Linux kernel, your
+instrumentation needs are probably already covered by LTTng's built-in
+<<lttng-modules,Linux kernel tracepoints>>. You may also wish to trace a
+user application which is already instrumented for LTTng tracing.
+In such cases, you can skip this whole section and read the topics of
+the <<controlling-tracing,Tracing control>> section.
+
+Many methods are available to instrument a piece of software for LTTng
+tracing. They are:
+
+* <<c-application,User space instrumentation for C and $$C++$$
+ applications>>.
+* <<prebuilt-ust-helpers,Prebuilt user space tracing helpers>>.
+* <<java-application,User space Java agent>>.
+* <<python-application,User space Python agent>>.
+* <<proc-lttng-logger-abi,LTTng logger>>.
+* <<instrumenting-linux-kernel,LTTng kernel tracepoints>>.
+
+
+[[c-application]]
+=== [[cxx-application]]User space instrumentation for C and $$C++$$ applications
+
+The procedure to instrument a C or $$C++$$ user application with
+the <<lttng-ust,LTTng user space tracing library>>, `liblttng-ust`, is:
+
+. <<tracepoint-provider,Create the source files of a tracepoint provider
+ package>>.
+. <<probing-the-application-source-code,Add tracepoints to
+ the application's source code>>.
+. <<building-tracepoint-providers-and-user-application,Build and link
+ a tracepoint provider package and the user application>>.
+
+If you need quick, man:printf(3)-like instrumentation, you can skip
+those steps and use <<tracef,`tracef()`>> or <<tracelog,`tracelog()`>>
+instead.
+
+IMPORTANT: You need to <<installing-lttng,install>> LTTng-UST to
+instrument a user application with `liblttng-ust`.
+
+
+[[tracepoint-provider]]
+==== Create the source files of a tracepoint provider package
+
+A _tracepoint provider_ is a set of compiled functions which provide
+**tracepoints** to an application, the type of instrumentation point
+supported by LTTng-UST. Those functions can emit events with
+user-defined fields and serialize those events as event records to one
+or more LTTng-UST <<channel,channel>> sub-buffers. The `tracepoint()`
+macro, which you <<probing-the-application-source-code,insert in a user
+application's source code>>, calls those functions.
+
+A _tracepoint provider package_ is an object file (`.o`) or a shared
+library (`.so`) which contains one or more tracepoint providers.
+Its source files are:
+
+* One or more <<tpp-header,tracepoint provider header>> (`.h`).
+* A <<tpp-source,tracepoint provider package source>> (`.c`).
+
+A tracepoint provider package is dynamically linked with `liblttng-ust`,
+the LTTng user space tracer, at run time.
+
+[role="img-100"]
+.User application linked with `liblttng-ust` and containing a tracepoint provider.
+image::ust-app.png[]
+
+NOTE: If you need quick, man:printf(3)-like instrumentation, you can
+skip creating and using a tracepoint provider and use
+<<tracef,`tracef()`>> or <<tracelog,`tracelog()`>> instead.
+
+
+[[tpp-header]]
+===== Create a tracepoint provider header file template
+
+A _tracepoint provider header file_ contains the tracepoint
+definitions of a tracepoint provider.
+
+To create a tracepoint provider header file:
+
+. Start from this template:
++
+--
+[source,c]
+.Tracepoint provider header file template (`.h` file extension).
+----
+#undef TRACEPOINT_PROVIDER
+#define TRACEPOINT_PROVIDER provider_name
+
+#undef TRACEPOINT_INCLUDE
+#define TRACEPOINT_INCLUDE "./tp.h"
+
+#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
+#define _TP_H
+
+#include <lttng/tracepoint.h>
+
+/*
+ * Use TRACEPOINT_EVENT(), TRACEPOINT_EVENT_CLASS(),
+ * TRACEPOINT_EVENT_INSTANCE(), and TRACEPOINT_LOGLEVEL() here.
+ */
+
+#endif /* _TP_H */
+
+#include <lttng/tracepoint-event.h>
+----
+--
+
+. Replace:
++
+* `provider_name` with the name of your tracepoint provider.
+* `"tp.h"` with the name of your tracepoint provider header file.
+
+. Below the `#include <lttng/tracepoint.h>` line, put your
+ <<defining-tracepoints,tracepoint definitions>>.
+
+Your tracepoint provider name must be unique amongst all the possible
+tracepoint provider names used on the same target system. We
+suggest to include the name of your project or company in the name,
+for example, `org_lttng_my_project_tpp`.
+
+TIP: [[lttng-gen-tp]]You can use the man:lttng-gen-tp(1) tool to create
+this boilerplate for you. When using cmd:lttng-gen-tp, all you need to
+write are the <<defining-tracepoints,tracepoint definitions>>.
+
+
+[[defining-tracepoints]]
+===== Create a tracepoint definition
+
+A _tracepoint definition_ defines, for a given tracepoint:
+
+* Its **input arguments**. They are the macro parameters that the
+ `tracepoint()` macro accepts for this particular tracepoint
+ in the user application's source code.
+* Its **output event fields**. They are the sources of event fields
+ that form the payload of any event that the execution of the
+ `tracepoint()` macro emits for this particular tracepoint.
+
+You can create a tracepoint definition by using the
+`TRACEPOINT_EVENT()` macro below the `#include <lttng/tracepoint.h>`
+line in the
+<<tpp-header,tracepoint provider header file template>>.
+
+The syntax of the `TRACEPOINT_EVENT()` macro is:
+
+[source,c]
+.`TRACEPOINT_EVENT()` macro syntax.
+----
+TRACEPOINT_EVENT(
+ /* Tracepoint provider name */
+ provider_name,
+
+ /* Tracepoint name */
+ tracepoint_name,
+
+ /* Input arguments */
+ TP_ARGS(
+ arguments
+ ),
+
+ /* Output event fields */
+ TP_FIELDS(
+ fields
+ )
+)
+----
+
+Replace:
+
+* `provider_name` with your tracepoint provider name.
+* `tracepoint_name` with your tracepoint name.
+* `arguments` with the <<tpp-def-input-args,input arguments>>.
+* `fields` with the <<tpp-def-output-fields,output event field>>
+ definitions.
+
+This tracepoint emits events named `provider_name:tracepoint_name`.
+
+[IMPORTANT]
+.Event name's length limitation
+====
+The concatenation of the tracepoint provider name and the
+tracepoint name must not exceed **254 characters**. If it does, the
+instrumented application compiles and runs, but LTTng throws multiple
+warnings and you could experience serious issues.
+====
+
+[[tpp-def-input-args]]The syntax of the `TP_ARGS()` macro is:
+
+[source,c]
+.`TP_ARGS()` macro syntax.
+----
+TP_ARGS(
+ type, arg_name
+)
+----
+
+Replace:
+
+* `type` with the C type of the argument.
+* `arg_name` with the argument name.
+
+You can repeat `type` and `arg_name` up to 10 times to have
+more than one argument.
+
+.`TP_ARGS()` usage with three arguments.
+====
+[source,c]
+----
+TP_ARGS(
+ int, count,
+ float, ratio,
+ const char*, query
+)
+----
+====
+
+The `TP_ARGS()` and `TP_ARGS(void)` forms are valid to create a
+tracepoint definition with no input arguments.
+
+[[tpp-def-output-fields]]The `TP_FIELDS()` macro contains a list of
+`ctf_*()` macros. Each `ctf_*()` macro defines one event field.
+See <<liblttng-ust-tp-fields,Tracepoint fields macros>> for a
+complete description of the available `ctf_*()` macros.
+A `ctf_*()` macro specifies the type, size, and byte order of
+one event field.
+
+Each `ctf_*()` macro takes an _argument expression_ parameter. This is a
+C expression that the tracer evalutes at the `tracepoint()` macro site
+in the application's source code. This expression provides a field's
+source of data. The argument expression can include input argument names
+listed in the `TP_ARGS()` macro.
+
+Each `ctf_*()` macro also takes a _field name_ parameter. Field names
+must be unique within a given tracepoint definition.
+
+Here's a complete tracepoint definition example:
+
+.Tracepoint definition.
+====
+The following tracepoint definition defines a tracepoint which takes
+three input arguments and has four output event fields.
+
+[source,c]
+----
+#include "my-custom-structure.h"
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ const struct my_custom_structure*, my_custom_structure,
+ float, ratio,
+ const char*, query
+ ),
+ TP_FIELDS(
+ ctf_string(query_field, query)
+ ctf_float(double, ratio_field, ratio)
+ ctf_integer(int, recv_size, my_custom_structure->recv_size)
+ ctf_integer(int, send_size, my_custom_structure->send_size)
+ )
+)
+----
+
+You can refer to this tracepoint definition with the `tracepoint()`
+macro in your application's source code like this:
+
+[source,c]
+----
+tracepoint(my_provider, my_tracepoint,
+ my_structure, some_ratio, the_query);
+----
+====
+
+NOTE: The LTTng tracer only evaluates tracepoint arguments at run time
+if they satisfy an enabled <<event,event rule>>.
+
+
+[[using-tracepoint-classes]]
+===== Use a tracepoint class
+
+A _tracepoint class_ is a class of tracepoints which share the same
+output event field definitions. A _tracepoint instance_ is one
+instance of such a defined tracepoint class, with its own tracepoint
+name.
+
+The <<defining-tracepoints,`TRACEPOINT_EVENT()` macro>> is actually a
+shorthand which defines both a tracepoint class and a tracepoint
+instance at the same time.
+
+When you build a tracepoint provider package, the C or $$C++$$ compiler
+creates one serialization function for each **tracepoint class**. A
+serialization function is responsible for serializing the event fields
+of a tracepoint to a sub-buffer when tracing.
+
+For various performance reasons, when your situation requires multiple
+tracepoint definitions with different names, but with the same event
+fields, we recommend that you manually create a tracepoint class
+and instantiate as many tracepoint instances as needed. One positive
+effect of such a design, amongst other advantages, is that all
+tracepoint instances of the same tracepoint class reuse the same
+serialization function, thus reducing
+https://en.wikipedia.org/wiki/Cache_pollution[cache pollution].
+
+.Use a tracepoint class and tracepoint instances.
+====
+Consider the following three tracepoint definitions:
+
+[source,c]
+----
+TRACEPOINT_EVENT(
+ my_app,
+ get_account,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+TRACEPOINT_EVENT(
+ my_app,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+----
+
+In this case, we create three tracepoint classes, with one implicit
+tracepoint instance for each of them: `get_account`, `get_settings`, and
+`get_transaction`. However, they all share the same event field names
+and types. Hence three identical, yet independent serialization
+functions are created when you build the tracepoint provider package.
+
+A better design choice is to define a single tracepoint class and three
+tracepoint instances:
+
+[source,c]
+----
+/* The tracepoint class */
+TRACEPOINT_EVENT_CLASS(
+ /* Tracepoint provider name */
+ my_app,
+
+ /* Tracepoint class name */
+ my_class,
+
+ /* Input arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+
+ /* Output event fields */
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+/* The tracepoint instances */
+TRACEPOINT_EVENT_INSTANCE(
+ /* Tracepoint provider name */
+ my_app,
+
+ /* Tracepoint class name */
+ my_class,
+
+ /* Tracepoint name */
+ get_account,
+
+ /* Input arguments */
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_settings,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+TRACEPOINT_EVENT_INSTANCE(
+ my_app,
+ my_class,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ )
+)
+----
+====
+
+
+[[assigning-log-levels]]
+===== Assign a log level to a tracepoint definition
+
+You can assign an optional _log level_ to a
+<<defining-tracepoints,tracepoint definition>>.
+
+Assigning different levels of severity to tracepoint definitions can
+be useful: when you <<enabling-disabling-events,create an event rule>>,
+you can target tracepoints having a log level as severe as a specific
+value.
+
+The concept of LTTng-UST log levels is similar to the levels found
+in typical logging frameworks:
+
+* In a logging framework, the log level is given by the function
+ or method name you use at the log statement site: `debug()`,
+ `info()`, `warn()`, `error()`, and so on.
+* In LTTng-UST, you statically assign the log level to a tracepoint
+ definition; any `tracepoint()` macro invocation which refers to
+ this definition has this log level.
+
+You can assign a log level to a tracepoint definition with the
+`TRACEPOINT_LOGLEVEL()` macro. You must use this macro _after_ the
+<<defining-tracepoints,`TRACEPOINT_EVENT()`>> or
+<<using-tracepoint-classes,`TRACEPOINT_INSTANCE()`>> macro for a given
+tracepoint.
+
+The syntax of the `TRACEPOINT_LOGLEVEL()` macro is:
+
+[source,c]
+.`TRACEPOINT_LOGLEVEL()` macro syntax.
+----
+TRACEPOINT_LOGLEVEL(provider_name, tracepoint_name, log_level)
+----
+
+Replace:
+
+* `provider_name` with the tracepoint provider name.
+* `tracepoint_name` with the tracepoint name.
+* `log_level` with the log level to assign to the tracepoint
+ definition named `tracepoint_name` in the `provider_name`
+ tracepoint provider.
++
+See <<liblttng-ust-tracepoint-loglevel,Tracepoint log levels>> for
+a list of available log level names.
+
+.Assign the `TRACE_DEBUG_UNIT` log level to a tracepoint definition.
+====
+[source,c]
+----
+/* Tracepoint definition */
+TRACEPOINT_EVENT(
+ my_app,
+ get_transaction,
+ TP_ARGS(
+ int, userid,
+ size_t, len
+ ),
+ TP_FIELDS(
+ ctf_integer(int, userid, userid)
+ ctf_integer(size_t, len, len)
+ )
+)
+
+/* Log level assignment */
+TRACEPOINT_LOGLEVEL(my_app, get_transaction, TRACE_DEBUG_UNIT)
+----
+====
+
+
+[[tpp-source]]
+===== Create a tracepoint provider package source file
+
+A _tracepoint provider package source file_ is a C source file which
+includes a <<tpp-header,tracepoint provider header file>> to expand its
+macros into event serialization and other functions.
+
+You can always use the following tracepoint provider package source
+file template:
+
+[source,c]
+.Tracepoint provider package source file template.
+----
+#define TRACEPOINT_CREATE_PROBES
+
+#include "tp.h"
+----
+
+Replace `tp.h` with the name of your <<tpp-header,tracepoint provider
+header file>> name. You may also include more than one tracepoint
+provider header file here to create a tracepoint provider package
+holding more than one tracepoint providers.
+
+
+[[probing-the-application-source-code]]
+==== Add tracepoints to an application's source code
+
+Once you <<tpp-header,create a tracepoint provider header file>>, you
+can use the `tracepoint()` macro in your application's
+source code to insert the tracepoints that this header
+<<defining-tracepoints,defined>> defines.
+
+The `tracepoint()` macro takes at least two parameters: the tracepoint
+provider name and the tracepoint name. The corresponding tracepoint
+definition defines the other parameters.
+
+.`tracepoint()` usage.
+====
+The following <<defining-tracepoints,tracepoint definition>> defines a
+tracepoint which takes two input arguments and has two output event
+fields.
+
+[source,c]
+.Tracepoint provider header file.
+----
+#include "my-custom-structure.h"
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, argc,
+ const char*, cmd_name
+ ),
+ TP_FIELDS(
+ ctf_string(cmd_name, cmd_name)
+ ctf_integer(int, number_of_args, argc)
+ )
+)
+----
+
+You can refer to this tracepoint definition with the `tracepoint()`
+macro in your application's source code like this:
+
+[source,c]
+.Application's source file.
+----
+#include "tp.h"
+
+int main(int argc, char* argv[])
+{
+ tracepoint(my_provider, my_tracepoint, argc, argv[0]);
+
+ return 0;
+}
+----
+
+Note how the application's source code includes
+the tracepoint provider header file containing the tracepoint
+definitions to use, path:{tp.h}.
+====
+
+.`tracepoint()` usage with a complex tracepoint definition.
+====
+Consider this complex tracepoint definition, where multiple event
+fields refer to the same input arguments in their argument expression
+parameter:
+
+[source,c]
+.Tracepoint provider header file.
+----
+/* For `struct stat` */
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
+
+TRACEPOINT_EVENT(
+ my_provider,
+ my_tracepoint,
+ TP_ARGS(
+ int, my_int_arg,
+ char*, my_str_arg,
+ struct stat*, st
+ ),
+ TP_FIELDS(
+ ctf_integer(int, my_constant_field, 23 + 17)
+ ctf_integer(int, my_int_arg_field, my_int_arg)
+ ctf_integer(int, my_int_arg_field2, my_int_arg * my_int_arg)
+ ctf_integer(int, sum4_field, my_str_arg[0] + my_str_arg[1] +
+ my_str_arg[2] + my_str_arg[3])
+ ctf_string(my_str_arg_field, my_str_arg)
+ ctf_integer_hex(off_t, size_field, st->st_size)
+ ctf_float(double, size_dbl_field, (double) st->st_size)
+ ctf_sequence_text(char, half_my_str_arg_field, my_str_arg,
+ size_t, strlen(my_str_arg) / 2)
+ )
+)
+----
+
+You can refer to this tracepoint definition with the `tracepoint()`
+macro in your application's source code like this:
+
+[source,c]
+.Application's source file.
+----
+#define TRACEPOINT_DEFINE
+#include "tp.h"
+
+int main(void)
+{
+ struct stat s;
+
+ stat("/etc/fstab", &s);
+ tracepoint(my_provider, my_tracepoint, 23, "Hello, World!", &s);
+
+ return 0;
+}
+----
+
+If you look at the event record that LTTng writes when tracing this
+program, assuming the file size of path:{/etc/fstab} is 301{nbsp}bytes,
+it should look like this:
+
+.Event record fields
+|====
+|Field's name |Field's value
+|`my_constant_field` |40
+|`my_int_arg_field` |23
+|`my_int_arg_field2` |529
+|`sum4_field` |389
+|`my_str_arg_field` |`Hello, World!`
+|`size_field` |0x12d
+|`size_dbl_field` |301.0
+|`half_my_str_arg_field` |`Hello,`
+|====
+====
+
+Sometimes, the arguments you pass to `tracepoint()` are expensive to
+compute--they use the call stack, for example. To avoid this
+computation when the tracepoint is disabled, you can use the
+`tracepoint_enabled()` and `do_tracepoint()` macros.
+
+The syntax of the `tracepoint_enabled()` and `do_tracepoint()` macros
+is:
+
+[source,c]
+.`tracepoint_enabled()` and `do_tracepoint()` macros syntax.
+----
+tracepoint_enabled(provider_name, tracepoint_name)
+do_tracepoint(provider_name, tracepoint_name, ...)
+----
+
+Replace:
+
+* `provider_name` with the tracepoint provider name.
+* `tracepoint_name` with the tracepoint name.
+
+`tracepoint_enabled()` returns a non-zero value if the tracepoint named
+`tracepoint_name` from the provider named `provider_name` is enabled
+**at run time**.
+
+`do_tracepoint()` is like `tracepoint()`, except that it doesn't check
+if the tracepoint is enabled. Using `tracepoint()` with
+`tracepoint_enabled()` is dangerous since `tracepoint()` also contains
+the `tracepoint_enabled()` check, thus a race condition is
+possible in this situation:
+
+[source,c]
+.Possible race condition when using `tracepoint_enabled()` with `tracepoint()`.
+----
+if (tracepoint_enabled(my_provider, my_tracepoint)) {
+ stuff = prepare_stuff();
+}
+
+tracepoint(my_provider, my_tracepoint, stuff);
+----
+
+If the tracepoint is enabled after the condition, then `stuff` is not
+prepared: the emitted event will either contain wrong data, or the whole
+application could crash (segmentation fault, for example).
+
+NOTE: Neither `tracepoint_enabled()` nor `do_tracepoint()` have an
+`STAP_PROBEV()` call. If you need it, you must emit
+this call yourself.
+
+
+[[building-tracepoint-providers-and-user-application]]
+==== Build and link a tracepoint provider package and an application
+
+Once you have one or more <<tpp-header,tracepoint provider header
+files>> and a <<tpp-source,tracepoint provider package source file>>,
+you can create the tracepoint provider package by compiling its source
+file. From here, multiple build and run scenarios are possible. The
+following table shows common application and library configurations
+along with the required command lines to achieve them.
+
+In the following diagrams, we use the following file names:
+
+`app`::
+ Executable application.
+
+`app.o`::
+ Application's object file.
+
+`tpp.o`::
+ Tracepoint provider package object file.
+
+`tpp.a`::
+ Tracepoint provider package archive file.
+
+`libtpp.so`::
+ Tracepoint provider package shared object file.
+
+`emon.o`::
+ User library object file.
+
+`libemon.so`::
+ User library shared object file.
+
+The red star indicates that this object file is instrumented
+(contains code which uses the `tracepoint()` macro). The spring
+symbol between the application and a library means the application is
+linked with the library at build time.
+
+We assume that path:{.} is part of the env:LD_LIBRARY_PATH environment
+variable in the following instructions.
+
+[role="growable ust-scenarios",cols="asciidoc,asciidoc"]
+.Common tracepoint provider package scenarios.
+|====
+|Scenario |Instructions
+
+|
+The instrumented application is statically linked with
+the tracepoint provider package object.
+
+image::ust-sit+app-linked-with-tp-o+app-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-o.txt[]
+
+To build the instrumented application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o tpp.o -llttng-ust -ldl
+----
+--
+
+To run the instrumented application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The instrumented application is statically linked with the
+tracepoint provider package archive file.
+
+image::ust-sit+app-linked-with-tp-a+app-instrumented.png[]
+
+|
+To create the tracepoint provider package archive file:
+
+. Compile the <<tpp-source,tracepoint provider package source file>>:
++
+--
+[role="term"]
+----
+gcc -I. -c tpp.c
+----
+--
+
+. Create the tracepoint provider package archive file:
++
+--
+[role="term"]
+----
+ar rcs tpp.a tpp.o
+----
+--
+
+To build the instrumented application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o tpp.a -llttng-ust -ldl
+----
+--
+
+To run the instrumented application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The instrumented application is linked with the tracepoint provider
+package shared object.
+
+image::ust-sit+app-linked-with-tp-so+app-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -ldl -L. -ltpp
+----
+--
+
+To run the instrumented application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The tracepoint provider package shared object is preloaded before the
+instrumented application starts.
+
+image::ust-sit+tp-so-preloaded+app-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the
+ following lines:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -ldl
+----
+--
+
+To run the instrumented application with tracing support:
+
+* Preload the tracepoint provider package shared object and
+ start the application:
++
+--
+[role="term"]
+----
+LD_PRELOAD=./libtpp.so ./app
+----
+--
+
+To run the instrumented application without tracing support:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The instrumented application dynamically loads the tracepoint provider
+package shared object.
+
+See the <<dlclose-warning,warning about `dlclose()`>>.
+
+image::ust-sit+app-dlopens-tp-so+app-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the
+ following lines:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -ldl
+----
+--
+
+To run the instrumented application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application is linked with the instrumented user library.
+
+The instrumented user library is statically linked with the tracepoint
+provider package object file.
+
+image::ust-sit+app-linked-with-lib+lib-linked-with-tp-o+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-o-fpic.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o tpp.o -llttng-ust -ldl
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -L. -lemon
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application is linked with the instrumented user library.
+
+The instrumented user library is linked with the tracepoint provider
+package shared object.
+
+image::ust-sit+app-linked-with-lib+lib-linked-with-tp-so+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -L. -lemon
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The tracepoint provider package shared object is preloaded before the
+application starts.
+
+The application is linked with the instrumented user library.
+
+image::ust-sit+tp-so-preloaded+app-linked-with-lib+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -L. -lemon
+----
+--
+
+To run the application with tracing support:
+
+* Preload the tracepoint provider package shared object and
+ start the application:
++
+--
+[role="term"]
+----
+LD_PRELOAD=./libtpp.so ./app
+----
+--
+
+To run the application without tracing support:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application is linked with the instrumented user library.
+
+The instrumented user library dynamically loads the tracepoint provider
+package shared object.
+
+See the <<dlclose-warning,warning about `dlclose()`>>.
+
+image::ust-sit+app-linked-with-lib+lib-dlopens-tp-so+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -L. -lemon
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application dynamically loads the instrumented user library.
+
+The instrumented user library is linked with the tracepoint provider
+package shared object.
+
+See the <<dlclose-warning,warning about `dlclose()`>>.
+
+image::ust-sit+app-dlopens-lib+lib-linked-with-tp-so+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -ldl -L. -lemon
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application dynamically loads the instrumented user library.
+
+The instrumented user library dynamically loads the tracepoint provider
+package shared object.
+
+See the <<dlclose-warning,warning about `dlclose()`>>.
+
+image::ust-sit+app-dlopens-lib+lib-dlopens-tp-so+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -ldl -L. -lemon
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The tracepoint provider package shared object is preloaded before the
+application starts.
+
+The application dynamically loads the instrumented user library.
+
+image::ust-sit+tp-so-preloaded+app-dlopens-lib+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-so.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o -ldl
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o -L. -lemon
+----
+--
+
+To run the application with tracing support:
+
+* Preload the tracepoint provider package shared object and
+ start the application:
++
+--
+[role="term"]
+----
+LD_PRELOAD=./libtpp.so ./app
+----
+--
+
+To run the application without tracing support:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application is statically linked with the tracepoint provider
+package object file.
+
+The application is linked with the instrumented user library.
+
+image::ust-sit+app-linked-with-tp-o+app-linked-with-lib+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-o.txt[]
+
+To build the instrumented user library:
+
+. In path:{emon.c}, before including path:{tpp.h}, add the
+ following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o
+----
+--
+
+To build the application:
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -o app app.o tpp.o -llttng-ust -ldl -L. -lemon
+----
+--
+
+To run the instrumented application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+
+|
+The application is statically linked with the tracepoint provider
+package object file.
+
+The application dynamically loads the instrumented user library.
+
+image::ust-sit+app-linked-with-tp-o+app-dlopens-lib+lib-instrumented.png[]
+
+|
+include::../common/ust-sit-step-tp-o.txt[]
+
+To build the application:
+
+. In path:{app.c}, before including path:{tpp.h}, add the following line:
++
+--
+[source,c]
+----
+#define TRACEPOINT_DEFINE
+----
+--
+
+. Compile the application source file:
++
+--
+[role="term"]
+----
+gcc -c app.c
+----
+--
+
+. Build the application:
++
+--
+[role="term"]
+----
+gcc -Wl,--export-dynamic -o app app.o tpp.o \
+ -llttng-ust -ldl
+----
+--
++
+The `--export-dynamic` option passed to the linker is necessary for the
+dynamically loaded library to ``see'' the tracepoint symbols defined in
+the application.
+
+To build the instrumented user library:
+
+. Compile the user library source file:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c emon.c
+----
+--
+
+. Build the user library shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libemon.so emon.o
+----
+--
+
+To run the application:
+
+* Start the application:
++
+--
+[role="term"]
+----
+./app
+----
+--
+|====
+
+[[dlclose-warning]]
+[IMPORTANT]
+.Do not use man:dlclose(3) on a tracepoint provider package
+====
+Never use man:dlclose(3) on any shared object which:
+
+* Is linked with, statically or dynamically, a tracepoint provider
+ package.
+* Calls man:dlopen(3) itself to dynamically open a tracepoint provider
+ package shared object.
+
+This is currently considered **unsafe** due to a lack of reference
+counting from LTTng-UST to the shared object.
+
+A known workaround (available since glibc 2.2) is to use the
+`RTLD_NODELETE` flag when calling man:dlopen(3) initially. This has the
+effect of not unloading the loaded shared object, even if man:dlclose(3)
+is called.
+
+You can also preload the tracepoint provider package shared object with
+the env:LD_PRELOAD environment variable to overcome this limitation.
+====
+
+
+[[using-lttng-ust-with-daemons]]
+===== Use noch:{LTTng-UST} with daemons
+
+If your instrumented application calls man:fork(2), man:clone(2),
+or BSD's man:rfork(2), without a following man:exec(3)-family
+system call, you must preload the path:{liblttng-ust-fork.so} shared
+object when starting the application.
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-fork.so ./my-app
+----
+
+If your tracepoint provider package is
+a shared library which you also preload, you must put both
+shared objects in env:LD_PRELOAD:
+
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-fork.so:/path/to/tp.so ./my-app
+----
+
+
+[[lttng-ust-pkg-config]]
+===== Use noch:{pkg-config}
+
+On some distributions, LTTng-UST ships with a
+https://www.freedesktop.org/wiki/Software/pkg-config/[pkg-config]
+metadata file. If this is your case, then you can use cmd:pkg-config to
+build an application on the command line:
+
+[role="term"]
+----
+gcc -o my-app my-app.o tp.o $(pkg-config --cflags --libs lttng-ust)
+----
+
+
+[[instrumenting-32-bit-app-on-64-bit-system]]
+===== [[advanced-instrumenting-techniques]]Build a 32-bit instrumented application for a 64-bit target system
+
+In order to trace a 32-bit application running on a 64-bit system,
+LTTng must use a dedicated 32-bit
+<<lttng-consumerd,consumer daemon>>.
+
+The following steps show how to build and install a 32-bit consumer
+daemon, which is _not_ part of the default 64-bit LTTng build, how to
+build and install the 32-bit LTTng-UST libraries, and how to build and
+link an instrumented 32-bit application in that context.
+
+To build a 32-bit instrumented application for a 64-bit target system,
+assuming you have a fresh target system with no installed Userspace RCU
+or LTTng packages:
+
+. Download, build, and install a 32-bit version of Userspace RCU:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/urcu/userspace-rcu-latest-0.9.tar.bz2 &&
+tar -xf userspace-rcu-latest-0.9.tar.bz2 &&
+cd userspace-rcu-0.9.* &&
+./configure --libdir=/usr/local/lib32 CFLAGS=-m32 &&
+make &&
+sudo make install &&
+sudo ldconfig
+----
+--
+
+. Using your distribution's package manager, or from source, install
+ the following 32-bit versions of the following dependencies of
+ LTTng-tools and LTTng-UST:
++
+--
+* https://sourceforge.net/projects/libuuid/[libuuid]
+* http://directory.fsf.org/wiki/Popt[popt]
+* http://www.xmlsoft.org/[libxml2]
+--
+
+. Download, build, and install a 32-bit version of the latest
+ LTTng-UST{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.7.tar.bz2 &&
+tar -xf lttng-ust-latest-2.7.tar.bz2 &&
+cd lttng-ust-2.7.* &&
+./configure --libdir=/usr/local/lib32 \
+ CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' &&
+make &&
+sudo make install &&
+sudo ldconfig
+----
+--
++
+[NOTE]
+====
+Depending on your distribution,
+32-bit libraries could be installed at a different location than
+`/usr/lib32`. For example, Debian is known to install
+some 32-bit libraries in `/usr/lib/i386-linux-gnu`.
+
+In this case, make sure to set `LDFLAGS` to all the
+relevant 32-bit library paths, for example:
+
+[role="term"]
+----
+LDFLAGS='-L/usr/lib/i386-linux-gnu -L/usr/lib32'
+----
+====
+
+. Download the latest LTTng-tools{nbsp}{revision}, build, and install
+ the 32-bit consumer daemon:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.7.tar.bz2 &&
+tar -xf lttng-tools-latest-2.7.tar.bz2 &&
+cd lttng-tools-2.7.* &&
+./configure --libdir=/usr/local/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
+ LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' &&
+make &&
+cd src/bin/lttng-consumerd &&
+sudo make install &&
+sudo ldconfig
+----
+--
+
+. From your distribution or from source,
+ <<installing-lttng,install>> the 64-bit versions of
+ LTTng-UST and Userspace RCU.
+. Download, build, and install the 64-bit version of the
+ latest LTTng-tools{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.7.tar.bz2 &&
+tar -xf lttng-tools-latest-2.7.tar.bz2 &&
+cd lttng-tools-2.7.* &&
+./configure --with-consumerd32-libdir=/usr/local/lib32 \
+ --with-consumerd32-bin=/usr/local/lib32/lttng/libexec/lttng-consumerd &&
+make &&
+sudo make install &&
+sudo ldconfig
+----
+--
+
+. Pass the following options to man:gcc(1), man:g++(1), or man:clang(1)
+ when linking your 32-bit application:
++
+----
+-m32 -L/usr/lib32 -L/usr/local/lib32 \
+-Wl,-rpath,/usr/lib32,-rpath,/usr/local/lib32
+----
++
+For example, let's rebuild the quick start example in
+<<tracing-your-own-user-application,Trace a user application>> as an
+instrumented 32-bit application:
++
+--
+[role="term"]
+----
+gcc -m32 -c -I. hello-tp.c
+gcc -m32 -c hello.c
+gcc -m32 -o hello hello.o hello-tp.o \
+ -L/usr/lib32 -L/usr/local/lib32 \
+ -Wl,-rpath,/usr/lib32,-rpath,/usr/local/lib32 \
+ -llttng-ust -ldl
+----
+--
+
+No special action is required to execute the 32-bit application and
+to trace it: use the command-line man:lttng(1) tool as usual.
+
+
+[role="since-2.5"]
+[[tracef]]
+==== Use `tracef()`
+
+`tracef()` is a small LTTng-UST API designed for quick,
+man:printf(3)-like instrumentation without the burden of
+<<tracepoint-provider,creating>> and
+<<building-tracepoint-providers-and-user-application,building>>
+a tracepoint provider package.
+
+To use `tracef()` in your application:
+
+. In the C or C++ source files where you need to use `tracef()`,
+ include `<lttng/tracef.h>`:
++
+--
+[source,c]
+----
+#include <lttng/tracef.h>
+----
+--
+
+. In the application's source code, use `tracef()` like you would use
+ man:printf(3):
++
+--
+[source,c]
+----
+ /* ... */
+
+ tracef("my message: %d (%s)", my_integer, my_string);
+
+ /* ... */
+----
+--
+
+. Link your application with `liblttng-ust`:
++
+--
+[role="term"]
+----
+gcc -o app app.c -llttng-ust
+----
+--
+
+To trace the events that `tracef()` calls emit:
+
+* <<enabling-disabling-events,Create an event rule>> which matches the
+ `lttng_ust_tracef:*` event name:
++
+--
+[role="term"]
+----
+lttng enable-event --userspace 'lttng_ust_tracef:*'
+----
+--
+
+[IMPORTANT]
+.Limitations of `tracef()`
+====
+The `tracef()` utility function was developed to make user space tracing
+super simple, albeit with notable disadvantages compared to
+<<defining-tracepoints,user-defined tracepoints>>:
+
+* All the emitted events have the same tracepoint provider and
+ tracepoint names, respectively `lttng_ust_tracef` and `event`.
+* There is no static type checking.
+* The only event record field you actually get, named `msg`, is a string
+ potentially containing the values you passed to `tracef()`
+ using your own format string. This also means that you cannot filter
+ events with a custom expression at run time because there are no
+ isolated fields.
+* Since `tracef()` uses the C standard library's man:vasprintf(3)
+ function behind the scenes to format the strings at run time, its
+ expected performance is lower than with user-defined tracepoints,
+ which do not require a conversion to a string.
+
+Taking this into consideration, `tracef()` is useful for some quick
+prototyping and debugging, but you should not consider it for any
+permanent and serious applicative instrumentation.
+====
+
+
+[role="since-2.7"]
+[[tracelog]]
+==== Use `tracelog()`
+
+The `tracelog()` API is very similar to <<tracef,`tracef()`>>, with
+the difference that it accepts an additional log level parameter.
+
+The goal of `tracelog()` is to ease the migration from logging to
+tracing.
+
+To use `tracelog()` in your application:
+
+. In the C or C++ source files where you need to use `tracelog()`,
+ include `<lttng/tracelog.h>`:
++
+--
+[source,c]
+----
+#include <lttng/tracelog.h>
+----
+--
+
+. In the application's source code, use `tracelog()` like you would use
+ man:printf(3), except for the first parameter which is the log
+ level:
++
+--
+[source,c]
+----
+ /* ... */
+
+ tracelog(TRACE_WARNING, "my message: %d (%s)",
+ my_integer, my_string);
+
+ /* ... */
+----
+--
++
+See <<liblttng-ust-tracepoint-loglevel,Tracepoint log levels>> for
+a list of available log level names.
+
+. Link your application with `liblttng-ust`:
++
+--
+[role="term"]
+----
+gcc -o app app.c -llttng-ust
+----
+--
+
+To trace the events that `tracelog()` calls emit with a log level
+_as severe as_ a specific log level:
+
+* <<enabling-disabling-events,Create an event rule>> which matches the
+ `lttng_ust_tracelog:*` event name and a minimum level
+ of severity:
++
+--
+[role="term"]
+----
+lttng enable-event --userspace 'lttng_ust_tracelog:*'
+ --loglevel=TRACE_WARNING
+----
+--
+
+To trace the events that `tracelog()` calls emit with a
+_specific log level_:
+
+* Create an event rule which matches the `lttng_ust_tracelog:*`
+ event name and a specific log level:
++
+--
+[role="term"]
+----
+lttng enable-event --userspace 'lttng_ust_tracelog:*'
+ --loglevel-only=TRACE_INFO
+----
+--
+
+
+[[prebuilt-ust-helpers]]
+=== Prebuilt user space tracing helpers
+
+The LTTng-UST package provides a few helpers in the form or preloadable
+shared objects which automatically instrument system functions and
+calls.
+
+The helper shared objects are normally found in dir:{/usr/lib}. If you
+built LTTng-UST <<building-from-source,from source>>, they are probably
+located in dir:{/usr/local/lib}.
+
+The installed user space tracing helpers in LTTng-UST{nbsp}{revision}
+are:
+
+path:{liblttng-ust-libc-wrapper.so}::
+path:{liblttng-ust-pthread-wrapper.so}::
+ <<liblttng-ust-libc-pthread-wrapper,C{nbsp}standard library
+ memory and POSIX threads function tracing>>.
+
+path:{liblttng-ust-cyg-profile.so}::
+path:{liblttng-ust-cyg-profile-fast.so}::
+ <<liblttng-ust-cyg-profile,Function entry and exit tracing>>.
+
+path:{liblttng-ust-dl.so}::
+ <<liblttng-ust-dl,Dynamic linker tracing>>.
+
+To use a user space tracing helper with any user application:
+
+* Preload the helper shared object when you start the application:
++
+--
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
+----
+--
++
+You can preload more than one helper:
++
+--
+[role="term"]
+----
+LD_PRELOAD=liblttng-ust-libc-wrapper.so:liblttng-ust-dl.so my-app
+----
+--
+
+
+[role="since-2.3"]
+[[liblttng-ust-libc-pthread-wrapper]]
+==== Instrument C standard library memory and POSIX threads functions
+
+The path:{liblttng-ust-libc-wrapper.so} and
+path:{liblttng-ust-pthread-wrapper.so} helpers
+add instrumentation to some C standard library and POSIX
+threads functions.
+
+[role="growable"]
+.Functions instrumented by preloading path:{liblttng-ust-libc-wrapper.so}.
+|====
+|TP provider name |TP name |Instrumented function
+
+.6+|`lttng_ust_libc` |`malloc` |man:malloc(3)
+ |`calloc` |man:calloc(3)
+ |`realloc` |man:realloc(3)
+ |`free` |man:free(3)
+ |`memalign` |man:memalign(3)
+ |`posix_memalign` |man:posix_memalign(3)
+|====
+
+[role="growable"]
+.Functions instrumented by preloading path:{liblttng-ust-pthread-wrapper.so}.
+|====
+|TP provider name |TP name |Instrumented function
+
+.4+|`lttng_ust_pthread` |`pthread_mutex_lock_req` |man:pthread_mutex_lock(3p) (request time)
+ |`pthread_mutex_lock_acq` |man:pthread_mutex_lock(3p) (acquire time)
+ |`pthread_mutex_trylock` |man:pthread_mutex_trylock(3p)
+ |`pthread_mutex_unlock` |man:pthread_mutex_unlock(3p)
+|====
+
+When you preload the shared object, it replaces the functions listed
+in the previous tables by wrappers which contain tracepoints and call
+the replaced functions.
+
+
+[[liblttng-ust-cyg-profile]]
+==== Instrument function entry and exit
+
+The path:{liblttng-ust-cyg-profile*.so} helpers can add instrumentation
+to the entry and exit points of functions.
+
+man:gcc(1) and man:clang(1) have an option named
+https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html[`-finstrument-functions`]
+which generates instrumentation calls for entry and exit to functions.
+The LTTng-UST function tracing helpers,
+path:{liblttng-ust-cyg-profile.so} and
+path:{liblttng-ust-cyg-profile-fast.so}, take advantage of this feature
+to add tracepoints to the two generated functions (which contain
+`cyg_profile` in their names, hence the helper's name).
+
+To use the LTTng-UST function tracing helper, the source files to
+instrument must be built using the `-finstrument-functions` compiler
+flag.
+
+There are two versions of the LTTng-UST function tracing helper:
+
+* **path:{liblttng-ust-cyg-profile-fast.so}** is a lightweight variant
+ that you should only use when it can be _guaranteed_ that the
+ complete event stream is recorded without any lost event record.
+ Any kind of duplicate information is left out.
++
+This version contains the following tracepoints:
++
+--
+[role="growable"]
+.Points instrumented by preloading path:{liblttng-ust-cyg-profile-fast.so}.
+|====
+|TP provider name |TP name |Instrumented points
+
+.2+|`lttng_ust_cyg_profile_fast`
+
+|`func_entry`
+a|Function entry.
+
+`addr`::
+ Address of called function.
+
+|`func_exit`
+|Function exit.
+|====
+--
++
+Assuming no event record is lost, having only the function addresses on
+entry is enough to create a call graph, since an event record always
+contains the ID of the CPU that generated it.
++
+You can use a tool like
+https://sourceware.org/binutils/docs/binutils/addr2line.html[cmd:addr2line]
+to convert function addresses back to source file names and
+line numbers.
+
+* **path:{liblttng-ust-cyg-profile.so}** is a more robust variant
+which also works in use cases where event records might get discarded or
+not recorded from application startup.
+In these cases, the trace analyzer needs more information to be
+able to reconstruct the program flow.
++
+This version contains the following tracepoints:
++
+--
+[role="growable"]
+.Points instrumented by preloading path:{liblttng-ust-cyg-profile.so}.
+|====
+|TP provider name |TP name |Instrumented point
+
+.2+|`lttng_ust_cyg_profile`
+
+|`func_entry`
+a|Function entry.
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+
+|`func_exit`
+a|Function exit.
+
+`addr`::
+ Address of called function.
+
+`call_site`::
+ Call site address.
+|====
+--
+
+TIP: It's sometimes a good idea to limit the number of source files that
+you compile with the `-finstrument-functions` option to prevent LTTng
+from writing an excessive amount of trace data at run time. When using
+man:gcc(1), you can use the
+`-finstrument-functions-exclude-function-list` option to avoid
+instrument entries and exits of specific function names.
+
+All the tracepoints that this helper contains have the
+<<liblttng-ust-tracepoint-loglevel,log level>> `TRACE_DEBUG_FUNCTION`.
+
+
+[role="since-2.4"]
+[[liblttng-ust-dl]]
+==== Instrument the dynamic linker
+
+The path:{liblttng-ust-dl.so} helper adds instrumentation to the
+man:dlopen(3) and man:dlclose(3) function calls.
+
+[role="growable"]
+.Functions instrumented by preloading path:{liblttng-ust-dl.so}.
+|====
+|TP provider name |TP name |Instrumented function
+
+.2+|`lttng_ust_dl`
+
+|`dlopen`
+a|man:dlopen(3)
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+
+`sopath`::
+ File system path to the loaded shared object.
+
+`size`::
+ File size of the the loaded shared object.
+
+`mtime`::
+ Last modification time (seconds since Epoch time) of the loaded shared
+ object.
+
+|`dlclose`
+a|man:dlclose(3)
+
+`baddr`::
+ Memory base address (where the dynamic linker placed the shared
+ object).
+|====
+
+
+[role="since-2.4"]
+[[java-application]]
+=== User space Java agent
+
+You can instrument a Java application which uses one of the following
+logging frameworks:
+
+* The https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[**`java.util.logging`**]
+ (JUL) core logging facilities.
+* http://logging.apache.org/log4j/1.2/[**Apache log4j 1.2**], since
+ LTTng 2.6. Note that Apache Log4j{nbsp}2 is not supported.
+
+Each log statement emits an LTTng event once the
+application initializes the <<lttng-ust-agents,LTTng-UST Java agent>>
+package.
+
+[role="img-100"]
+.LTTng-UST Java agent imported by a Java application.
+image::java-app.png[]
+
+NOTE: We use http://openjdk.java.net/[OpenJDK] 7 for development and
+https://ci.lttng.org/[continuous integration], thus this version is
+directly supported. However, the LTTng-UST Java agent is also
+tested with OpenJDK 6.
+
+To use the LTTng-UST Java agent:
+
+. In the Java application's source code, import the LTTng-UST Java
+ agent:
++
+--
+[source,java]
+----
+import org.lttng.ust.agent.LTTngAgent;
+----
+--
+
+. As soon as possible after the entry point of the application,
+ initialize the LTTng-UST Java agent:
++
+--
+[source,java]
+----
+LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+----
+--
++
+Any log statement that the application executes before this
+initialization does not emit an LTTng event.
+
+. Use `java.util.logging` and/or log4j log statements and configuration
+ as usual. Since the LTTng-UST Java agent adds a handler to the _root_
+ loggers, you can trace any log statement from any logger.
+
+. Before exiting the application, dispose the LTTng-UST Java agent:
++
+--
+[source,java]
+----
+lttngAgent.dispose();
+----
+--
++
+This is not strictly necessary, but it is recommended for a clean
+disposal of the agent's resources.
++
+Any log statement that the application executes after this disposal does
+not emit an LTTng event.
+
+. Include the LTTng-UST Java agent's JAR file, path:{liblttng-ust-agent.jar},
+ in the
+ https://docs.oracle.com/javase/tutorial/essential/environment/paths.html[class path]
+ when building the Java application.
++
+path:{liblttng-ust-agent.jar} is typically located in
+dir:{/usr/share/java}.
++
+IMPORTANT: The LTTng-UST Java agent must be
+<<installing-lttng,installed>> for the logging framework your
+application uses.
+
+.[[jul]]Use the LTTng-UST Java agent with `java.util.logging`.
+====
+[source,java]
+.path:{Test.java}
+----
+import java.util.logging.Logger;
+import org.lttng.ust.agent.LTTngAgent;
+
+public class Test
+{
+ private static final int answer = 42;
+
+ public static void main(String[] argv) throws Exception
+ {
+ // Create a logger
+ Logger logger = Logger.getLogger("jello");
+
+ // Call this as soon as possible (before logging)
+ LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+
+ // Log at will!
+ logger.info("some info");
+ logger.warning("some warning");
+ Thread.sleep(500);
+ logger.finer("finer information; the answer is " + answer);
+ Thread.sleep(123);
+ logger.severe("error!");
+
+ // Not mandatory, but cleaner
+ lttngAgent.dispose();
+ }
+}
+----
+
+You can build this example like this:
+
+[role="term"]
+----
+javac -cp /usr/share/java/liblttng-ust-agent.jar Test.java
+----
+
+You can run the compiled class like this:
+
+[role="term"]
+----
+java -cp /usr/share/java/liblttng-ust-agent.jar:. Test
+----
+====
+
+.[[log4j]]Use the LTTng-UST Java agent with Apache log4j 1.2.
+====
+[source,java]
+.path:{Test.java}
+----
+import org.apache.log4j.Logger;
+import org.apache.log4j.BasicConfigurator;
+import org.lttng.ust.agent.LTTngAgent;
+
+public class Test
+{
+ private static final int answer = 42;
+
+ public static void main(String[] argv) throws Exception
+ {
+ // Create and configure a logger
+ Logger logger = Logger.getLogger(Test.class);
+ BasicConfigurator.configure();
+
+ // Call this as soon as possible (before logging)
+ LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
+
+ // Log at will!
+ logger.info("some info");
+ logger.warn("some warning");
+ Thread.sleep(500);
+ logger.debug("debug information; the answer is " + answer);
+ Thread.sleep(123);
+ logger.error("error!");
+ logger.fatal("fatal error!");
+
+ // Not mandatory, but cleaner
+ lttngAgent.dispose();
+ }
+}
+----
+
+You can build this example like this:
+
+[role="term"]
+----
+javac -cp /usr/share/java/liblttng-ust-agent.jar:$LOG4JCP Test.java
+----
+
+where `$LOG4JCP` is the path to log4j's JAR file.
+
+You can run the compiled class like this:
+
+[role="term"]
+----
+java -cp /usr/share/java/liblttng-ust-agent.jar:$LOG4JCP:. Test
+----
+====
+
+When you <<enabling-disabling-events,create an event rule>>, use the
+`--jul` (`java.util.logging`) or `--log4j` (log4j) option to target
+the desired Java
+<<domain,tracing domain>>. You can also use the `--loglevel` or
+`--loglevel-only` option to target a range of JUL/log4j log levels or a
+specific JUL/log4j log level.
+
+
+[role="since-2.7"]
+[[python-application]]
+=== User space Python agent
+
+You can instrument a Python 2 or Python 3 application which uses the
+standard https://docs.python.org/3/library/logging.html[`logging`]
+package.
+
+Each log statement emits an LTTng event once the
+application module imports the
+<<lttng-ust-agents,LTTng-UST Python agent>> package.
+
+[role="img-100"]
+.A Python application importing the LTTng-UST Python agent.
+image::python-app.png[]
+
+To use the LTTng-UST Python agent:
+
+. In the Python application's source code, import the LTTng-UST Python
+ agent:
++
+--
+[source,python]
+----
+import lttngust
+----
+--
++
+The LTTng-UST Python agent automatically adds its logging handler to the
+root logger at import time.
++
+Any log statement that the application executes before this import does
+not emit an LTTng event.
++
+IMPORTANT: The LTTng-UST Python agent must be
+<<installing-lttng,installed>>.
+
+. Use log statements and logging configuration as usual.
+ Since the LTTng-UST Python agent adds a handler to the _root_
+ logger, you can trace any log statement from any logger.
+
+.Use the LTTng-UST Python agent.
+====
+[source,python]
+----
+import lttngust
+import logging
+import time
+
+
+def example():
+ logging.basicConfig()
+ logger = logging.getLogger('my-logger')
+
+ while True:
+ logger.debug('debug message')
+ logger.info('info message')
+ logger.warn('warn message')
+ logger.error('error message')
+ logger.critical('critical message')
+ time.sleep(1)
+
+
+if __name__ == '__main__':
+ example()
+----
+
+NOTE: `logging.basicConfig()`, which adds to the root logger a basic
+logging handler which prints to the standard error stream, is not
+strictly required for LTTng-UST tracing to work, but in versions of
+Python preceding 3.2, you could see a warning message which indicates
+that no handler exists for the logger `my-logger`.
+====
+
+When you <<enabling-disabling-events,create an event rule>>, use the
+`--python` option to target the Python
+<<domain,tracing domain>>. You can also use
+the `--loglevel` or `--loglevel-only` option to target a range of
+Python log levels or a specific Python log level.
+
+When an application imports the LTTng-UST Python agent, the agent tries
+to register to a <<lttng-sessiond,session daemon>>. Note that you must
+start the session daemon _before_ you start the Python application.
+If a session daemon is found, the agent tries to register to it
+during 5{nbsp}seconds, after which the application continues without
+LTTng tracing support. You can override this timeout value with the
+env:LTTNG_UST_PYTHON_REGISTER_TIMEOUT environment variable
+(milliseconds).
+
+If the session daemon stops while a Python application with an imported
+LTTng-UST Python agent runs, the agent retries to connect and to
+register to a session daemon every 3{nbsp}seconds. You can override this
+delay with the env:LTTNG_UST_PYTHON_REGISTER_RETRY_DELAY environment
+variable.
+
+
+[role="since-2.5"]
+[[proc-lttng-logger-abi]]
+=== LTTng logger
+
+The `lttng-tracer` Linux kernel module, part of
+<<lttng-modules,LTTng-modules>>, creates the special LTTng logger file
+path:{/proc/lttng-logger} when it's loaded. Any application can write
+text data to this file to emit an LTTng event.
+
+[role="img-100"]
+.An application writes to the LTTng logger file to emit an LTTng event.
+image::lttng-logger.png[]
+
+The LTTng logger is the quickest method--not the most efficient,
+however--to add instrumentation to an application. It is designed
+mostly to instrument shell scripts:
+
+[role="term"]
+----
+echo "Some message, some $variable" > /proc/lttng-logger
+----
+
+Any event that the LTTng logger emits is named `lttng_logger` and
+belongs to the Linux kernel <<domain,tracing domain>>. However, unlike
+other instrumentation points in the kernel tracing domain, **any Unix
+user** can <<enabling-disabling-events,create an event rule>> which
+matches its event name, not only the root user or users in the tracing
+group.
+
+To use the LTTng logger:
+
+* From any application, write text data to the path:{/proc/lttng-logger}
+ file.
+
+The `msg` field of `lttng_logger` event records contains the
+recorded message.
+
+NOTE: The maximum message length of an LTTng logger event is
+1024{nbsp}bytes. Writing more than this makes the LTTng logger emit more
+than one event to contain the remaining data.
+
+You should not use the LTTng logger to trace a user application which
+can be instrumented in a more efficient way, namely:
+
+* <<c-application,C and $$C++$$ applications>>.
+* <<java-application,Java applications>>.
+* <<python-application,Python applications>>.
+
+
+[[instrumenting-linux-kernel]]
+=== LTTng kernel tracepoints
+
+NOTE: This section shows how to _add_ instrumentation points to the
+Linux kernel. The kernel's subsystems are already thoroughly
+instrumented at strategic places for LTTng when you
+<<installing-lttng,install>> the <<lttng-modules,LTTng-modules>>
+package.
+
+////
+There are two methods to instrument the Linux kernel:
+
+. <<linux-add-lttng-layer,Add an LTTng layer>> over an existing ftrace
+ tracepoint which uses the `TRACE_EVENT()` API.
++
+Choose this if you want to instrumentation a Linux kernel tree with an
+instrumentation point compatible with ftrace, perf, and SystemTap.
+
+. Use an <<linux-lttng-tracepoint-event,LTTng-only approach>> to
+ instrument an out-of-tree kernel module.
++
+Choose this if you don't need ftrace, perf, or SystemTap support.
+////
+
+
+[[linux-add-lttng-layer]]
+==== [[instrumenting-linux-kernel-itself]][[mainline-trace-event]][[lttng-adaptation-layer]]Add an LTTng layer to an existing ftrace tracepoint
+
+This section shows how to add an LTTng layer to existing ftrace
+instrumentation using the `TRACE_EVENT()` API.
+
+This section does not document the `TRACE_EVENT()` macro. You can
+read the following articles to learn more about this API:
+
+* http://lwn.net/Articles/379903/[Using the TRACE_EVENT() macro (Part 1)]
+* http://lwn.net/Articles/381064/[Using the TRACE_EVENT() macro (Part 2)]
+* http://lwn.net/Articles/383362/[Using the TRACE_EVENT() macro (Part 3)]
+
+The following procedure assumes that your ftrace tracepoints are
+correctly defined in their own header and that they are created in
+one source file using the `CREATE_TRACE_POINTS` definition.
+
+To add an LTTng layer over an existing ftrace tracepoint:
+
+. Make sure the following kernel configuration options are
+ enabled:
++
+--
+* `CONFIG_MODULES`
+* `CONFIG_KALLSYMS`
+* `CONFIG_HIGH_RES_TIMERS`
+* `CONFIG_TRACEPOINTS`
+--
+
+. Build the Linux source tree with your custom ftrace tracepoints.
+. Boot the resulting Linux image on your target system.
++
+Confirm that the tracepoints exist by looking for their names in the
+dir:{/sys/kernel/debug/tracing/events/subsys} directory, where `subsys`
+is your subsystem's name.
+
+. Get a copy of the latest LTTng-modules{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.8.tar.bz2 &&
+tar -xf lttng-modules-latest-2.8.tar.bz2 &&
+cd lttng-modules-2.8.*
+----
+--
+
+. In dir:{instrumentation/events/lttng-module}, relative to the root
+ of the LTTng-modules source tree, create a header file named
+ +__subsys__.h+ for your custom subsystem +__subsys__+ and write your
+ LTTng-modules tracepoint definitions using the LTTng-modules
+ macros in it.
++
+Start with this template:
++
+--
+[source,c]
+.path:{instrumentation/events/lttng-module/my_subsys.h}
+----
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM my_subsys
+
+#if !defined(_LTTNG_MY_SUBSYS_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _LTTNG_MY_SUBSYS_H
+
+#include "../../../probes/lttng-tracepoint-event.h"
+#include <linux/tracepoint.h>
+
+LTTNG_TRACEPOINT_EVENT(
+ /*
+ * Format is identical to TRACE_EVENT()'s version for the three
+ * following macro parameters:
+ */
+ my_subsys_my_event,
+ TP_PROTO(int my_int, const char *my_string),
+ TP_ARGS(my_int, my_string),
+
+ /* LTTng-modules specific macros */
+ TP_FIELDS(
+ ctf_integer(int, my_int_field, my_int)
+ ctf_string(my_bar_field, my_bar)
+ )
+)
+
+#endif /* !defined(_LTTNG_MY_SUBSYS_H) || defined(TRACE_HEADER_MULTI_READ) */
+
+#include "../../../probes/define_trace.h"
+----
+--
++
+The entries in the `TP_FIELDS()` section are the list of fields for the
+LTTng tracepoint. This is similar to the `TP_STRUCT__entry()` part of
+ftrace's `TRACE_EVENT()` macro.
++
+See <<lttng-modules-tp-fields,Tracepoint fields macros>> for a
+complete description of the available `ctf_*()` macros.
+
+. Create the LTTng-modules probe's kernel module C source file,
+ +probes/lttng-probe-__subsys__.c+, where +__subsys__+ is your
+ subsystem name:
++
+--
+[source,c]
+.path:{probes/lttng-probe-my-subsys.c}
+----
+#include <linux/module.h>
+#include "../lttng-tracer.h"
+
+/*
+ * Build-time verification of mismatch between mainline
+ * TRACE_EVENT() arguments and the LTTng-modules adaptation
+ * layer LTTNG_TRACEPOINT_EVENT() arguments.
+ */
+#include <trace/events/my_subsys.h>
+
+/* Create LTTng tracepoint probes */
+#define LTTNG_PACKAGE_BUILD
+#define CREATE_TRACE_POINTS
+#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module
+
+#include "../instrumentation/events/lttng-module/my_subsys.h"
+
+MODULE_LICENSE("GPL and additional rights");
+MODULE_AUTHOR("Your name <your-email>");
+MODULE_DESCRIPTION("LTTng my_subsys probes");
+MODULE_VERSION(__stringify(LTTNG_MODULES_MAJOR_VERSION) "."
+ __stringify(LTTNG_MODULES_MINOR_VERSION) "."
+ __stringify(LTTNG_MODULES_PATCHLEVEL_VERSION)
+ LTTNG_MODULES_EXTRAVERSION);
+----
+--
+
+. Edit path:{probes/Makefile} and add your new kernel module object
+ next to the existing ones:
++
+--
+[source,make]
+.path:{probes/Makefile}
+----
+# ...
+
+obj-m += lttng-probe-module.o
+obj-m += lttng-probe-power.o
+
+obj-m += lttng-probe-my-subsys.o
+
+# ...
+----
+--
+
+. Build and install the LTTng kernel modules:
++
+--
+[role="term"]
+----
+make KERNELDIR=/path/to/linux
+sudo make modules_install
+----
+--
++
+Replace `/path/to/linux` with the path to the Linux source tree where
+you defined and used tracepoints with ftrace's `TRACE_EVENT()` macro.
+
+Note that you can also use the
+<<lttng-tracepoint-event-code,`LTTNG_TRACEPOINT_EVENT_CODE()` macro>>
+instead of `LTTNG_TRACEPOINT_EVENT()` to use custom local variables and
+C code that need to be executed before the event fields are recorded.
+
+The best way to learn how to use the previous LTTng-modules macros is to
+inspect the existing LTTng-modules tracepoint definitions in the
+dir:{instrumentation/events/lttng-module} header files. Compare them
+with the Linux kernel mainline versions in the
+dir:{include/trace/events} directory of the Linux source tree.
+
+
+[role="since-2.7"]
+[[lttng-tracepoint-event-code]]
+===== Use custom C code to access the data for tracepoint fields
+
+Although we recommended to always use the
+<<lttng-adaptation-layer,`LTTNG_TRACEPOINT_EVENT()`>> macro to describe
+the arguments and fields of an LTTng-modules tracepoint when possible,
+sometimes you need a more complex process to access the data that the
+tracer records as event record fields. In other words, you need local
+variables and multiple C{nbsp}statements instead of simple
+argument-based expressions that you pass to the
+<<lttng-modules-tp-fields,`ctf_*()` macros of `TP_FIELDS()`>>.
+
+You can use the `LTTNG_TRACEPOINT_EVENT_CODE()` macro instead of
+`LTTNG_TRACEPOINT_EVENT()` to declare custom local variables and define
+a block of C{nbsp}code to be executed before LTTng records the fields.
+The structure of this macro is:
+
+[source,c]
+.`LTTNG_TRACEPOINT_EVENT_CODE()` macro syntax.
+----
+LTTNG_TRACEPOINT_EVENT_CODE(
+ /*
+ * Format identical to the LTTNG_TRACEPOINT_EVENT()
+ * version for the following three macro parameters:
+ */
+ my_subsys_my_event,
+ TP_PROTO(int my_int, const char *my_string),
+ TP_ARGS(my_int, my_string),
+
+ /* Declarations of custom local variables */
+ TP_locvar(
+ int a = 0;
+ unsigned long b = 0;
+ const char *name = "(undefined)";
+ struct my_struct *my_struct;
+ ),
+
+ /*
+ * Custom code which uses both tracepoint arguments
+ * (in TP_ARGS()) and local variables (in TP_locvar()).
+ *
+ * Local variables are actually members of a structure pointed
+ * to by the special variable tp_locvar.
+ */
+ TP_code(
+ if (my_int) {
+ tp_locvar->a = my_int + 17;
+ tp_locvar->my_struct = get_my_struct_at(tp_locvar->a);
+ tp_locvar->b = my_struct_compute_b(tp_locvar->my_struct);
+ tp_locvar->name = my_struct_get_name(tp_locvar->my_struct);
+ put_my_struct(tp_locvar->my_struct);
+
+ if (tp_locvar->b) {
+ tp_locvar->a = 1;
+ }
+ }
+ ),
+
+ /*
+ * Format identical to the LTTNG_TRACEPOINT_EVENT()
+ * version for this, except that tp_locvar members can be
+ * used in the argument expression parameters of
+ * the ctf_*() macros.
+ */
+ TP_FIELDS(
+ ctf_integer(unsigned long, my_struct_b, tp_locvar->b)
+ ctf_integer(int, my_struct_a, tp_locvar->a)
+ ctf_string(my_string_field, my_string)
+ ctf_string(my_struct_name, tp_locvar->name)
+ )
+)
+----
+
+IMPORTANT: The C code defined in `TP_code()` must not have any side
+effects when executed. In particular, the code must not allocate
+memory or get resources without deallocating this memory or putting
+those resources afterwards.
+
+
+[[instrumenting-linux-kernel-tracing]]
+==== Load and unload a custom probe kernel module
+
+You must load a <<lttng-adaptation-layer,created LTTng-modules probe
+kernel module>> in the kernel before it can emit LTTng events.
+
+To load the default probe kernel modules and a custom probe kernel
+module:
+
+* Use the `--extra-kmod-probes` option to give extra probe modules
+ to load when starting a root <<lttng-sessiond,session daemon>>:
++
+--
+.Load the `my_subsys`, `usb`, and the default probe modules.
+====
+[role="term"]
+----
+sudo lttng-sessiond --extra-kmod-probes=my_subsys,usb
+----
+====
+--
++
+You only need to pass the subsystem name, not the whole kernel module
+name.
+
+To load _only_ a given custom probe kernel module:
+
+* Use the `--kmod-probes` option to give the probe modules
+ to load when starting a root session daemon:
++
+--
+.Load only the `my_subsys` and `usb` probe modules.
+====
+[role="term"]
+----
+sudo lttng-sessiond --kmod-probes=my_subsys,usb
+----
+====
+--
+
+To confirm that a probe module is loaded:
+
+* Use man:lsmod(8):
++
+--
+[role="term"]
+----
+lsmod | grep lttng_probe_usb
+----
+--
+
+To unload the loaded probe modules:
+
+* Kill the session daemon with `SIGTERM`:
++
+--
+[role="term"]
+----
+sudo pkill lttng-sessiond
+----
+--
++
+You can also use man:modprobe(8)'s `--remove` option if the session
+daemon terminates abnormally.
+
+
+[[controlling-tracing]]
+== Tracing control
+
+Once an application or a Linux kernel is
+<<instrumenting,instrumented>> for LTTng tracing,
+you can _trace_ it.
+
+This section is divided in topics on how to use the various
+<<plumbing,components of LTTng>>, in particular the <<lttng-cli,cmd:lttng
+command-line tool>>, to _control_ the LTTng daemons and tracers.
+
+Note that the <<online-lttng-manpages,Online LTTng man pages>> are
+more comprehensive than the guides of this section. Refer to them if
+your use case is not included in this section.
+
+
+[[start-sessiond]]
+=== Start a session daemon
+
+In some situations, you need to run a <<lttng-sessiond,session daemon>>
+_before_ you can use the cmd:lttng command-line tool.
+
+You will see the following error when you run a command while no session
+daemon is running:
+
+----
+Error: No session daemon is available
+----
+
+The only command that automatically runs a session daemon is `create`,
+which you use to <<creating-destroying-tracing-sessions,create a tracing
+session>>. While this is most of the time the first operation that you
+do, sometimes it's not. Some examples are:
+
+* <<list-instrumentation-points,List the available instrumentation points>>.
+* <<saving-loading-tracing-session,Load a tracing session configuration>>.
+
+[[tracing-group]] Each Unix user must have its own running session
+daemon to trace user applications. The session daemon that the root user
+starts is the only one allowed to control the LTTng kernel tracer. Users
+that are part of the _tracing group_ can control the root session
+daemon. The default tracing group name is `tracing`; you can set it to
+something else with the `--group` option when you start the root session
+daemon.
+
+To start a user session daemon:
+
+* Run cmd:lttng-sessiond:
++
+--
+[role="term"]
+----
+lttng-sessiond --daemonize
+----
+--
+
+To start the root session daemon:
+
+* Run cmd:lttng-sessiond as the root user:
++
+--
+[role="term"]
+----
+sudo lttng-sessiond --daemonize
+----
+--
+
+In both cases, remove the `--daemonize` option to start the session
+daemon in foreground.
+
+To stop a session daemon, use cmd:kill on its process ID (standard
+`TERM` signal).
+
+Note that some Linux distributions could manage the LTTng session daemon
+as a service. In this case, you should use the service manager to
+start, restart, and stop session daemons.
+
+
+[[creating-destroying-tracing-sessions]]
+=== Create and destroy a tracing session
+
+Almost all the LTTng control operations happen in the scope of
+a <<tracing-session,tracing session>>, which is the dialogue between the
+<<lttng-sessiond,session daemon>> and you.
+
+To create a tracing session with a generated name:
+
+* Use the `create` command:
++
+--
+[role="term"]
+----
+lttng create
+----
+--
+
+The created tracing session's name is `auto` followed by the
+creation date.
+
+To create a tracing session with a specific name:
+
+* Use the optional argument of the `create` command:
++
+--
+[role="term"]
+----
+lttng create my-session
+----
+--
++
+Replace `my-session` with the specific tracing session name.
+
+LTTng appends the creation date to the created tracing session's name.
+
+LTTng writes the traces of a tracing session in
++$LTTNG_HOME/lttng-trace/__name__+ by default, where +__name__+ is the
+name of the tracing session. Note that the env:LTTNG_HOME environment
+variable defaults to `$HOME` if not set.
+
+To output LTTng traces to a non-default location:
+
+* Use the `--output` option of the `create` command:
++
+--
+[role="term"]
+----
+lttng create --output=/tmp/some-directory my-session
+----
+--
+
+You may create as many tracing sessions as you wish.
+
+To list all the existing tracing sessions for your Unix user:
+
+* Use the `list` command:
++
+--
+[role="term"]
+----
+lttng list
+----
+--
+
+When you create a tracing session, it is set as the _current tracing
+session_. The following man:lttng(1) commands operate on the current
+tracing session when you don't specify one:
+
+[role="list-3-cols"]
+* `add-context`
+* `destroy`
+* `disable-channel`
+* `disable-event`
+* `enable-channel`
+* `enable-event`
+* `load`
+* `save`
+* `snapshot`
+* `start`
+* `stop`
+* `track`
+* `untrack`
+* `view`
+
+To change the current tracing session:
+
+* Use the `set-session` command:
++
+--
+[role="term"]
+----
+lttng set-session new-session
+----
+--
++
+Replace `new-session` by the name of the new current tracing session.
+
+When you are done tracing in a given tracing session, you can destroy
+it. This operation frees the resources taken by the tracing session
+to destroy; it does not destroy the trace data that LTTng wrote for
+this tracing session.
+
+To destroy the current tracing session:
+
+* Use the `destroy` command:
++
+--
+[role="term"]
+----
+lttng destroy
+----
+--
+
+
+[[list-instrumentation-points]]
+=== List the available instrumentation points
+
+The <<lttng-sessiond,session daemon>> can query the running instrumented
+user applications and the Linux kernel to get a list of available
+instrumentation points. For the Linux kernel <<domain,tracing domain>>,
+they are tracepoints and system calls. For the user space tracing
+domain, they are tracepoints. For the other tracing domains, they are
+logger names.
+
+To list the available instrumentation points:
+
+* Use the `list` command with the requested tracing domain's option
+amongst:
++
+--
+* `--kernel`: Linux kernel tracepoints (your Unix user must be a root
+ user, or it must be a member of the tracing group).
+* `--kernel --syscall`: Linux kernel system calls (your Unix user must
+ be a root user, or it must be a member of the tracing group).
+* `--userspace`: user space tracepoints.
+* `--jul`: `java.util.logging` loggers.
+* `--log4j`: Apache log4j loggers.
+* `--python`: Python loggers.
+--
+
+.List the available user space tracepoints.
+====
+[role="term"]
+----
+lttng list --userspace
+----
+====
+
+.List the available Linux kernel system call tracepoints.
+====
+[role="term"]
+----
+lttng list --kernel --syscall
+----
+====
+
+
+[[enabling-disabling-events]]
+=== Create and enable an event rule
+
+Once you <<creating-destroying-tracing-sessions,create a tracing
+session>>, you can create <<event,event rules>> with the
+`enable-event` command.
+
+You specify each condition with a command-line option. The available
+condition options are shown in the following table.
+
+[role="growable",cols="asciidoc,asciidoc,default"]
+.Condition command-line options for the `enable-event` command.
+|====
+|Option |Description |Applicable tracing domains
+
+|
+One of:
+
+. `--syscall`
+. +--probe=__ADDR__+
+. +--function=__ADDR__+
+
+|
+Instead of using the default _tracepoint_ instrumentation type, use:
+
+. A Linux system call.
+. A Linux https://lwn.net/Articles/132196/[KProbe] (symbol or address).
+. The entry and return points of a Linux function (symbol or address).
+
+|Linux kernel.
+
+|First positional argument.
+
+|
+Tracepoint or system call name. In the case of a Linux KProbe or
+function, this is a custom name given to the event rule. With the
+JUL, log4j, and Python domains, this is a logger name.
+
+With a tracepoint, logger, or system call name, the last character
+can be `*` to match anything that remains.
+
+|All.
+
+|
+One of:
+
+. +--loglevel=__LEVEL__+
+. +--loglevel-only=__LEVEL__+
+
+|
+. Match only tracepoints or log statements with a logging level at
+ least as severe as +__LEVEL__+.
+. Match only tracepoints or log statements with a logging level
+ equal to +__LEVEL__+.
+
+You can get the list of available logging level names with
+`lttng enable-event --help`.
+
+|User space, JUL, log4j, and Python.
+
+|+--exclude=__EXCLUSIONS__+
+
+|
+When you use a `*` character at the end of the tracepoint or logger
+name (first positional argument), exclude the specific names in the
+comma-delimited list +__EXCLUSIONS__+.
+
+|
+User space, JUL, log4j, and Python.
+
+|+--filter=__EXPR__+
+
+|
+Match only events which satisfy the expression +__EXPR__+.
+
++__EXPR__+ is a C-like logical expression where identifiers are event
+fields (preceded with `$ctx.` for context fields). Nested expressions
+with `(` and `)`, and all the logical and comparison operators of the C
+language are supported. The precedence rules of those operators are the
+same as in the C language.
+
+When a comparison includes a non-existent event field, the whole filter
+expression evaluates to false.
+
+C integer and floating point number constants are supported, as well as
+literal strings between double quotes (`"`). Literal strings can
+contain a wildcard character (`*`) at the end to match anything that
+remains. This wildcard can be escaped using `\*`.
+
+Note that, although it is possible to use this option with the JUL,
+log4j, and Python tracing domains, the tracer evalutes the expression
+against the equivalent user space event.
+
+|All.
+
+|====
+
+See man:lttng(1)
+for more details about those command-line options.
+
+You attach an event rule to a <<channel,channel>> on creation. If you
+do not specify the channel with the `--channel` option, and if the event
+rule to create is the first in its <<domain,tracing domain>> for a given
+tracing session, then LTTng creates a _default channel_ for you. This
+default channel is reused in subsequent invocations of the
+`enable-event` command for the same tracing domain.
+
+An event rule is always enabled at creation time.
+
+The following examples show how you can combine the previous
+command-line options to create simple to more complex event rules.
+
+.Create an event rule targetting a Linux kernel tracepoint (default channel).
+====
+[role="term"]
+----
+lttng enable-event --kernel sched_switch
+----
+====
+
+.Create an event rule matching four Linux kernel system calls (default channel).
+====
+[role="term"]
+----
+lttng enable-event --kernel --syscall open,write,read,close
+----
+====
+
+.Create an event rule matching a Linux kernel tracepoint with a filter expression (default channel).
+====
+[role="term"]
+----
+lttng enable-event --kernel sched_switch --filter='prev_comm == "bash"'
+----
+
+IMPORTANT: Make sure to always quote the filter string when you
+use man:lttng(1) from a shell.
+====
+
+.Create an event rule matching any user space tracepoint of a given tracepoint provider with a log level range (default channel).
+====
+[role="term"]
+----
+lttng enable-event --userspace my_app:'*' --loglevel=TRACE_INFO
+----
+
+IMPORTANT: Make sure to always quote the wildcard character when you
+use man:lttng(1) from a shell.
+====
+
+.Create an event rule matching multiple Python loggers with a wildcard and with exclusions (default channel).
+====
+[role="term"]
+----
+lttng enable-event --python my-app.'*' \
+ --exclude='my-app.module,my-app.hello'
+----
+====
+
+.Create an event rule matching any Apache log4j logger with a specific log level (default channel).
+====
+[role="term"]
+----
+lttng enable-event --log4j --all --loglevel-only=LOG4J_WARN
+----
+====
+
+.Create an event rule attached to a specific channel matching a specific user space tracepoint provider and tracepoint.
+====
+[role="term"]
+----
+lttng enable-event --userspace my_app:my_tracepoint --channel=my-channel
+----
+====
+
+The event rules of a given channel form a whitelist: as soon as an
+emitted event passes one of them, LTTng can record the event. For
+example, an event named `my_app:my_tracepoint` emitted from a user space
+tracepoint with a `TRACE_ERROR` log level passes both of the following
+rules:
+
+[role="term"]
+----
+lttng enable-event --userspace my_app:my_tracepoint
+lttng enable-event --userspace my_app:my_tracepoint \
+ --loglevel=TRACE_INFO
+----
+
+The second event rule is redundant: the first one includes
+the second one.
+
+
+[[disable-event-rule]]
+=== Disable an event rule
+
+To disable an event rule that you <<enabling-disabling-events,created>>
+previously, use the `disable-event` command. This command disables _all_
+the event rules (of a given tracing domain and channel) which match an
+instrumentation point. The other conditions are not supported as of
+LTTng{nbsp}{revision}.
+
+The LTTng tracer does not record an emitted event which passes
+a _disabled_ event rule.
+
+.Disable an event rule matching a Python logger (default channel).
+====
+[role="term"]
+----
+lttng disable-event --python my-logger
+----
+====
+
+.Disable an event rule matching all `java.util.logging` loggers (default channel).
+====
+[role="term"]
+----
+lttng disable-event --jul '*'
+----
+====
+
+.Disable _all_ the event rules of the default channel.
+====
+The `--all-events` option is not, like the `--all` option of
+`enable-event`, the equivalent of the event name `*` (wildcard): it
+disables _all_ the event rules of a given channel.
+
+[role="term"]
+----
+lttng disable-event --jul --all-events
+----
+====
+
+NOTE: You cannot delete an event rule once you create it.
+
+
+[[status]]
+=== Get the status of a tracing session
+
+To get the status of a tracing session, that is, its channels, event
+rules, and their attributes:
+
+* Use the `list` command with the tracing session's name:
++
+--
+[role="term"]
+----
+lttng list my-session
+----
+--
++
+Replace `my-session` with your tracing session's name.
+
+
+[[basic-tracing-session-control]]
+=== Start and stop a tracing session
+
+Once you <<creating-destroying-tracing-sessions,create a tracing
+session>> and
+<<enabling-disabling-events,create one or more event rules>>,
+you can start and stop the tracers for this tracing session.
+
+To start tracing in the current tracing session:
+
+* Use the `start` command:
++
+--
+[role="term"]
+----
+lttng start
+----
+--
+
+To stop tracing in the current tracing session:
+
+* Use the `stop` command:
++
+--
+[role="term"]
+----
+lttng stop
+----
+--
+
+LTTng is very flexible: you can launch user applications before
+or after the you start the tracers. The tracers only record the events
+if they pass enabled event rules and if they occur while the tracers are
+started.
+
+
+[[enabling-disabling-channels]]
+=== Create a channel
+
+Once you create a tracing session, you can create a <<channel,channel>>
+with the `enable-channel` command.
+
+Note that LTTng automatically creates a default channel when, for a
+given <<domain,tracing domain>>, no channels exist and you
+<<enabling-disabling-events,create>> the first event rule. This default
+channel is named `channel0` and its attributes are set to reasonable
+values. Therefore, you only need to create a channel when you need
+non-default attributes.
+
+You specify each non-default channel attribute with a command-line
+option when you use the `enable-channel` command. The available
+command-line options are:
+
+[role="growable",cols="asciidoc,asciidoc"]
+.Command-line options for the `enable-channel` command.
+|====
+|Option |Description
+
+|`--overwrite`
+
+|
+Use the _overwrite_
+<<channel-overwrite-mode-vs-discard-mode,event loss mode>> instead of
+the default _discard_ mode.
+
+|`--buffers-pid` (user space tracing domain only)
+
+|
+Use the per-process <<channel-buffering-schemes,buffering scheme>>
+instead of the default per-user buffering scheme.
+
+|+--subbuf-size=__SIZE__+
+
+|
+Allocate sub-buffers of +__SIZE__+ bytes (power of two), for each CPU,
+either for each Unix user (default), or for each instrumented process.
+
+See <<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>>.
+
+|+--num-subbuf=__COUNT__+
+
+|
+Allocate +__COUNT__+ sub-buffers (power of two), for each CPU, either
+for each Unix user (default), or for each instrumented process.
+
+See <<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>>.
+
+|+--tracefile-size=__SIZE__+
+
+|
+Set the maximum size of each trace file that this channel writes within
+a stream to +__SIZE__+ bytes instead of no maximum.
+
+See <<tracefile-rotation,Trace file count and size>>.
+
+|+--tracefile-count=__COUNT__+
+
+|
+Limit the number of trace files that this channel creates to
++__COUNT__+ channels instead of no limit.
+
+See <<tracefile-rotation,Trace file count and size>>.
+
+|+--switch-timer=__PERIODUS__+
+
+|
+Set the <<channel-switch-timer,switch timer period>>
+to +__PERIODUS__+{nbsp}µs.
+
+|+--read-timer=__PERIODUS__+
+
+|
+Set the <<channel-read-timer,read timer period>>
+to +__PERIODUS__+{nbsp}µs.
+
+|+--output=__TYPE__+ (Linux kernel tracing domain only)
+
+|
+Set the channel's output type to +__TYPE__+, either `mmap` or `splice`.
+
+|====
+
+See man:lttng(1)
+for more details about those command-line options.
+
+You can only create a channel in the Linux kernel and user space
+<<domain,tracing domains>>: other tracing domains have their own
+channel created on the fly when
+<<enabling-disabling-events,creating event rules>>.
+
+[IMPORTANT]
+====
+Because of a current LTTng limitation, you must create all channels
+_before_ you <<basic-tracing-session-control,start tracing>> in a given
+tracing session, that is, before the first time you run `lttng start`.
+
+Since LTTng automatically creates a default channel when you use the
+`enable-event` command with a specific tracing domain, you cannot, for
+example, create a Linux kernel event rule, start tracing, and then
+create a user space event rule, because no user space channel exists yet
+and it's too late to create one.
+
+For this reason, make sure to configure your channels properly
+before starting the tracers for the first time!
+====
+
+The following examples show how you can combine the previous
+command-line options to create simple to more complex channels.
+
+.Create a Linux kernel channel with default attributes.
+====
+[role="term"]
+----
+lttng enable-channel --kernel my-channel
+----
+====
+
+.Create a user space channel with 4 sub-buffers or 1{nbsp}MiB each, per CPU, per instrumented process.
+====
+[role="term"]
+----
+lttng enable-channel --userspace --num-subbuf=4 --subbuf-size=1M \
+ --buffers-pid my-channel
+----
+====
+
+.Create a Linux kernel channel which rotates 8 trace files of 4{nbsp}MiB each for each stream
+====
+[role="term"]
+----
+lttng enable-channel --kernel --tracefile-count=8 \
+ --tracefile-size=4194304 my-channel
+----
+====
+
+.Create a user space channel in overwrite (or _flight recorder_) mode.
+====
+[role="term"]
+----
+lttng enable-channel --userspace --overwrite my-channel
+----
+====
+
+You can <<enabling-disabling-events,create>> the same event rule in
+two different channels:
+
+[role="term"]
+----
+lttng enable-event --userspace --channel=my-channel app:tp
+lttng enable-event --userspace --channel=other-channel app:tp
+----
+
+If both channels are enabled, when a tracepoint named `app:tp` is
+reached, LTTng records two events, one for each channel.
+
+
+[[disable-channel]]
+=== Disable a channel
+
+To disable a specific channel that you <<enabling-disabling-channels,created>>
+previously, use the `disable-channel` command.
+
+.Disable a specific Linux kernel channel.
+====
+[role="term"]
+----
+lttng disable-channel --kernel my-channel
+----
+====
+
+The state of a channel precedes the individual states of event rules
+attached to it: event rules which belong to a disabled channel, even if
+they are enabled, are also considered disabled.
+
+
+[[adding-context]]
+=== Add context fields to a channel
+
+Event record fields in trace files provide important information about
+events that occured previously, but sometimes some external context may
+help you solve a problem faster. Examples of context fields are:
+
+* The **process ID**, **thread ID**, **process name**, and
+ **process priority** of the thread in which the event occurs.
+* The **hostname** of the system on which the event occurs.
+* The current values of many possible **performance counters** using
+ perf, for example:
+** CPU cycles, stalled cycles, idle cycles, and the other cycle types.
+** Cache misses.
+** Branch instructions, misses, and loads.
+** CPU faults.
+
+To get the full list of available context fields, see
+`lttng add-context --help`. Some context fields are reserved for a
+specific <<domain,tracing domain>> (Linux kernel or user space).
+
+You add context fields to <<channel,channels>>. All the events
+that a channel with added context fields records contain those fields.
+
+To add context fields to one or all the channels of a given tracing
+session, use the `add-context` command.
+
+.Add context fields to all the channels of the current tracing session.
+====
+The following command line adds the virtual process identifier and
+the per-thread CPU cycles count fields to all the user space channels
+of the current tracing session.
+
+[role="term"]
+----
+lttng add-context --userspace --type=vpid --type=perf:thread:cpu-cycles
+----
+====
+
+.Add a context field to a specific channel.
+====
+The following command line adds the thread identifier context field
+to the Linux kernel channel named `my-channel` in the current
+tracing session.
+
+[role="term"]
+----
+lttng add-context --kernel --channel=my-channel --type=tid
+----
+====
+
+NOTE: You cannot remove context fields from a channel once you add it.
+
+
+[role="since-2.7"]
+[[pid-tracking]]
+=== Track process IDs
+
+It's often useful to allow only specific process IDs (PIDs) to emit
+events. For example, you may wish to record all the system calls made by
+a given process (à la http://linux.die.net/man/1/strace[strace]).
+
+The `track` and `untrack` commands serve this purpose. Both commands
+operate on a whitelist of process IDs. You _add_ entries to this
+whitelist with the `track` command and remove entries with the `untrack`
+command. Any process which has one of the PIDs in the whitelist is
+allowed to emit LTTng events which pass an enabled <<event,event rule>>.
+
+NOTE: The PID tracker tracks the _numeric process IDs_. Should a
+process with a given tracked ID exit and another process be given this
+ID, then the latter would also be allowed to emit events.
+
+.Track and untrack process IDs.
+====
+For the sake of the following example, assume the target system has 16
+possible PIDs.
+
+When you
+<<creating-destroying-tracing-sessions,create a tracing session>>,
+the whitelist contains all the possible PIDs:
+
+[role="img-100"]
+.All PIDs are tracked.
+image::track-all.png[]
+
+When the whitelist is full and you use the `track` command to specify
+some PIDs to track, LTTng first clears the whitelist, then it tracks
+the specific PIDs. After:
+
+[role="term"]
+----
+lttng track --pid=3,4,7,10,13
+----
+
+the whitelist is:
+
+[role="img-100"]
+.PIDs 3, 4, 7, 10, and 13 are tracked.
+image::track-3-4-7-10-13.png[]
+
+You can add more PIDs to the whitelist afterwards:
+
+[role="term"]
+----
+lttng track --pid=1,15,16
+----
+
+The result is:
+
+[role="img-100"]
+.PIDs 1, 15, and 16 are added to the whitelist.
+image::track-1-3-4-7-10-13-15-16.png[]
+
+The `untrack` command removes entries from the PID tracker's whitelist.
+Given the previous example, the following command:
+
+[role="term"]
+----
+lttng untrack --pid=3,7,10,13
+----
+
+leads to this whitelist:
+
+[role="img-100"]
+.PIDs 3, 7, 10, and 13 are removed from the whitelist.
+image::track-1-4-15-16.png[]
+
+LTTng can track all possible PIDs again using the `--all` option:
+
+[role="term"]
+----
+lttng track --pid --all
+----
+
+The result is, again:
+
+[role="img-100"]
+.All PIDs are tracked.
+image::track-all.png[]
+====
+
+.Track only specific PIDs
+====
+A very typical use case with PID tracking is to start with an empty
+whitelist, then <<basic-tracing-session-control,start the tracers>>,
+and then add PIDs manually while tracers are active. You can accomplish
+this by using the `--all` option of the `untrack` command to clear the
+whitelist after you create a tracing session:
+
+[role="term"]
+----
+lttng untrack --pid --all
+----
+
+gives:
+
+[role="img-100"]
+.No PIDs are tracked.
+image::untrack-all.png[]
+
+If you trace with this whitelist configuration, the tracer records no
+events for this <<domain,tracing domain>> because no processes are
+tracked. You can use the `track` command as usual to track specific
+PIDs, for example:
+
+[role="term"]
+----
+lttng track --pid=6,11
+----
+
+Result:
+
+[role="img-100"]
+.PIDs 6 and 11 are tracked.
+image::track-6-11.png[]
+====
+
+
+[role="since-2.5"]
+[[saving-loading-tracing-session]]
+=== Save and load tracing session configurations
+
+Configuring a <<tracing-session,tracing session>> can be long. Some of
+the tasks involved are:
+
+* <<enabling-disabling-channels,Create channels>> with
+ specific attributes.
+* <<adding-context,Add context fields>> to specific channels.
+* <<enabling-disabling-events,Create event rules>> with specific log
+ level and filter conditions.
+
+If you use LTTng to solve real world problems, chances are you have to
+record events using the same tracing session setup over and over,
+modifying a few variables each time in your instrumented program
+or environment. To avoid constant tracing session reconfiguration,
+the cmd:lttng command-line tool can save and load tracing session
+configurations to/from XML files.
+
+To save a given tracing session configuration:
+
+* Use the `save` command:
++
+--
+[role="term"]
+----
+lttng save my-session
+----
+--
++
+Replace `my-session` with the name of the tracing session to save.
+
+LTTng saves tracing session configurations to
+dir:{$LTTNG_HOME/.lttng/sessions} by default. Note that the
+env:LTTNG_HOME environment variable defaults to `$HOME` if not set. Use
+the `--output-path` option to change this destination directory.
+
+LTTng saves all configuration parameters, for example:
+
+* The tracing session name.
+* The trace data output path.
+* The channels with their state and all their attributes.
+* The context fields you added to channels.
+* The event rules with their state, log level and filter conditions.
+
+To load a tracing session:
+
+* Use the `load` command:
++
+--
+[role="term"]
+----
+lttng load my-session
+----
+--
++
+Replace `my-session` with the name of the tracing session to load.
+
+When LTTng loads a configuration, it restores your saved tracing session
+as if you just configured it manually.
+
+See man:lttng(1) for the complete list of command-line options. You
+can also save and load all many sessions at a time, and decide in which
+directory to output the XML files.
+
+
+[[sending-trace-data-over-the-network]]
+=== Send trace data over the network
+
+LTTng can send the recorded trace data to a remote system over the
+network instead of writing it to the local file system.
+
+To send the trace data over the network:
+
+. On the _remote_ system (which can also be the target system),
+ start an LTTng <<lttng-relayd,relay daemon>>:
++
+--
+[role="term"]
+----
+lttng-relayd
+----
+--
+
+. On the _target_ system, create a tracing session configured to
+ send trace data over the network:
++
+--
+[role="term"]
+----
+lttng create my-session --set-url=net://remote-system
+----
+--
++
+Replace `remote-system` by the host name or IP address of the
+remote system. See `lttng create --help` for the exact URL format.
+
+. On the target system, use the cmd:lttng command-line tool as usual.
+ When tracing is active, the target's consumer daemon sends sub-buffers
+ to the relay daemon running on the remote system intead of flushing
+ them to the local file system. The relay daemon writes the received
+ packets to the local file system.
+
+The relay daemon writes trace files to
++$LTTNG_HOME/lttng-traces/__hostname__/__session__+ by default, where
++__hostname__+ is the host name of the target system and +__session__+
+is the tracing session name. Note that the env:LTTNG_HOME environment
+variable defaults to `$HOME` if not set. Use the `--output` option of
+cmd:lttng-relayd to write trace files to another base directory.
+
+
+[role="since-2.4"]
+[[lttng-live]]
+=== View events as LTTng emits them (noch:{LTTng} live)
+
+LTTng live is a network protocol implemented by the
+<<lttng-relayd,relay daemon>> to allow compatible trace viewers to
+display events as LTTng emits them on the target system while tracing
+is active.
+
+The relay daemon creates a _tee_: it forwards the trace data to both
+the local file system and to connected live viewers:
+
+[role="img-90"]
+.The relay daemon creates a _tee_, forwarding the trace data to both trace files and a connected live viewer.
+image::live.png[]
+
+To use LTTng live:
+
+. On the _target system_, create a <<tracing-session,tracing session>>
+ in _live mode_:
++
+--
+[role="term"]
+----
+lttng create --live my-session
+----
+--
++
+This spawns a local relay daemon.
+
+. Start the live viewer and configure it to connect to the relay
+ daemon. For example, with http://diamon.org/babeltrace[Babeltrace]:
++
+--
+[role="term"]
+----
+babeltrace --input-format=lttng-live net://localhost/host/hostname/my-session
+----
+--
++
+Replace:
++
+--
+* `hostname` with the host name of the target system.
+* `my-session` with the name of the tracing session to view.
+--
+
+. Configure the tracing session as usual with the cmd:lttng
+ command-line tool, and <<basic-tracing-session-control,start tracing>>.
+
+You can list the available live tracing sessions with Babeltrace:
+
+[role="term"]
+----
+babeltrace --input-format=lttng-live net://localhost
+----
+
+You can start the relay daemon on another system. In this case, you need
+to specify the relay daemon's URL when you create the tracing session
+with the `--set-url` option. You also need to replace `localhost`
+in the procedure above with the host name of the system on which the
+relay daemon is running.
+
+See man:lttng(1) and man:lttng-relayd(8) for the complete list of
+command-line options.
+
+
+[role="since-2.3"]
+[[taking-a-snapshot]]
+=== Take a snapshot of the current sub-buffers of a tracing session
+
+The normal behavior of LTTng is to append full sub-buffers to growing
+trace data files. This is ideal to keep a full history of the events
+that occurred on the target system, but it can
+represent too much data in some situations. For example, you may wish
+to trace your application continuously until some critical situation
+happens, in which case you only need the latest few recorded
+events to perform the desired analysis, not multi-gigabyte trace files.
+
+With the `snapshot` command, you can take a snapshot of the current
+sub-buffers of a given <<tracing-session,tracing session>>. LTTng can
+write the snapshot to the local file system or send it over the network.
+
+To take a snapshot:
+
+. Create a tracing session in _snapshot mode_:
++
+--
+[role="term"]
+----
+lttng create --snapshot my-session
+----
+--
++
+The <<channel-overwrite-mode-vs-discard-mode,event loss mode>> of
+<<channel,channels>> created in this mode is automatically set to
+_overwrite_ (flight recorder mode).
+
+. Configure the tracing session as usual with the cmd:lttng
+ command-line tool, and <<basic-tracing-session-control,start tracing>>.
+
+. **Optional**: When you need to take a snapshot, stop tracing.
++
+You can take a snapshot when the tracers are active, but if you stop
+them first, you are sure that the data in the sub-buffers does not
+change before you actually take the snapshot.
+
+. Take a snapshot:
++
+--
+[role="term"]
+----
+lttng snapshot record --name=my-first-snapshot
+----
+--
++
+LTTng writes the current sub-buffers of all the current tracing
+session's channels to trace files on the local file system. Those trace
+files have `my-first-snapshot` in their name.
+
+There is no difference between the format of a normal trace file and the
+format of a snapshot: viewers of LTTng traces also support LTTng
+snapshots.
+
+By default, LTTng writes snapshot files to the path shown by
+`lttng snapshot list-output`. You can change this path or decide to send
+snapshots over the network using either:
+
+. An output path or URL that you specify when you create the
+ tracing session.
+. An snapshot output path or URL that you add using
+ `lttng snapshot add-output`
+. An output path or URL that you provide directly to the
+ `lttng snapshot record` command.
+
+Method 3 overrides method 2, which overrides method 1. When you
+specify a URL, a relay daemon must listen on a remote system (see
+<<sending-trace-data-over-the-network,Send trace data over the network>>).
+
+
+[role="since-2.6"]
+[[mi]]
+=== Use the machine interface
+
+With any command of the cmd:lttng command-line tool, you can use the
+`--mi=xml` argument (before the command name) to get an XML machine
+interface output, for example:
+
+[role="term"]
+----
+lttng --mi=xml enable-event --kernel --syscall open
+----
+
+A schema definition (XSD) is
+https://github.com/lttng/lttng-tools/blob/stable-{revision}/src/common/mi_lttng.xsd[available]
+to ease the integration with external tools as much as possible.
+
+
+[role="since-2.7"]
+[[persistent-memory-file-systems]]
+=== Record trace data on persistent memory file systems
+
+https://en.wikipedia.org/wiki/Non-volatile_random-access_memory[Non-volatile random-access memory]
+(NVRAM) is random-access memory that retains its information when power
+is turned off (non-volatile). Systems with such memory can store data
+structures in RAM and retrieve them after a reboot, without flushing
+to typical _storage_.
+
+Linux supports NVRAM file systems thanks to either
+http://pramfs.sourceforge.net/[PRAMFS] or
+https://www.kernel.org/doc/Documentation/filesystems/dax.txt[DAX]{nbsp}+{nbsp}http://lkml.iu.edu/hypermail/linux/kernel/1504.1/03463.html[pmem]
+(requires Linux 4.1+).
+
+This section does not describe how to operate such file systems;
+we assume that you have a working persistent memory file system.
+
+When you create a <<tracing-session,tracing session>>, you can specify
+the path of the shared memory holding the sub-buffers. If you specify a
+location on an NVRAM file system, then you can retrieve the latest
+recorded trace data when the system reboots after a crash.
+
+To record trace data on a persistent memory file system and retrieve the
+trace data after a system crash:
+
+. Create a tracing session with a sub-buffer shared memory path located
+ on an NVRAM file system:
++
+--
+[role="term"]
+----
+lttng create --shm-path=/path/to/shm
+----
+--
+
+. Configure the tracing session as usual with the cmd:lttng
+ command-line tool, and <<basic-tracing-session-control,start tracing>>.
+
+. After a system crash, use the cmd:lttng-crash command-line tool to
+ view the trace data recorded on the NVRAM file system:
++
+--
+[role="term"]
+----
+lttng-crash /path/to/shm
+----
+--
+
+The binary layout of the ring buffer files is not exactly the same as
+the trace files layout. This is why you need to use the cmd:lttng-crash
+utility instead of your preferred trace viewer directly.
+
+To convert the ring buffer files to LTTng trace files:
+
+* Use the `--extract` option of cmd:lttng-crash:
++
+--
+[role="term"]
+----
+lttng-crash --extract=/path/to/trace /path/to/shm
+----
+--
+
+See man:lttng-crash(1) for the complete list of command-line options.
+
+
+[[reference]]
+== Reference
+
+This section presents various references for LTTng packages such as
+links to online manpages, tables that the rest of the text needs,
+descriptions of library functions, and more.
+
+
+[[online-lttng-manpages]]
+=== Online noch:{LTTng} manpages
+
+LTTng packages currently install the following link:/man[man pages],
+available online using the links below:
+
+* **LTTng-tools**
+** man:lttng(1)
+** man:lttng-crash(1)
+** man:lttng-sessiond(8)
+** man:lttng-relayd(8)
+* **LTTng-UST**
+** man:lttng-gen-tp(1)
+** man:lttng-ust(3)
+** man:lttng-ust-cyg-profile(3)
+** man:lttng-ust-dl(3)
+
+
+[[lttng-ust-ref]]
+=== noch:{LTTng-UST}
+
+This section presents references of the LTTng-UST package.
+
+
+[[liblttng-ust]]
+==== noch:{LTTng-UST} library (+liblttng‑ust+)
+
+The LTTng-UST library, or `liblttng-ust`, is the main shared object
+against which user applications are linked to make LTTng user space
+tracing possible.
+
+The <<c-application,C application>> guide shows the complete
+process to instrument, build and run a C/$$C++$$ application using
+LTTng-UST, while this section contains a few important tables.
+
+
+[[liblttng-ust-tp-fields]]
+===== Tracepoint fields macros (for `TP_FIELDS()`)
+
+The available macros to define tracepoint fields, which you must use
+within `TP_FIELDS()` in `TRACEPOINT_EVENT()`, are:
+
+[role="func-desc growable",cols="asciidoc,asciidoc"]
+.Available macros to define LTTng-UST tracepoint fields
+|====
+|Macro |Description and parameters
+
+|
++ctf_integer(__t__, __n__, __e__)+
+
++ctf_integer_nowrite(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `long`, `size_t`, ...).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_hex(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network(__t__, __n__, __e__)+
+|
+Integer in network byte order (big-endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_network_hex(__t__, __n__, __e__)+
+|
+Integer in network byte order, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_float(__t__, __n__, __e__)+
+
++ctf_float_nowrite(__t__, __n__, __e__)+
+|
+Floating point number.
+
++__t__+::
+ Floating point number C type (`float` or `double`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_string(__n__, __e__)+
+
++ctf_string_nowrite(__n__, __e__)+
+|
+Null-terminated string; undefined behavior if +__e__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_array(__t__, __n__, __e__, __s__)+
+
++ctf_array_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array of integers
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_array_text(__t__, __n__, __e__, __s__)+
+
++ctf_array_text_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array, printed as text.
+
+The string does not need to be null-terminated.
+
++__t__+::
+ Array element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_sequence(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers.
+
+The type of +__E__+ must be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|
++ctf_sequence_text(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array, displayed as text.
+
+The string does not need to be null-terminated.
+
+The type of +__E__+ must be unsigned.
+
+The behaviour is undefined if +__e__+ is `NULL`.
+
++__t__+::
+ Sequence element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+|====
+
+The `_nowrite` versions omit themselves from the session trace, but are
+otherwise identical. This means the tracer does not write the `_nowrite`
+fields to the trace. Their primary purpose is to make some of the event
+context available to the <<enabling-disabling-events,event filters>>
+without having to commit the data to sub-buffers.
+
+
+[[liblttng-ust-tracepoint-loglevel]]
+===== Tracepoint log levels (for `TRACEPOINT_LOGLEVEL()`)
+
+The following table shows the available log level values for the
+`TRACEPOINT_LOGLEVEL()` macro:
+
+`TRACE_EMERG`::
+ System is unusable.
+
+`TRACE_ALERT`::
+ Action must be taken immediately.
+
+`TRACE_CRIT`::
+ Critical conditions.
+
+`TRACE_ERR`::
+ Error conditions.
+
+`TRACE_WARNING`::
+ Warning conditions.
+
+`TRACE_NOTICE`::
+ Normal, but significant, condition.
+
+`TRACE_INFO`::
+ Informational message.
+
+`TRACE_DEBUG_SYSTEM`::
+ Debug information with system-level scope (set of programs).
+
+`TRACE_DEBUG_PROGRAM`::
+ Debug information with program-level scope (set of processes).
+
+`TRACE_DEBUG_PROCESS`::
+ Debug information with process-level scope (set of modules).
+
+`TRACE_DEBUG_MODULE`::
+ Debug information with module (executable/library) scope (set of units).
+
+`TRACE_DEBUG_UNIT`::
+ Debug information with compilation unit scope (set of functions).
+
+`TRACE_DEBUG_FUNCTION`::
+ Debug information with function-level scope.
+
+`TRACE_DEBUG_LINE`::
+ Debug information with line-level scope (TRACEPOINT_EVENT default).
+
+`TRACE_DEBUG`::
+ Debug-level message.
+
+Log levels `TRACE_EMERG` through `TRACE_INFO` and `TRACE_DEBUG` match
+http://man7.org/linux/man-pages/man3/syslog.3.html[syslog]
+level semantics. Log levels `TRACE_DEBUG_SYSTEM` through `TRACE_DEBUG`
+offer more fine-grained selection of debug information.
+
+
+[[lttng-modules-ref]]
+=== noch:{LTTng-modules}
+
+This section presents references of the LTTng-modules package.
+
+
+[role="since-2.7"]
+[[lttng-modules-tp-fields]]
+==== Tracepoint fields macros (for `TP_FIELDS()`)
+
+[[tp-fast-assign]][[tp-struct-entry]]The available macros to define
+tracepoint fields, which must be listed within `TP_FIELDS()` in
+`LTTNG_TRACEPOINT_EVENT()`, are:
+
+[role="func-desc growable",cols="asciidoc,asciidoc"]
+.Available macros to define LTTng-modules tracepoint fields
+|====
+|Macro |Description and parameters
+
+|
++ctf_integer(__t__, __n__, __e__)+
+
++ctf_integer_nowrite(__t__, __n__, __e__)+
+
++ctf_user_integer(__t__, __n__, __e__)+
+
++ctf_user_integer_nowrite(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 10.
+
++__t__+::
+ Integer C type (`int`, `long`, `size_t`, ...).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_integer_hex(__t__, __n__, __e__)+
+
++ctf_user_integer_hex(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|+ctf_integer_oct(__t__, __n__, __e__)+
+|
+Standard integer, displayed in base 8.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_integer_network(__t__, __n__, __e__)+
+
++ctf_user_integer_network(__t__, __n__, __e__)+
+|
+Integer in network byte order (big-endian), displayed in base 10.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_integer_network_hex(__t__, __n__, __e__)+
+
++ctf_user_integer_network_hex(__t__, __n__, __e__)+
+|
+Integer in network byte order, displayed in base 16.
+
++__t__+::
+ Integer C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_string(__n__, __e__)+
+
++ctf_string_nowrite(__n__, __e__)+
+
++ctf_user_string(__n__, __e__)+
+
++ctf_user_string_nowrite(__n__, __e__)+
+|
+Null-terminated string; undefined behavior if +__e__+ is `NULL`.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
+|
++ctf_array(__t__, __n__, __e__, __s__)+
+
++ctf_array_nowrite(__t__, __n__, __e__, __s__)+
+
++ctf_user_array(__t__, __n__, __e__, __s__)+
+
++ctf_user_array_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array of integers
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_array_text(__t__, __n__, __e__, __s__)+
+
++ctf_array_text_nowrite(__t__, __n__, __e__, __s__)+
+
++ctf_user_array_text(__t__, __n__, __e__, __s__)+
+
++ctf_user_array_text_nowrite(__t__, __n__, __e__, __s__)+
+|
+Statically-sized array, printed as text.
+
+The string does not need to be null-terminated.
+
++__t__+::
+ Array element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__s__+::
+ Number of elements.
+
+|
++ctf_sequence(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_user_sequence(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_user_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers.
+
+The type of +__E__+ must be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|+ctf_sequence_hex(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers, displayed in base 16.
+
+The type of +__E__+ must be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|+ctf_sequence_network(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array of integers in network byte order (big-endian),
+displayed in base 10.
+
+The type of +__E__+ must be unsigned.
+
++__t__+::
+ Array element C type.
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+
+|
++ctf_sequence_text(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_user_sequence_text(__t__, __n__, __e__, __T__, __E__)+
+
++ctf_user_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
+|
+Dynamically-sized array, displayed as text.
+
+The string does not need to be null-terminated.
+
+The type of +__E__+ must be unsigned.
+
+The behaviour is undefined if +__e__+ is `NULL`.
+
++__t__+::
+ Sequence element C type (always `char`).
+
++__n__+::
+ Field name.
+
++__e__+::
+ Argument expression.
+
++__T__+::
+ Length expression C type.
+
++__E__+::
+ Length expression.
+|====
+
+Use the `_user` versions when the argument expression, `e`, is
+a user space address. In the cases of `ctf_user_integer*()` and
+`ctf_user_float*()`, `&e` must be a user space address, thus `e` must
+be addressable.
+
+The `_nowrite` versions omit themselves from the session trace, but are
+otherwise identical. This means the `_nowrite` fields won't be written
+in the recorded trace. Their primary purpose is to make some
+of the event context available to the
+<<enabling-disabling-events,event filters>> without having to
+commit the data to sub-buffers.
+
+
+[[glossary]]
+== Glossary
+
+Terms related to LTTng and to tracing in general:
+
+Babeltrace::
+ The http://diamon.org/babeltrace[Babeltrace] project, which includes
+ the cmd:babeltrace command, some libraries, and Python bindings.
+
+<<channel-buffering-schemes,buffering scheme>>::
+ A layout of sub-buffers applied to a given channel.
+
+<<channel,channel>>::
+ An entity which is responsible for a set of ring buffers.
++
+<<event,Event rules>> are always attached to a specific channel.
+
+clock::
+ A reference of time for a tracer.
+
+<<lttng-consumerd,consumer daemon>>::
+ A process which is responsible for consuming the full sub-buffers
+ and write them to a file system or send them over the network.
+
+<<channel-overwrite-mode-vs-discard-mode,discard mode>>:: The event loss
+ mode in which the tracer _discards_ new event records when there's no
+ sub-buffer space left to store them.
+
+event::
+ The consequence of the execution of an instrumentation
+ point, like a tracepoint that you manually place in some source code,
+ or a Linux kernel KProbe.
++
+An event is said to _occur_ at a specific time. Different actions can
+be taken upon the occurance of an event, like record the event's payload
+to a sub-buffer.
+
+<<channel-overwrite-mode-vs-discard-mode,event loss mode>>::
+ The mechanism by which event records of a given channel are lost
+ (not recorded) when there is no sub-buffer space left to store them.
+
+[[def-event-name]]event name::
+ The name of an event, which is also the name of the event record.
+ This is also called the _instrumentation point name_.
+
+event record::
+ A record, in a trace, of the payload of an event which occured.
+
+<<event,event rule>>::
+ Set of conditions which must be satisfied for one or more occuring
+ events to be recorded.
+
+`java.util.logging`::
+ Java platform's
+ https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[core logging facilities].
+
+<<instrumenting,instrumentation>>::
+ The use of LTTng probes to make a piece of software traceable.
+
+instrumentation point::
+ A point in the execution path of a piece of software that, when
+ reached by this execution, can emit an event.
+
+instrumentation point name::
+ See _<<def-event-name,event name>>_.
+
+log4j::
+ A http://logging.apache.org/log4j/1.2/[logging library] for Java
+ developed by the Apache Software Foundation.
+
+log level::
+ Level of severity of a log statement or user space
+ instrumentation point.
+
+LTTng::
+ The _Linux Trace Toolkit: next generation_ project.
+
+<<lttng-cli,cmd:lttng>>::
+ A command-line tool provided by the LTTng-tools project which you
+ can use to send and receive control messages to and from a
+ session daemon.
+
+LTTng analyses::
+ The https://github.com/lttng/lttng-analyses[LTTng analyses] project,
+ which is a set of analyzing programs that are used to obtain a
+ higher level view of an LTTng trace.
+
+cmd:lttng-consumerd::
+ The name of the consumer daemon program.
+
+cmd:lttng-crash::
+ A utility provided by the LTTng-tools project which can convert
+ ring buffer files (usually
+ <<persistent-memory-file-systems,saved on a persistent memory file system>>)
+ to trace files.
+
+LTTng Documentation::
+ This document.
+
+<<lttng-live,LTTng live>>::
+ A communication protocol between the relay daemon and live viewers
+ which makes it possible to see events "live", as they are received by
+ the relay daemon.
+
+<<lttng-modules,LTTng-modules>>::
+ The https://github.com/lttng/lttng-modules[LTTng-modules] project,
+ which contains the Linux kernel modules to make the Linux kernel
+ instrumentation points available for LTTng tracing.
+
+cmd:lttng-relayd::
+ The name of the relay daemon program.
+
+cmd:lttng-sessiond::
+ The name of the session daemon program.
+
+LTTng-tools::
+ The https://github.com/lttng/lttng-tools[LTTng-tools] project, which
+ contains the various programs and libraries used to
+ <<controlling-tracing,control tracing>>.
+
+<<lttng-ust,LTTng-UST>>::
+ The https://github.com/lttng/lttng-ust[LTTng-UST] project, which
+ contains libraries to instrument user applications.
+
+<<lttng-ust-agents,LTTng-UST Java agent>>::
+ A Java package provided by the LTTng-UST project to allow the
+ LTTng instrumentation of `java.util.logging` and Apache log4j 1.2
+ logging statements.
+
+<<lttng-ust-agents,LTTng-UST Python agent>>::
+ A Python package provided by the LTTng-UST project to allow the
+ LTTng instrumentation of Python logging statements.
+
+<<channel-overwrite-mode-vs-discard-mode,overwrite mode>>::
+ The event loss mode in which new event records overwrite older
+ event records when there's no sub-buffer space left to store them.
+
+<<channel-buffering-schemes,per-process buffering>>::
+ A buffering scheme in which each instrumented process has its own
+ sub-buffers for a given user space channel.
+
+<<channel-buffering-schemes,per-user buffering>>::
+ A buffering scheme in which all the processes of a Unix user share the
+ same sub-buffer for a given user space channel.
+
+<<lttng-relayd,relay daemon>>::
+ A process which is responsible for receiving the trace data sent by
+ a distant consumer daemon.
+
+ring buffer::
+ A set of sub-buffers.
+
+<<lttng-sessiond,session daemon>>::
+ A process which receives control commands from you and orchestrates
+ the tracers and various LTTng daemons.
+
+<<taking-a-snapshot,snapshot>>::
+ A copy of the current data of all the sub-buffers of a given tracing
+ session, saved as trace files.
+
+sub-buffer::
+ One part of an LTTng ring buffer which contains event records.
+
+timestamp::
+ The time information attached to an event when it is emitted.
+
+trace (_noun_)::
+ A set of files which are the concatenations of one or more
+ flushed sub-buffers.
+
+trace (_verb_)::
+ The action of recording the events emitted by an application
+ or by a system, or to initiate such recording by controlling
+ a tracer.
+
+Trace Compass::
+ The http://tracecompass.org[Trace Compass] project and application.
+
+tracepoint::
+ An instrumentation point using the tracepoint mechanism of the Linux
+ kernel or of LTTng-UST.
+
+tracepoint definition::
+ The definition of a single tracepoint.
+
+tracepoint name::
+ The name of a tracepoint.
+
+tracepoint provider::
+ A set of functions providing tracepoints to an instrumented user
+ application.
++
+Not to be confused with a _tracepoint provider package_: many tracepoint
+providers can exist within a tracepoint provider package.
+
+tracepoint provider package::
+ One or more tracepoint providers compiled as an object file or as
+ a shared library.
+
+tracer::
+ A software which records emitted events.
+
+<<domain,tracing domain>>::
+ A namespace for event sources.
+
+tracing group::
+ The Unix group in which a Unix user can be to be allowed to trace the
+ Linux kernel.
+
+<<tracing-session,tracing session>>::
+ A stateful dialogue between you and a <<lttng-sessiond,session
+ daemon>>.
+
+user application::
+ An application running in user space, as opposed to a Linux kernel
+ module, for example.
--- /dev/null
+The LTTng Documentation: Contributor's guide
+============================================
+Philippe Proulx
+v1.0, 21 October 2016
+
+This guide presents the structure and conventions of the LTTng
+Documentation's source. Make sure you read it thoroughly before
+you contribute a change.
+
+
+[[principles]]
+== Principles
+
+The LTTng Documentation exists to make the
+https://lttng.org/[LTTng project] useable.
+Without such a complete documentation consolidating the various
+concepts, features, and procedures of LTTng-tools, LTTng-UST, and
+LTTng-modules, most of the project would only be useable by
+its authors.
+
+Why not simply read the man pages? While the LTTng man pages are
+complementary to the LTTng Documentation, they remain formal
+references: they lack the introductory quality and procedural user
+guides found in this documentation.
+
+The core principle of the LTTng Documentation is to make the text as
+cleverly organized, easy to follow, precise, and consistent as possible.
+This involves keeping a high level of rigor as to such things as the
+document's style, voice, grammar, and layout.
+
+Of course, those guidelines are not new to the technical writing realm,
+and it would be bold to devise a brand new manual of style for the sole
+existence of the LTTng Documentation when so many have already proven
+their value. This is why the LTTng Documentation (especially starting
+from version 2.7) does its best to follow the rules of the
+https://en.wikipedia.org/wiki/Microsoft_Manual_of_Style[Microsoft Manual
+of Style (4th edition)], a landmark work in its field. Of particular
+interest in this book are:
+
+* Chapter 1, _Microsoft style and voice_.
+* Chapter 6, _Procedures and technical content_.
+* Chapter 7, _Practical issues of style_.
+* Chapter 8, _Grammar_.
+* Chapter 9, _Punctuation_.
+* Chapter 11, _Acronyms and other abbreviations_.
+
+The <<terminology,Terminology>> section of this contributor's guide
+adds terms to or overrides terms of Part 2, _Usage Dictionary_.
+
+
+== Organization of the repository and format
+
+The Git repository of the LTTng Documentation contains all the official
+versions of the documentation as separate source files. Each source file
+is in its own +2.__x__+ directory, along with documentation resources
+specific to this version of LTTng. You can find common source files in
+the `common` directory.
+
+The source files are written in
+http://www.methods.co.nz/asciidoc/[AsciiDoc], a rich, lightweight markup
+language with all the blocks and inline elements needed to write
+backend-agnostic content.
+
+Although the official LTTng website uses a custom script to generate
+its own HTML version of the LTTng Documentation, it is possible to
+generate an autonomous HTML preview (see
+link:README.adoc[`README.adoc`]). The `asciidoc.html5.conf` AsciiDoc
+configuration file sets a few attributes and implements the required
+macros for this preview target.
+
+
+== Validation script
+
+Before you submit any change, make sure that the check script passes.
+This is a Python script which validates some elements of a specific
+document.
+
+You need the following dependencies to run the check script:
+
+* http://www.methods.co.nz/asciidoc/[AsciiDoc]
+* Python 3
+* http://lxml.de/[lxml] Python 3 package
+* https://pypi.python.org/pypi/termcolor[termcolor] Python 3 package
+
+Run the check script:
+
+----
+python3 tools/check.py 2.7/lttng-docs-2.7.txt
+----
+
+Replace `2.7` by the version of the document to validate in the previous
+command line.
+
+
+== Style considerations
+
+As stated in <<principles,Principles>>, the LTTng Documentation follows
+the Microsoft Manual of Style (4th edition). We encourage you to read
+this work before contributing a major change to the document.
+
+You also need to consider the following rules, often specific to the
+AsciiDoc format used to write the LTTng Documentation, when you edit
+existing content or when you create new sections.
+
+
+=== Macros
+
+* **Man page references**: Always use the +man:__command__(__section__)+
+ macro you refer to a man page. The official online version of the
+ LTTng Documentation has hyperlinks to the correct online versions
+ of the LTTng man pages thanks to this macro.
++
+.Using the `man` macro.
+====
+----
+See man:lttng-ust(3) for more details about ...
+----
+====
+
+* **File names**: Always use the +path:{__path__}+
+ macro when you need to write a file name.
++
+.Using the `path` macro.
+====
+----
+Load the configuration file path:{hello.lttng} directory by default.
+----
+====
+
+* **Directory names**: Always use the +dir:{__path__}+
+ macro when you need to write a directory name.
++
+.Using the `dir` macro.
+====
+----
+Traces are recorded to the dir:{~/lttng-traces} directory by default.
+----
+====
+
+* **Environment variable**: Always use the +env:__VAR__+ macro when
+ you need to write an environment variable name. +__VAR__+ must not
+ contain the shell's `$` prefix.
++
+.Using the `env` macro.
+====
+----
+You can set the env:LTTNG_UST_DEBUG environment variable to `1` to
+activate LTTng-UST's debug output.
+----
+====
+
+* **Command names**: Always use the +cmd:__cmd__+
+ macro when you need to write a command name.
++
+.Using the `cmd` macro.
+====
+----
+Run cmd:lttng-sessiond as the root user.
+----
+====
+
+
+=== Dashes
+
+Em dashes can usually be written using `--` in AsciiDoc, but sometimes
+the two hyphens are outputted as is, for example if the character at the
+left or at the right of them is a punctuation. You can avoid this
+by using the equivalent `—` HTML entity.
+
+.Using `--` for an em dash.
+====
+----
+And yet, when the car was finally delivered--nearly three months after it
+was ordered--she decided she no longer wanted it, leaving the dealer with
+an oddly equipped car that would be difficult to sell.
+----
+====
+
+.Using `—` for an em dash.
+====
+----
+As the frequency of recorded events increases--either because the event
+throughput is actually higher or because you enabled more events than
+usual—__event loss__ might be experienced.
+----
+====
+
+
+=== Non-breaking spaces
+
+Always use a non-breaking space (`{nbsp}`, or HTML entity ` `)
+between a quantity and its unit, or when it would be unnatural to have
+two related words split on two lines.
+
+.Using a non-breaking space between a quantity and its unit.
+====
+----
+The size of this file is 1039{nbsp}bytes.
+----
+====
+
+.Using a non-breaking space to avoid an odd line break.
+====
+----
+This integer is displayed in base{nbsp}16.
+----
+====
+
+
+=== Placeholders in inline code
+
+When a section of an inline code element is a placeholder, or variable,
+use the `+` form of the element (instead of +`+), and place `__`
+around the placeholder.
+
+.Using a placeholder in an inline code element.
+====
+----
+Name your file +something.__sys__.c+, where +__sys__+ is your system name.
+----
+====
+
+
+=== Listing blocks
+
+There are two types of listing blocks:
+
+* [[term-box]]**Terminal boxes** are used to show commands to be entered in a
+ terminal exclusively, that is, the output of commands must not be
+ written in terminal boxes. A terminal box is an AsciiDoc literal
+ block with the `term` role.
++
+.Using a terminal box.
+====
+[listing]
+....
+[role="term"]
+----
+lttng create my-session
+lttng enable-event --kernel --all
+----
+....
+====
++
+The output of a command line can be written using a simple, role-less
+listing block.
+
+* **Source code boxes** are used to show syntax-highlighted snippets of
+ source code. A source code box is an AsciiDoc source code block.
++
+.Using a source code box.
+====
+[listing]
+....
+[source,c]
+----
+#include <stdio.h>
+
+int main(void)
+{
+ puts("Hello, World!");
+
+ return 0;
+}
+----
+....
+====
++
+The second attribute is the name of the programming language for
+proper syntax highlighting (for example, `c`, `python`, `make`, `java`).
+This name must be known to http://pygments.org/[Pygments].
++
+Always indent source code examples with 4{nbsp}spaces.
+
+In any listing block, the lines must not exceed 80 characters (prefer a
+maximum of 72 characters).
+
+
+=== Command-line options
+
+When specifying command-line options:
+
+* Always use the long form of the option (with two hyphens).
+* Use a code element for the option name (backticks).
+* Always follow the option name by the _option_ word.
+
+.Using a command-line option.
+====
+----
+You can use the `lttng` tool's `--group` option to specify a custom
+tracing group.
+----
+====
+
+In <<term-box,terminal boxes>>, always put `=` between the option name
+and its argument, if any.
+
+.Terminal box.
+====
+In this example, `provider:'sys_*'` is not the argument of the
+`--userspace` option: it's the first positional argument, and
+the `--userspace` option has no arguments.
+
+[listing]
+....
+[role="term"]
+----
+lttng enable-event --userspace provider:'sys_*' --filter='field < 23'
+ --exclude=sys_send,sys_block --loglevel=TRACE_INFO
+----
+....
+====
+
+
+=== Procedures
+
+Use an ordered list to write a procedure.
+
+If a step is optional, prepend `**Optional**:` followed by a space to
+the step's first sentence. Start the first sentence with a capital
+letter. Do not use an optional step followed by a condition; use a
+conditional step for this.
+
+If a step is conditional, put the condition (_If something_) in bold,
+followed by a comma, followed by the step itself.
+
+
+=== External links
+
+When using a hyperlink to an LTTng repository's file or directory,
+link to the GitHub code browser. Make sure to link to the appropriate
+Git branch (usually +stable-2.__x__+). You can use the `revision`
+attribute in the URL.
+
+.Link to source file.
+====
+----
+See the file
+https://github.com/lttng/lttng-tools/blob/stable-{revision}/src/common/daemonize.c[path:{src/common/daemonize.c}]
+for more details about [...]
+----
+====
+
+
+=== "Since" sections
+
+If a whole section describes a feature which was introduced in LTTng 2.1
+or later, add the +since-2.__x__+ role to the section's heading, where
++__x__+ is the minor version of the LTTng release which introduced
+the feature.
+
+.Section heading describing a feature introduced in LTTng 2.5.
+====
+----
+[role="since-2.5"]
+[[tracef]]
+==== Use `tracef()`
+----
+====
+
+
+[[terminology]]
+== Terminology
+
+What follows is an official, partial list of technical terms used by the
+LTTng Documentation. Other forms of those terms are _not_ permitted. For
+example, do not write `use-case` or `filesystem`.
+
+Autotools::
+ The GNU Autotools.
++
+Do not use _autotools_.
+
+Babeltrace::
+ The Babeltrace project, which includes the `babeltrace` command, some
+ libraries, and Python bindings.
++
+Use +`babeltrace`+ to refer to the actual `babeltrace` command.
+
+Babeltrace Python bindings::
+ The Python bindings of Babeltrace.
++
+The plural _bindings_ is important.
+
+Bash::
+ The Bash shell.
++
+Do not use _bash_.
+
+buffering scheme::
+ A layout of tracing buffers applied to a given channel.
+
+channel::
+ An LTTng channel.
+
+CLI::
+ Prefer expanding this acronym to _command-line interface_ in the text.
+
+clock::
+ A reference of time for a tracer.
++
+Use _system time_ to refer to the date and time as seen by a user.
+
+command-line::
+ Adjective version of _command line_: _command-line option_,
+ _command-line interface_.
+
+command-line interface::
+ An interface in which the user enters command lines to instruct the
+ system what to do.
++
+Prefer using _command_ or _command-line tool_ to refer to a
+specific command.
+
+command line::
+ An actual line of command entered by the user in a terminal, at a
+ command prompt.
++
+Write _command-line_ when used as an adjective.
+
+consumer daemon::
+ The LTTng consumer daemon.
++
+Do not use _consumerd_.
++
+Use +`lttng-consumerd`+ to refer to the consumer daemon
+executable.
+
+domain::
+ Do not use when referring to a _tracing domain_.
+
+event::
+ Occurrence recognised by software, emitted by a tracer when specific
+ conditions are met, at a given time. An event _occurs_ at a specific
+ time, after which a tracer can record its payload.
+
+event loss mode::
+ The mechanism by which event records of a given channel are lost
+ (not recorded) when there is no sub-buffer space left to store them.
+
+event name::
+ The name of an event, which is also the name of the event record.
+ This is different from a _tracepoint name_, which is only the name
+ of the instrumentation point, not necessarily equal to the event
+ name.
+
+event record::
+ Record, in a trace, of the payload of an event which occured.
+
+event rule::
+ Set of conditions which must be satisfied for one or more events
+ to occur. The `lttng enable-event` command creates and enables
+ _event rules_, not _events_.
+
+file system::
+ Contains directories, files, and links in an organized structure.
++
+Do not use _filesystem_ or _file-system_.
+
++`java.util.logging`+::
+ Even though the `--jul` command-line option is an acronym for this
+ term, there is no such thing as _Java Util Logging_. The only
+ correct form is the name of the Java package,
+ +`java.util.logging`+.
+
+instrumentation::
+ The use of LTTng probes to make a software traceable.
+
+libc::
+ Do not use.
++
+Use _the C standard library_ to refer to the standard library for
+the C programming language, or _glibc_ to refer to the GNU C Library
+specifically.
+
+log4j::
+ LTTng-UST supports Java logging using Apache _log4j_, not Apache
+ Log4j 2.
+
+log level::
+ Level of severity of a log statement.
++
+Do not hyphenate.
+
+kernel::
+ In general, do not use _kernel_ to refer to the _Linux kernel_: use
+ the whole _Linux kernel_ term, because other operating system kernels
+ exist. Since the _L_ in _LTTng_ means _Linux_, it's okay to use _LTTng
+ kernel modules_.
+
+Linux Trace Toolkit: next generation::
+ The expansion of the _LTTng_ acronym.
++
+The colon and the lowercase _n_ and _g_ are important.
+
+LTTng-analyses::
+ The LTTng-analyses project.
+
+LTTng-modules::
+ The LTTng-modules project.
+
+LTTng-tools::
+ The LTTng-tools project.
+
+LTTng-UST::
+ The LTTng-UST project.
+
+LTTng-UST Java agent::
+LTTng-UST Python agent::
+ An LTTng user space agent.
++
+Do not use _Java LTTng-UST agent_ or _Python LTTng-UST agent_.
+
+LTTng Documentation::
+ The name of this project.
++
+Do not use _LTTng documentation_.
++
+When referring to the project, the _the_ determiner can be lowercase:
+_Welcome to the LTTng Documentation!_.
+
+LTTng live::
+ The name of a communication protocol between Babeltrace and the
+ relay daemon which makes it possible to see events "live",
+ as they are received by the relay daemon.
++
+Do not hyphenate.
+
+the +`lttng`+ tool::
+the +`lttng`+ command line tool::
+ The `lttng` command line tool.
++
+When _tool_ has been mentioned in the previous sentences, you can use
++`lttng`+ alone.
+
+Makefile::
+ An input for the make tool.
++
+Do not use _makefile_ or _make file_.
+
+man page::
+ Unix-style reference manual page.
++
+Do not hyphenate.
+
+per-process buffering::
+ A buffering scheme in which each process has its own buffer for a
+ given user space channel.
++
+Do not use _per-PID buffering_.
+
+per-user buffering::
+ A buffering scheme in which all the processes of a user share the same
+ buffer for a given user space channel.
++
+Do not use _per-UID buffering_.
+
+probe::
+ An instrumentation point.
++
+Prefer _tracepoint_ when referring to a user space or Linux kernel
+LTTng tracepoint.
+
+real-time clock::
+ A clock which keeps track of the current time, including eventual
+ time corrections.
++
+Do not use _realtime clock_ or _real time clock_.
+
+relay daemon::
+ The LTTng relay daemon.
++
+Do not use _relayd_.
++
+Use +`lttng-relayd`+ to refer to the relay daemon executable.
+
+root user::
+ A superuser of a Linux system.
++
+Do not use +`root`+.
+
+session::
+ Do not use when referring to a _tracing session_.
+
+session daemon::
+ The LTTng session daemon.
++
+Do not use _sessiond_.
++
+Use +`lttng-sessiond`+ to refer to the session daemon
+executable.
+
+snapshot::
+ Copy of the current data of all the buffers of a given tracing
+ session, saved as a trace.
+
+sub-buffer::
+ One part of an LTTng ring buffer.
++
+Do not use _subbuffer_ since it's harder to read with the two
+contiguous b's.
+
+timestamp::
+ Time information attached to an event when it is emitted. This is not
+ necessarily a _Unix timestamp_.
++
+Do not use _time stamp_.
+
+trace::
+ As a verb: a user or a tracer can _trace_ an application.
+
+Trace Compass::
+ The Trace Compass project and application.
++
+Do not hyphenate. Do not use _Trace compass_, _TraceCompass_, or
+_Tracecompass_.
+
+tracepoint::
+ An instrumentation point using the tracepoint mechanism of
+ the Linux kernel or of LTTng-UST.
++
+Do not use _trace point_ or _trace-point_.
+
+tracepoint definition::
+ The definition of a single tracepoint.
+
+tracepoint name::
+ The name of a _tracepoint_.
++
+Not to be confused with an _event name_.
+
+tracepoint provider::
+ A set of functions providing tracepoints to an instrumented user
+ application.
++
+Not to be confused with a _tracepoint provider package_: many tracepoint
+providers can exist within a tracepoint provider package.
+
+tracepoint provider package::
+ One or more tracepoint providers compiled as an object file or as
+ a shared library.
+
+tracing domain::
+ An LTTng tracing domain.
++
+Always use the complete _tracing domain_ term, not _domain_ alone,
+unless _tracing domain_ has been used in the few preceding sentences.
+
+tracing group::
+ The Unix group in which a user can be to be allowed to trace the
+ Linux kernel.
++
+Do not use _`tracing` group_, as the name of the tracing
+group is configurable.
+
+tracing session::
+ An LTTng tracing session.
++
+Always use the complete _tracing session_ term, not _session_ alone.
+
+Unix::
+ Unix operating system or philosophy.
++
+Do not use _UNIX_.
+
+Unix epoch::
+ Absolute reference of a real-time clock.
++
+Use the term as a proper noun: do not precede it with _the_.
++
+Do not use _Epoch_ alone.
+
+Unix timestamp::
+ Timestamp represented as the number of seconds since Unix epoch.
+
+use case::
+ According to Wikipedia: List of actions or event steps, typically
+ defining the interactions between a role and a system, to
+ achieve a goal.
++
+Do not hyphenate.
+
+user application::
+ An application running in user space, as opposed to a Linux kernel
+ module, for example.
++
+Do not use _user space application_, as this is redundant.
+
+user space::
+ User processes.
++
+Do not hyphenate.
+++ /dev/null
-Contributor's guide
-===================
-
-This guide presents the structure and conventions of the LTTng
-Documentation's source. Make sure you read it thoroughly before
-contributing a change.
-
-
-Branches
---------
-
-The online documentation published at <http://lttng.org/docs/> is always
-compiled from the sources of this repository's latest stable branch.
-The `master` branch contains the current documentation of the upcoming
-LTTng release.
-
-
-Structure of sources
---------------------
-
-`toc/docs.yml` is a YAML tree of all chapters, sections and subsections.
-It indicates which unique ID is linked to which position in the
-hierarchy and its true title.
-
-In the `contents` directory, the `preface.md` file is the preface contents.
-Each chapter has its own directory (directory names are not significant).
-Within those, `intro.md` files are partial introductions and then each
-section has its own directory, and so on, unless a section has no
-subsections, in which case all its contents is in a single Markdown file
-named _more or less_ like its ID.
-
-Each Markdown file begins with a YAML front matter which only contains
-the unique ID of this chapter/section:
-
-```yaml
----
-id: unique-id-goes-here
----
-
-First paragraph goes here.
-```
-
-Editable image sources are placed in `images/src` and their rendered
-equivalents are located in `images/export`.
-
-`tools/docs2json.py` is a Python 3 script which may be used to get
-the graph of internal and external links and to find
-typical errors in the whole documentation, like dead internal links.
-It needs the
-[`termcolor`](https://pypi.python.org/pypi/termcolor) Python 3 package.
-Run it from the repository's root and ignore its standard output
-to view the warnings and errors:
-
- tools/docs2json.py > /dev/null
-
-
-Format of sources
------------------
-
-The sources are made of a fusion of Markdown and HTML processed by
-[kramdown](http://kramdown.gettalong.org/). Markdown is preferred,
-HTML being only used for specific cases that need special formatting
-not available using plain Markdown. The kramdown processor is clever
-enough to support both languages in the same file, even in the same
-paragraph!
-
-
-### HTML specifics
-
-Here's a list of HTML blocks and inline code used throughout the
-document. If you need to contribute, please use them when needed to
-preserve the document's visual consistency.
-
-
-#### Tip/note/warning/error blocks
-
-Tip/note block:
-
-```html
-<div class="tip">
- <p>
- <span class="t">Title goes here followed by colon:</span>Text goes
- here; plain HTML.
- </p>
- <p>
- Multiple paragraphs is allowed.
- </p>
-</div>
-```
-
-Replace the `tip` class with `warn` for a warning block, and with `err`
-for an error message block (when JavaScript is needed but is disabled, etc.).
-
-Title should be `Tip:` for a tip, `Note:` for a note, `Warning:` for a
-warning, and `Error:` for an error.
-
-
-#### External links
-
-Internal links should always use Markdown
-(`[caption](#doc-section)`). External links, however, need a special
-style and must use the `<a>` tag with the `ext` CSS class:
-
-```html
-The LTTng Documentation is
-<a href="https://github.com/lttng/lttng-docs" class="ext">public</a>.
-```
-
-Sometimes, however, it is necessary to write internal links in plain
-HTML, for example in tip blocks, since Markdown code is not processed.
-In these cases, add the `int` CSS class as a hint to prevent the static
-analyzer from complaining (`tools/checkdocs.py`).
-
-
-#### Abbreviations
-
-Use `<abbr>` for describing abbreviations. This should only be used
-for the first use of the abbreviation:
-
-```html
-The <abbr title="Linux Trace Toolkit: next generation">LTTng</abbr>
-project is an open source system software package [...]
-```
-
-
-#### Non-breaking spaces
-
-Sometimes, a non-breaking space HTML entity (` `) needs to be
-explicitly written.
-
-Examples:
-
-```html
-The size of this file is 1039 bytes.
-
-This integer is displayed in base 16.
-
-A check is performed every 3000 ms.
-```
-
-
-#### Placeholders in inline code
-
-You must use `<em>` to emphasize a placeholder within a `<code>` tag
-because Markdown backticks (<code>`</code>) always render their
-content literally:
-
-```html
-Name your file <code>something_<em>sys</em>.c</code>, where
-<code><em>sys</em></code> is your system name.
-```
-
-
-#### Terminal boxes
-
-A terminal box, where command lines are shown, is a simple `<pre>`
-with the `term` class:
-
-```html
-<pre class="term">
-echo This is a terminal box
-</pre>
-```
-
-Do not prefix command lines with prompts (`$`/`#`) since this makes
-copy/paste operations painful.
-
-You may use `<strong>` tags to emphasize a part of the command line:
-
-```html
-<pre class="term">
-echo This is a <strong>terminal</strong> box
-</pre>
-```
-
-Results of commands, if needed, should be presented in a simple
-`text` kramdown code block:
-
-<pre>
-~~~ text
-[15:30:34.835895035] (+?.?????????) hostname hello_world: { cpu_id = 1 }, { my_int = 8, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
-[15:30:42.262781421] (+7.426886386) hostname hello_world: { cpu_id = 1 }, { my_int = 9, char0 = 80, char1 = 97, product = "Patriot Memory" }
-[15:30:48.175621778] (+5.912840357) hostname hello_world: { cpu_id = 1 }, { my_int = 10, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
-~~~
-</pre>
-
-
-#### Images
-
-Use
-
-```html
-<figure class="img img-70">
- <img src="/images/docs26/image-name.png" alt="Short description">
-</figure>
-```
-
-Replace `docs26` with the appropriate version tag depending on the
-checked out branch.
-
-to display an image. Change `img-70` to `img-` followed by the
-width percentage you wish.
-
-
-Convention
-----------
-
-A few rules to comply with in order to keep the text as
-consistent as possible:
-
- * Use _user space_, not _userspace_ nor _user-space_.
- (neither _user land_).
- * Use _file system_, not _filesystem_.
- * Use _use case_, not _use-case_ nor _usecase_.
- * Use _the C standard library_, not _libc_.
- * Use _log level_, not _loglevel_.
- * Use complete LTTng project names: _LTTng-modules_, _LTTng-UST_ and
- _LTTng-tools_, not _modules_, _UST_ and _tools_.
- * All code snippets should use 4 spaces for indentation (even C)
- so that they are not too large.
- * Prefer emphasis (Markdown: `_something_`, HTML: `<em>something</em>`)
- to strong (Markdown: `**something**`, HTML: `<strong>something</strong>`)
- for emphasizing text.
- * Try to stay behind the 72th column mark if possible, and behind
- the 80th column otherwise.
- * Do not end directory paths with a forward slash
- (good: `include/trace/events`, bad: `include/trace/events/`).
- * Minimize the use of the future tense (_will_).
- * Do not use Latin abbreviations (_e.g._, _i.e._, _etc._).
-
-
-Committing
-----------
-
-If you make a change to a single contents file, prefix your Git commit
-message's first line with the file ID followed by `: `, e.g:
-
- archlinux: minor fix
--- /dev/null
+CONF = asciidoc.html5.conf
+PREFIX = lttng-docs
+ALLVERSIONS = 2.5 2.6 2.7
+
+ASCIIDOC = asciidoc -v -f $(CONF) -a source-highlighter=pygments
+RM = rm -rf
+
+htmldst = $(1)/$(PREFIX)-$(1).html
+
+.PHONY: all
+
+all: $(ALLVERSIONS)
+
+.PHONY: $(ALLVERSIONS)
+
+2.5: $(call htmldst,2.5)
+
+2.6: $(call htmldst,2.6)
+
+2.7: $(call htmldst,2.7)
+
+%.html: %.txt $(CONF)
+ $(ASCIIDOC) $<
+
+.PHONY: clean
+
+clean:
+ $(RM) $(foreach version,$(ALLVERSIONS),$(call htmldst,$(version)))
--- /dev/null
+The LTTng Documentation: Source
+===============================
+
+This is the official repository of the
+http://lttng.org/docs[online LTTng Documentation's] source.
+
+
+== Generate
+
+You can generate your own version of the LTTng Documentation by running:
+
+----
+make
+----
+
+The generated HTML file is named
++__VERSION__/lttng-docs-__VERSION__.html+, where +__VERSION__+ is the
+version of the LTTng project documented by this file.
+
+You can also make a specific version:
+
+----
+make 2.7
+----
+
+NOTE: The SVG animations rendered on the LTTng website are not
+available in the generated version.
+
+You need the following tools to generate the LTTng Documentation:
+
+* http://www.methods.co.nz/asciidoc/[AsciiDoc] >= 8.6.5
+* http://pygments.org/[Pygments] and the `pygmentize` command
+
+
+== Contribute
+
+To contribute:
+
+. Fork this repo.
+. Make your change, following the link:CONTRIBUTING.adoc[contributor's guide].
+. Create a pull request.
+
+Thanks for your contributions and fixes!
+++ /dev/null
-LTTng Documentation
-===================
-
-This is the official repository of the
-[online LTTng Documentation's](http://lttng.org/docs) source.
-
-To contribute:
-
- 1. Fork this repo
- 2. Checkout the appropriate branch (`stable-2.x` or `master`)
- 3. Make your change, following the [contributor's guide](CONTRIBUTING.md)
- 4. Create a pull request
-
-Thanks for your contributions and fixes!
--- /dev/null
+[attributes]
+toc=
+toclevels=6
+imagesdir=images/export
+autowidth-option=
+note-no-anim="Animations are not supported in this output."
+
+[macros]
+(?su)[\\]?(?P<name>man):(?P<page>\S*?)\((?P<section>.*?)\)=
+(?su)[\\]?(?P<name>path):\{(?P<path>[^}]+)\}=
+(?su)[\\]?(?P<name>dir):\{(?P<dir>[^}]+)\}=
+(?su)[\\]?(?P<name>env):(?P<var>[a-zA-Z0-9_]+)=
+(?su)[\\]?(?P<name>cmd):(?P<cmd>[a-zA-Z0-9_-]+)=
+(?su)[\\]?(?P<name>noch):\{(?P<text>[^}]+)\}=
+
+[man-inlinemacro]
+<code><strong>{page}</strong>({section})</code>
+
+[path-inlinemacro]
+<code>{path}</code>
+
+[dir-inlinemacro]
+<code>{dir}</code>
+
+[env-inlinemacro]
+<code>{var}</code>
+
+[cmd-inlinemacro]
+<code>{cmd}</code>
+
+[noch-inlinemacro]
+<span style="text-transform: normal;">{text}</span>
+
+[tabledef-default]
+style=def
+def-style=options=("header",)
--- /dev/null
+[[acknowledgements]]
+=== Acknowledgements
+
+A few people made the online LTTng Documentation possible.
+
+http://eepp.ca/[Philippe Proulx] wrote most of the content, created
+the diagrams, and formatted the document. He is the current maintainer
+of the LTTng Documentation.
+
+Daniel U. Thibault, from the DRDC, wrote an open guide called _LTTng:
+The Linux Trace Toolkit Next Generation{nbsp}—{nbsp}A
+Comprehensive User's Guide (version 2.3 edition)_ which was used
+to complete parts of the <<core-concepts,Core concepts>> and
+<<plumbing,Components of LTTng>> sections and for a few passages here
+and there.
+
+The entire http://www.efficios.com/[EfficiOS] team made essential reviews
+of the whole document.
+
+We sincerely thank everyone who helped enhance the quality of this
+documentation.
--- /dev/null
+[[audience]]
+=== Target audience
+
+The material of this documentation is appropriate for intermediate to
+advanced software developers working in a Linux environment and
+interested in efficient software tracing. LTTng is also worth a
+try for students interested in the inner mechanics of their systems.
+
+If you do not have a programming background, you may wish to skip
+everything related to instrumentation, which often requires at least
+some programming language skills.
--- /dev/null
+[[convention]]
+=== Convention
+
+Function names, parameter names, variable names, command names, argument
+names, file system paths, file names, and other literal strings are
+written using a `monospace typeface` in this document. An +_italic_
+word+ within such a block is a placeholder, usually described in the
+following sentence.
+
+Practical tips and notes are given throughout the document using the
+following style:
+
+TIP: Make sure you read the tips.
+
+Terminal boxes are used to show command lines:
+
+[role="term"]
+----
+echo This is a terminal box
+----
+
+Typical command prompts, like `$` and `#`, are not shown in terminal
+boxes to make copy operations easier, especially for multiline commands
+which you can copy and paste as is to your terminal. Command lines
+which you need to execute as a root user start with cmd:sudo.
--- /dev/null
+[[copyright]]
+== Copyright
+
+Copyright © 2014-2016 http://lttng.org/[The LTTng Project]
+
+This work is licensed under a
+http://creativecommons.org/licenses/by/4.0/[Creative Commons
+Attribution 4.0 International License].
--- /dev/null
+To build the tracepoint provider package object file:
+
+* Compile the <<tpp-source,tracepoint provider package source file>>:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c tpp.c
+----
+--
--- /dev/null
+To build the tracepoint provider package object file:
+
+* Compile the <<tpp-source,tracepoint provider package source file>>:
++
+--
+[role="term"]
+----
+gcc -I. -c tpp.c
+----
+--
--- /dev/null
+To build the tracepoint provider package shared object:
+
+. Compile the <<tpp-source,tracepoint provider package source file>>:
++
+--
+[role="term"]
+----
+gcc -I. -fpic -c tpp.c
+----
+--
+
+. Build the tracepoint provider package shared object:
++
+--
+[role="term"]
+----
+gcc -shared -o libtpp.so tpp.o -llttng-ust -ldl
+----
+--
--- /dev/null
+[WARNING]
+[role="not-maintained"]
+=====
+This version of the LTTng Documentation is **not maintained anymore**,
+since the corresponding version of LTTng is not the current release,
+nor the previous release.
+
+Visit the link:/docs[latest LTTng Documentation version].
+=====
--- /dev/null
+[[welcome]]
+== Welcome!
+
+Welcome to the **LTTng Documentation**!
+
+The _Linux Trace Toolkit: next generation_ is an open source software
+toolkit used to simultaneously trace the Linux kernel, user applications,
+and user libraries. LTTng consists of <<lttng-modules,kernel modules>>
+(for Linux kernel tracing) and dynamically loaded
+<<lttng-ust,libraries>> (for user application and library tracing).
+A <<lttng-sessiond,session daemon>> controls the LTTng tracers after it
+receives commands from the <<lttng-cli,cmd:lttng command-line
+tool>>.
+
+[NOTE]
+.Open source documentation
+====
+This is an **open documentation**: its source is available in a
+https://github.com/lttng/lttng-docs[public Git repository].
+
+Should you find any error in the content of this text, any grammatical
+mistake, or any dead link, we would be very grateful if you would file a
+GitHub issue for it or, even better, contribute a patch to this
+documentation by creating a pull request.
+====
+++ /dev/null
----
-id: getting-started
----
-
-This is a small guide to get started quickly with LTTng kernel and user
-space tracing. For a more thorough understanding of LTTng and intermediate
-to advanced use cases and, see [Understanding LTTng](#doc-understanding-lttng)
-and [Using LTTng](#doc-using-lttng).
-
-Before reading this guide, make sure LTTng
-[is installed](#doc-installing-lttng). LTTng-tools is required. Also install
-LTTng-modules for
-[tracing the Linux kernel](#doc-tracing-the-linux-kernel) and LTTng-UST
-for
-[tracing your own user space applications](#doc-tracing-your-own-user-application).
-When the traces are finally written and complete, the
-[Viewing and analyzing your traces](#doc-viewing-and-analyzing-your-traces)
-section of this chapter will help you analyze your tracepoint events
-to investigate.
+++ /dev/null
----
-id: tracing-the-linux-kernel
----
-
-Make sure LTTng-tools and LTTng-modules packages
-[are installed](#doc-installing-lttng).
-
-Since you're about to trace the Linux kernel itself, let's look at the
-available kernel events using the `lttng` tool, which has a
-Git-like command line structure:
-
-<pre class="term">
-lttng list --kernel
-</pre>
-
-Before tracing, you need to create a session:
-
-<pre class="term">
-sudo lttng create
-</pre>
-
-<div class="tip">
-<p>
- <span class="t">Tip:</span>You can avoid using <code>sudo</code> in
- the previous and following commands if your user is a member of the
- <a href="#doc-lttng-sessiond" class="int"><code>tracing</code>
- group</a>.
-</p>
-</div>
-
-Let's now enable some events for this session:
-
-<pre class="term">
-sudo lttng enable-event --kernel sched_switch,sched_process_fork
-</pre>
-
-Or you might want to simply enable all available kernel events (beware
-that trace files grow rapidly when doing this):
-
-<pre class="term">
-sudo lttng enable-event --kernel --all
-</pre>
-
-Start tracing:
-
-<pre class="term">
-sudo lttng start
-</pre>
-
-By default, traces are saved in
-<code>~/lttng-traces/<em>name</em>-<em>date</em>-<em>time</em></code>,
-where <code><em>name</em></code> is the session name.
-
-When you're done tracing:
-
-<pre class="term">
-sudo lttng stop
-sudo lttng destroy
-</pre>
-
-Although `destroy` looks scary here, it doesn't actually destroy the
-written trace files: it only destroys the tracing session.
-
-What's next? Have a look at
-[Viewing and analyzing your traces](#doc-viewing-and-analyzing-your-traces)
-to view and analyze the trace you just recorded.
+++ /dev/null
----
-id: tracing-your-own-user-application
----
-
-The previous section helped you create a trace out of Linux kernel events.
-This section steps you through a simple example showing you how to trace
-a _Hello world_ program written in C.
-
-Make sure the LTTng-tools and LTTng-UST packages
-[are installed](#doc-installing-lttng).
-
-Tracing is just like having `printf()` calls at specific locations of
-your source code, albeit LTTng is much faster and more flexible than
-`printf()`. In the LTTng realm, **`tracepoint()`** is analogous to
-`printf()`.
-
-Unlike `printf()`, though, `tracepoint()` does not use a format string to
-know the types of its arguments: the formats of all tracepoints must be
-defined before using them. So before even writing our _Hello world_ program,
-we need to define the format of our tracepoint. This is done by creating a
-**tracepoint provider**, which consists of a tracepoint provider header
-(`.h` file) and a tracepoint provider definition (`.c` file).
-
-The tracepoint provider header contains some boilerplate as well as a
-list of tracepoint definitions and other optional definition entries
-which we skip for this quickstart. Each tracepoint is defined using the
-`TRACEPOINT_EVENT()` macro. For each tracepoint, you must provide:
-
- * a **provider name**, which is the "scope" or namespace of this
- tracepoint (this usually includes the company and project names)
- * a **tracepoint name**
- * a **list of arguments** for the eventual `tracepoint()` call, each
- item being:
- * the argument C type
- * the argument name
- * a **list of fields**, which correspond to the actual fields of the
- recorded events for this tracepoint
-
-Here's an example of a simple tracepoint provider header with two
-arguments: an integer and a string:
-
-~~~ c
-#undef TRACEPOINT_PROVIDER
-#define TRACEPOINT_PROVIDER hello_world
-
-#undef TRACEPOINT_INCLUDE
-#define TRACEPOINT_INCLUDE "./hello-tp.h"
-
-#if !defined(_HELLO_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
-#define _HELLO_TP_H
-
-#include <lttng/tracepoint.h>
-
-TRACEPOINT_EVENT(
- hello_world,
- my_first_tracepoint,
- TP_ARGS(
- int, my_integer_arg,
- char*, my_string_arg
- ),
- TP_FIELDS(
- ctf_string(my_string_field, my_string_arg)
- ctf_integer(int, my_integer_field, my_integer_arg)
- )
-)
-
-#endif /* _HELLO_TP_H */
-
-#include <lttng/tracepoint-event.h>
-~~~
-
-The exact syntax is well explained in the
-[C application](#doc-c-application) instrumentation guide of the
-[Using LTTng](#doc-using-lttng) chapter, as well as in the
-<a href="/man/3/lttng-ust/v2.7" class="ext">LTTng-UST man page</a>.
-
-Save the above snippet as `hello-tp.h`.
-
-Write the tracepoint provider definition as `hello-tp.c`:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-#define TRACEPOINT_DEFINE
-
-#include "hello-tp.h"
-~~~
-
-Create the tracepoint provider:
-
-<pre class="term">
-gcc -c -I. hello-tp.c
-</pre>
-
-Now, by including `hello-tp.h` in your own application, you may use the
-tracepoint defined above by properly refering to it when calling
-`tracepoint()`:
-
-~~~ c
-#include <stdio.h>
-#include "hello-tp.h"
-
-int main(int argc, char *argv[])
-{
- int x;
-
- puts("Hello, World!\nPress Enter to continue...");
-
- /*
- * The following getchar() call is only placed here for the purpose
- * of this demonstration, for pausing the application in order for
- * you to have time to list its events. It's not needed otherwise.
- */
- getchar();
-
- /*
- * A tracepoint() call. Arguments, as defined in hello-tp.h:
- *
- * 1st: provider name (always)
- * 2nd: tracepoint name (always)
- * 3rd: my_integer_arg (first user-defined argument)
- * 4th: my_string_arg (second user-defined argument)
- *
- * Notice the provider and tracepoint names are NOT strings;
- * they are in fact parts of variables created by macros in
- * hello-tp.h.
- */
- tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");
-
- for (x = 0; x < argc; ++x) {
- tracepoint(hello_world, my_first_tracepoint, x, argv[x]);
- }
-
- puts("Quitting now!");
-
- tracepoint(hello_world, my_first_tracepoint, x * x, "x^2");
-
- return 0;
-}
-~~~
-
-Save this as `hello.c`, next to `hello-tp.c`.
-
-Notice `hello-tp.h`, the tracepoint provider header, is included
-by `hello.c`.
-
-You are now ready to compile the application with LTTng-UST support:
-
-<pre class="term">
-gcc -c hello.c
-gcc -o hello hello.o hello-tp.o -llttng-ust -ldl</strong>
-</pre>
-
-Here's the whole build process:
-
-<figure class="img img-100">
-<img src="/images/docs26/ust-flow.png" alt="User space tracing's build process">
-<figcaption>
- User space tracing build process
-</figcaption>
-</figure>
-
-If you followed the
-[Tracing the Linux kernel](#doc-tracing-the-linux-kernel) tutorial, the
-following steps should look familiar.
-
-First, run the application with a few arguments:
-
-<pre class="term">
-./hello world and beyond
-</pre>
-
-You should see
-
-~~~ text
-Hello, World!
-Press Enter to continue...
-~~~
-
-Use the `lttng` tool to list all available user space events:
-
-<pre class="term">
-lttng list --userspace
-</pre>
-
-You should see the `hello_world:my_first_tracepoint` tracepoint listed
-under the `./hello` process.
-
-Create a tracing session:
-
-<pre class="term">
-lttng create
-</pre>
-
-Enable the `hello_world:my_first_tracepoint` tracepoint:
-
-<pre class="term">
-lttng enable-event --userspace hello_world:my_first_tracepoint
-</pre>
-
-Start tracing:
-
-<pre class="term">
-lttng start
-</pre>
-
-Go back to the running `hello` application and press Enter. All `tracepoint()`
-calls are executed and the program finally exits.
-
-Stop tracing:
-
-<pre class="term">
-lttng stop
-</pre>
-
-Done! You may use `lttng view` to list the recorded events. This command
-starts
-<a href="http://diamon.org/babeltrace" class="ext"><code>babeltrace</code></a>
-in the background, if it's installed:
-
-<pre class="term">
-lttng view
-</pre>
-
-should output something like:
-
-~~~ text
-[18:10:27.684304496] (+?.?????????) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "hi there!", my_integer_field = 23 }
-[18:10:27.684338440] (+0.000033944) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "./hello", my_integer_field = 0 }
-[18:10:27.684340692] (+0.000002252) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "world", my_integer_field = 1 }
-[18:10:27.684342616] (+0.000001924) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "and", my_integer_field = 2 }
-[18:10:27.684343518] (+0.000000902) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "beyond", my_integer_field = 3 }
-[18:10:27.684357978] (+0.000014460) hostname hello_world:my_first_tracepoint: { cpu_id = 0 }, { my_string_field = "x^2", my_integer_field = 16 }
-~~~
-
-When you're done, you may destroy the tracing session, which does _not_
-destroy the generated trace files, leaving them available for further
-analysis:
-
-<pre class="term">
-lttng destroy
-</pre>
-
-The next section presents other alternatives to view and analyze your
-LTTng traces.
+++ /dev/null
----
-id: viewing-and-analyzing-your-traces
----
-
-This section describes how to visualize the data gathered after tracing
-the Linux kernel or a user space application.
-
-Many ways exist to read LTTng traces:
-
- * **`babeltrace`** is a command line utility which converts trace formats;
- it supports the format used by LTTng,
- <abbr title="Common Trace Format">CTF</abbr>, as well as a basic
- text output which may be `grep`ed. The `babeltrace` command is
- part of the
- <a href="http://diamon.org/babeltrace" class="ext">Babeltrace</a> project.
- * Babeltrace also includes **Python bindings** so that you may
- easily open and read an LTTng trace with your own script, benefiting
- from the power of Python.
- * **<a href="http://tracecompass.org/" class="ext">Trace Compass</a>**
- is an Eclipse plugin used to visualize and analyze various types of
- traces, including LTTng's. It also comes as a standalone application.
-
-LTTng trace files are recorded in the `~/lttng-traces` directory by default.
-Let's now view the trace and perform a basic analysis using
-`babeltrace`.
-
-The simplest way to list all the recorded events of a trace is to pass its
-path to `babeltrace` with no options:
-
-<pre class="term">
-babeltrace ~/lttng-traces/my-session
-</pre>
-
-`babeltrace` finds all traces recursively within the given path and
-prints all their events, merging them in order of time.
-
-Listing all the system calls of a Linux kernel trace with their arguments is
-easy with `babeltrace` and `grep`:
-
-<pre class="term">
-babeltrace ~/lttng-traces/my-kernel-session | grep sys_
-</pre>
-
-Counting events is also straightforward:
-
-<pre class="term">
-babeltrace ~/lttng-traces/my-kernel-session | grep sys_read | wc --lines
-</pre>
-
-The text output of `babeltrace` is useful for isolating events by simple
-matching using `grep` and similar utilities. However, more elaborate filters
-such as keeping only events with a field value falling within a specific range
-are not trivial to write using a shell. Moreover, reductions and even the
-most basic computations involving multiple events are virtually impossible
-to implement.
-
-Fortunately, Babeltrace ships with Python 3 bindings which makes it
-really easy to read the events of an LTTng trace sequentially and compute
-the desired information.
-
-Here's a simple example using the Babeltrace Python bindings. The following
-script accepts an LTTng Linux kernel trace path as its first argument and
-prints the short names of the top 5 running processes on CPU 0 during the
-whole trace:
-
-~~~ python
-import sys
-from collections import Counter
-import babeltrace
-
-
-def top5proc():
- if len(sys.argv) != 2:
- msg = 'Usage: python {} TRACEPATH'.format(sys.argv[0])
- raise ValueError(msg)
-
- # a trace collection holds one to many traces
- col = babeltrace.TraceCollection()
-
- # add the trace provided by the user
- # (LTTng traces always have the 'ctf' format)
- if col.add_trace(sys.argv[1], 'ctf') is None:
- raise RuntimeError('Cannot add trace')
-
- # this counter dict will hold execution times:
- #
- # task command name -> total execution time (ns)
- exec_times = Counter()
-
- # this holds the last `sched_switch` timestamp
- last_ts = None
-
- # iterate events
- for event in col.events:
- # keep only `sched_switch` events
- if event.name != 'sched_switch':
- continue
-
- # keep only events which happened on CPU 0
- if event['cpu_id'] != 0:
- continue
-
- # event timestamp
- cur_ts = event.timestamp
-
- if last_ts is None:
- # we start here
- last_ts = cur_ts
-
- # previous task command (short) name
- prev_comm = event['prev_comm']
-
- # initialize entry in our dict if not yet done
- if prev_comm not in exec_times:
- exec_times[prev_comm] = 0
-
- # compute previous command execution time
- diff = cur_ts - last_ts
-
- # update execution time of this command
- exec_times[prev_comm] += diff
-
- # update last timestamp
- last_ts = cur_ts
-
- # display top 10
- for name, ns in exec_times.most_common(5):
- s = ns / 1000000000
- print('{:20}{} s'.format(name, s))
-
-
-if __name__ == '__main__':
- top5proc()
-~~~
-
-Save this script as `top5proc.py` and run it with Python 3, providing the
-path to an LTTng Linux kernel trace as the first argument:
-
-<pre class="term">
-python3 top5proc.py ~/lttng-sessions/my-session-.../kernel
-</pre>
-
-Make sure the path you provide is the directory containing actual trace
-files (`channel0_0`, `metadata`, and the rest): the `babeltrace` utility
-recurses directories, but the Python bindings do not.
-
-Here's an example of output:
-
-~~~ text
-swapper/0 48.607245889 s
-chromium 7.192738188 s
-pavucontrol 0.709894415 s
-Compositor 0.660867933 s
-Xorg.bin 0.616753786 s
-~~~
-
-Note that `swapper/0` is the "idle" process of CPU 0 on Linux; since we
-weren't using the CPU that much when tracing, its first position in the list
-makes sense.
+++ /dev/null
----
-id: building-from-source
----
-
-As [previously stated](#doc-installing-lttng), LTTng is shipped as
-three packages: LTTng-tools, LTTng-modules and LTTng-UST. LTTng-tools
-contains everything needed to control tracing sessions, while
-LTTng-modules is only needed for Linux kernel tracing and LTTng-UST is
-only needed for user space tracing.
-
-The tarballs are available in the
-<a href="http://lttng.org/download#build-from-source" class="ext">Download
-section</a> of the LTTng website.
-
-Please refer to the `README.md` files provided by each package to
-properly build and install them.
-
-<div class="tip">
-<p>
-<span class="t">Tip:</span>The aforementioned <code>README.md</code> files
-are rendered as rich text when
-<a href="https://github.com/lttng" class="ext">viewed on GitHub</a>.
-</p>
-</div>
+++ /dev/null
----
-id: archlinux
----
-
-LTTng 2.7 packages are currently available in the
-<abbr title="Arch User Repository">AUR</abbr> under the following names:
-<a href="https://aur.archlinux.org/packages/lttng-tools/" class="ext"><code>lttng-tools</code></a>,
-<a href="https://aur.archlinux.org/packages/lttng-modules/" class="ext"><code>lttng-modules</code></a>
-and
-<a href="https://aur.archlinux.org/packages/lttng-ust/" class="ext"><code>lttng-ust</code></a>.
-
-The three LTTng packages can be installed using the following
-<a href="https://wiki.archlinux.org/index.php/yaourt" class="ext">Yaourt</a> commands:
-
-<pre class="term">
-yaourt -S lttng-tools
-yaourt -S lttng-modules
-yaourt -S lttng-ust
-</pre>
-
-If you need to trace [Python applications](#doc-python-application), you
-need to install the LTTng-UST Python agent also:
-
- * Python 3: <a href="https://aur.archlinux.org/packages/python-lttngust/" class="ext"><code>python-lttngust</code></a>
- * Python 2: <a href="https://aur.archlinux.org/packages/python2-lttngust/" class="ext"><code>python2-lttngust</code></a>
-
-<div class="tip">
-<p>
- <span class="t">Note:</span> If you need to trace
- <a href="#doc-java-application" class="int">Java</a>
- applications on Arch Linux, you need to build and install LTTng-UST 2.7
- <a href="#doc-building-from-source" class="int">from source</a> and
- use the <code>--enable-java-agent-jul</code>,
- <code>--enable-java-agent-log4j</code>, or
- <code>--enable-java-agent-all</code> options.
-</p>
-</div>
+++ /dev/null
----
-id: debian
----
-
-Debian "stretch" and Debian "sid" have LTTng-modules 2.7 and
-LTTng-UST 2.7 packages:
-
-<pre class="term">
-sudo apt-get install lttng-modules-dkms
-sudo apt-get install liblttng-ust-dev
-</pre>
-
-LTTng-tools 2.7 still needs to be
-[built from source](#doc-building-from-source).
-
-If you need to trace [Java applications](#doc-java-application), you
-need to install the LTTng-UST Java agent also:
-
-<pre class="term">
-sudo apt-get install liblttng-ust-agent-java
-</pre>
-
-If you need to trace [Python applications](#doc-python-application),
-you need to install the LTTng-UST Python agent also:
-
-<pre class="term">
-sudo apt-get install python3-lttngust
-</pre>
+++ /dev/null
----
-id: desktop-distributions
----
-
-Official and unofficial LTTng 2.7 packages are available for
-[Ubuntu](#doc-ubuntu), [Debian](#doc-debian),
-[openSUSE](#doc-opensuse) (and other RPM-based distributions), and
-[Arch Linux](#doc-archlinux).
-
-Older versions of LTTng are available for Fedora.
-
-Should any issue arise when
-following the procedures below, please inform the
-<a href="/community" class="ext">community</a> about it.
+++ /dev/null
----
-id: opensuse
----
-
-The openSUSE Leap 42.1 repository includes LTTng 2.7 packages.
-
-Use `zypper` directly:
-
-<pre class="term">
-sudo zypper install lttng-tools
-sudo zypper install lttng-modules
-sudo zypper install lttng-ust-devel
-</pre>
-
-<div class="tip">
-<p>
- <span class="t">Note:</span> If you need to trace
- <a href="#doc-java-application" class="int">Java applications</a> on
- openSUSE, you need to build and install LTTng-UST 2.7
- <a href="#doc-building-from-source" class="int">from source</a> and
- use the <code>--enable-java-agent-jul</code>,
- <code>--enable-java-agent-log4j</code>, or
- <code>--enable-java-agent-all</code> options.
-</p>
-<p>
- If you need to trace
- <a href="#doc-python-application" class="int">Python applications</a> on
- openSUSE, you need to build and install LTTng-UST 2.7 from source
- and use the <code>--enable-python-agent</code> option.
-</p>
-</div>
+++ /dev/null
----
-id: ubuntu
----
-
-LTTng 2.7 packages are available in Ubuntu 16.04 _Xenial Xerus_'s
-[official repositories](#doc-ubuntu-official-repositories).
-
-It is also possible to install the latest stable LTTng 2.7 release
-on any current Ubuntu release (including Ubuntu 12.04 _Precise Pangolin_
-and Ubuntu 14.04 _Trusty Tahr_) by using the
-[LTTng latest stable PPA](#doc-ubuntu-ppa).
+++ /dev/null
----
-id: ubuntu-official-repositories
----
-
-To install LTTng 2.7 from the official Ubuntu 16.04 _Xenial Xerus_
-repositories, simply use `apt-get`:
-
-<pre class="term">
-sudo apt-get install lttng-tools
-sudo apt-get install lttng-modules-dkms
-sudo apt-get install liblttng-ust-dev
-</pre>
-
-If you need to trace [Java applications](#doc-java-application), you
-need to install the LTTng-UST Java agent also:
-
-<pre class="term">
-sudo apt-get install liblttng-ust-agent-java
-</pre>
-
-If you need to trace [Python applications](#doc-python-application),
-you need to install the LTTng-UST Python agent also:
-
-<pre class="term">
-sudo apt-get install python3-lttngust
-</pre>
+++ /dev/null
----
-id: ubuntu-ppa
----
-
-The
-<a href="https://launchpad.net/~lttng/+archive/ubuntu/ppa/" class="ext">LTTng latest stable PPA</a>
-offers the latest stable LTTng 2.7 packages. To get packages from the
-PPA, follow these steps:
-
-<pre class="term">
-sudo apt-add-repository ppa:lttng/ppa
-sudo apt-get update
-sudo apt-get install lttng-tools
-sudo apt-get install lttng-modules-dkms
-sudo apt-get install liblttng-ust-dev
-sudo apt-get install liblttng-ust-agent-java
-sudo apt-get install python3-lttngust
-</pre>
+++ /dev/null
----
-id: enterprise-distributions
----
-
-To install LTTng on enterprise Linux distributions
-(such as <abbr title="Red Hat Enterprise Linux">RHEL</abbr> and
-<abbr title="SUSE Linux Enterprise Server">SLES</abbr>), please see
-<a href="http://packages.efficios.com/" class="ext">EfficiOS
-Enterprise Packages</a>.
+++ /dev/null
----
-id: installing-lttng
----
-
-**LTTng** is a set of software components which interact to allow
-instrumenting the Linux kernel and user applications as well as
-controlling tracing sessions (starting/stopping tracing,
-enabling/disabling events, and more). Those components are bundled into
-the following packages:
-
- * **LTTng-tools**: libraries and command line interface to control
- tracing sessions.
- * **LTTng-modules**: Linux kernel modules for tracing the kernel.
- * **LTTng-UST**: user space tracing library.
-
-Most distributions mark the LTTng-modules and LTTng-UST packages as
-optional. In the following sections, the steps to install all three are
-always provided, but note that LTTng-modules is only required if
-you intend to trace the Linux kernel and LTTng-UST is only required if
-you intend to trace user space applications.
-
-This chapter shows how to install the above packages on a Linux
-system. The easiest way is to use the package manager of the system's
-[distribution](#doc-desktop-distributions). Embedded distributions
-(Buildroot and OpenEmbedded/Yocto) currently have no packages of
-LTTng 2.7 (LTTng 2.6 is available for both of them). Support is also
-available for
-[enterprise distributions](#doc-enterprise-distributions), such as
-Red Hat Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SLES).
-Otherwise, you can
-[build the LTTng packages from source](#doc-building-from-source).
+++ /dev/null
----
-id: nuts-and-bolts
----
-
-What is LTTng? As its name suggests, the
-_Linux Trace Toolkit: next generation_ is a modern toolkit for
-tracing Linux systems and applications. So your first question might
-rather be: **what is tracing?**
+++ /dev/null
----
-id: lttng-alternatives
----
-
-Excluding proprietary solutions, a few competing software tracers
-exist for Linux:
-
- * <a href="https://www.kernel.org/doc/Documentation/trace/ftrace.txt" class="ext">ftrace</a>
- is the de facto function tracer of the Linux kernel. Its user
- interface is a set of special files in sysfs.
- * <a href="https://perf.wiki.kernel.org/" class="ext">perf</a> is
- a performance analyzing tool for Linux which supports hardware
- performance counters, tracepoints, as well as other counters and
- types of probes. perf's controlling utility is the `perf` command
- line/curses tool.
- * <a href="http://linux.die.net/man/1/strace" class="ext">strace</a>
- is a command line utility which records system calls made by a
- user process, as well as signal deliveries and changes of process
- state. strace makes use of
- <a href="https://en.wikipedia.org/wiki/Ptrace" class="ext">ptrace</a>
- to fulfill its function.
- * <a href="https://sourceware.org/systemtap/" class="ext">SystemTap</a>
- is a Linux kernel and user space tracer which uses custom user scripts
- to produce plain text traces. Scripts are converted to the C language,
- then compiled as Linux kernel modules which are loaded to produce
- trace data. SystemTap's primary user interface is the `stap`
- command line tool.
- * <a href="http://www.sysdig.org/" class="ext">sysdig</a>, like
- SystemTap, uses scripts to analyze Linux kernel events. Scripts,
- or _chisels_ in sysdig's jargon, are written in Lua and executed
- while the system is being traced, or afterwards. sysdig's interface
- is the `sysdig` command line tool as well as the curses-based
- `csysdig` tool.
-
-The main distinctive features of LTTng is that it produces correlated
-kernel and user space traces, as well as doing so with the lowest
-overhead amongst other solutions. It produces trace files in the
-<a href="http://diamon.org/ctf" class="ext"><abbr title="Common Trace Format">CTF</abbr></a>
-format, an optimized file format for production and analyses of
-multi-gigabyte data. LTTng is the result of close to 10 years of
-active development by a community of passionate developers. LTTng 2.7
-is currently available on some major desktop and server Linux
-distributions.
-
-The main interface for tracing control is a single command line tool
-named `lttng`. The latter can create several tracing sessions,
-enable/disable events on the fly, filter them efficiently with custom
-user expressions, start/stop tracing, and do much more. Traces can be
-recorded on disk or sent over the network, kept totally or partially,
-and viewed once tracing becomes inactive or in real-time.
-
-[Install LTTng now](#doc-installing-lttng) and start tracing!
+++ /dev/null
----
-id: what-is-tracing
----
-
-As the history of software engineering progressed and led to what
-we now take for granted—complex, numerous and
-interdependent software applications running in parallel on
-sophisticated operating systems like Linux—the authors of such
-components, or software developers, began feeling a natural
-urge of having tools to ensure the robustness and good performance
-of their masterpieces.
-
-One major achievement in this field is, inarguably, the
-<a href="https://www.gnu.org/software/gdb/" class="ext">GNU debugger
-(GDB)</a>, which is an essential tool for developers to find and fix
-bugs. But even the best debugger won't help make your software run
-faster, and nowadays, faster software means either more work done by
-the same hardware, or cheaper hardware for the same work.
-
-A _profiler_ is often the tool of choice to identify performance
-bottlenecks. Profiling is suitable to identify _where_ performance is
-lost in a given software; the profiler outputs a profile, a
-statistical summary of observed events, which you may use to discover
-which functions took the most time to execute. However, a profiler
-won't report _why_ some identified functions are the bottleneck.
-Bottlenecks might only occur when specific conditions are met, sometimes
-almost impossible to capture by a statistical profiler, or impossible to
-reproduce with an application altered by the overhead of an event-based
-profiler. For a thorough investigation of software performance issues,
-a history of execution, with the recorded values of chosen variables
-and context, is essential. This is where tracing comes in handy.
-
-_Tracing_ is a technique used to understand what goes on in a running
-software system. The software used for tracing is called a _tracer_,
-which is conceptually similar to a tape recorder. When recording,
-specific probes placed in the software source code generate events
-that are saved on a giant tape: a _trace_ file. Both user applications
-and the operating system may be traced at the same time, opening the
-possibility of resolving a wide range of problems that are otherwise
-extremely challenging.
-
-Tracing is often compared to _logging_. However, tracers and loggers
-are two different tools, serving two different purposes. Tracers are
-designed to record much lower-level events that occur much more
-frequently than log messages, often in the thousands per second range,
-with very little execution overhead. Logging is more appropriate for
-very high-level analysis of less frequent events: user accesses,
-exceptional conditions (errors and warnings, for example), database
-transactions, instant messaging communications, and such. More formally,
-logging is one of several use cases that can be accomplished with
-tracing.
-
-The list of recorded events inside a trace file may be read manually
-like a log file for the maximum level of detail, but it is generally
-much more interesting to perform application-specific analyses to
-produce reduced statistics and graphs that are useful to resolve a
-given problem. Trace viewers and analysers are specialized tools
-designed to do this.
-
-So, in the end, this is what LTTng is: a powerful, open source set of
-tools to trace the Linux kernel and user applications at the same time.
-LTTng is composed of several components actively maintained and
-developed by its <a href="/community/#where" class="ext">community</a>.
+++ /dev/null
----
-id: preface
----
-
-<div class="copyright">
- <p>
- Copyright © 2014-2015 The LTTng Project
- </p>
-
- <p>
- This work is licensed under a
- <a class="ext" href="http://creativecommons.org/licenses/by/4.0/">Creative
- Commons Attribution 4.0 International License</a>.
- </p>
-</div>
-
-
-## Welcome!
-
-Welcome to the **LTTng Documentation**!
-
-The _Linux Trace Toolkit: next generation_
-is an open source system software package for correlated tracing of the
-Linux kernel, user applications and libraries. LTTng consists of kernel
-modules (for Linux kernel tracing) and dynamically loaded libraries (for
-user application and library tracing). It is controlled by a session
-daemon, which receives commands from a command line interface.
-
-
-### Convention
-
-Function and argument names, variable names, command names,
-file system paths, file names and other precise strings are written
-using a <code>monospaced typeface</code> in this document. An
-<code><em>italic</em> word</code> within such a block is a
-placeholder, usually described in the following sentence.
-
-Practical tips and sidenotes are given throughout the document using a
-blue background:
-
-<div class="tip">
-<p><span class="t">Tip:</span>Make sure you read the tips.</p>
-</div>
-
-Terminal boxes are used to show command lines:
-
-<pre class="term">
-echo This is a terminal box
-</pre>
-
-Typical command prompts, like `$` and `#`, are not shown in terminal
-boxes to make copy/paste operations easier, especially for multiline
-commands which may be copied and pasted as is in a user's terminal.
-Commands to be executed as a root user begin with `sudo`.
-
-
-### Target audience
-
-The material of this documentation is appropriate for intermediate to
-advanced software developers working in a Linux environment who are
-interested in efficient software tracing. LTTng may also be worth a
-try for students interested in the inner mechanics of their systems.
-
-Readers who do not have a programming background may wish to skip
-everything related to instrumentation, which requires, most of the
-time, some programming language skills.
-
-<div class="tip">
-<p><span class="t">Note to readers:</span>This is an <strong>open
-documentation</strong>: its source is available in a
-<a class="ext" href="https://github.com/lttng/lttng-docs">public Git
-repository</a>. Should you find any error in the contents of this text,
-any grammatical mistake, or any dead link, we would be very grateful if
-you would fill a GitHub issue for it or, even better, contribute a patch
-to this documentation by creating a pull request.</p>
-</div>
-
-### Chapter descriptions
-
-What follows is a list of brief descriptions of this documentation's
-chapters. The latter are ordered in such a way as to make the reading
-as linear as possible.
-
- 1. [Nuts and bolts](#doc-nuts-and-bolts) explains the
- rudiments of software tracing and the rationale behind the
- LTTng project.
- 2. [Installing LTTng](#doc-installing-lttng) is divided into
- sections describing the steps needed to get a working installation
- of LTTng packages for common Linux distributions and from its
- source.
- 3. [Getting started](#doc-getting-started) is a very concise guide to
- get started quickly with LTTng kernel and user space tracing. This
- chapter is recommended if you're new to LTTng or to software tracing
- in general.
- 4. [Understanding LTTng](#doc-understanding-lttng) deals with some
- core concepts and components of the LTTng suite. Understanding
- those is important since the next chapter assumes you're familiar
- with them.
- 5. [Using LTTng](#doc-using-lttng) is a complete user guide of the
- LTTng project. It shows in great details how to instrument user
- applications and the Linux kernel, how to control tracing sessions
- using the `lttng` command line tool, and miscellaneous practical use
- cases.
- 6. [Reference](#doc-reference) contains references of LTTng components.
-
-We recommend that you read the above chapters in this order, although
-some of them may be skipped depending on your situation. You may skip
-[Nuts and bolts](#doc-nuts-and-bolts) if you're familiar with tracing
-and the LTTng project. Also, you may jump over
-[Installing LTTng](#doc-installing-lttng) if LTTng is already properly
-installed on your target system.
-
-
-### Acknowledgements
-
-A few people made the online LTTng Documentation possible.
-
-Philippe Proulx wrote and formatted most of the text.
-Daniel U. Thibault, from the
-<abbr title="Defence Research and Development Canada">DRDC</abbr>,
-wrote an open guide called <em>LTTng: The Linux Trace Toolkit Next
-Generation — A Comprehensive User's Guide (version 2.3
-edition)</em> which was mostly used to complete parts of the
-[Understanding LTTng](#doc-understanding-lttng) chapter and for a few
-passages here and there.
-The whole <a href="http://www.efficios.com/" class="ext">EfficiOS</a>
-team (Christian Babeux, Antoine Busque, Julien Desfossez,
-Mathieu Desnoyers, Jérémie Galarneau and David Goulet) made essential
-reviews of the whole document.
-
-We sincerely thank everyone who helped make this documentation what
-it is. We hope you enjoy reading it as much as we did writing it.
+++ /dev/null
----
-id: reference
----
-
-This chapter presents various references for LTTng packages such as links
-to online man pages, tables needed by the rest of the text, descriptions
-of library functions, and more.
+++ /dev/null
----
-id: lttng-modules-ref
----
-
-This section presents references of the LTTng-modules package.
+++ /dev/null
----
-id: lttng-modules-tp-fields
-since: 2.7
-since-note: |
- Previous versions used the TP_STRUCT__entry() and TP_fast_assign()
- macros which also used different inner macros. The new ctf_*()
- macros are similar to their LTTng-UST counterpart.
----
-
-<a name="doc-lttng-modules-tp-fast-assign"></a>
-<a name="doc-lttng-modules-tp-struct-entry"></a>
-The available macros to define tracepoint fields, which should be listed
-within `TP_FIELDS()` in `LTTNG_TRACEPOINT_EVENT()`, are:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th>Macro</th>
- <th>Description/arguments</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_integer(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_integer_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_integer(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_integer_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Standard integer, displayed in base 10</p>
- <ul>
- <li>
- <code class="arg"><strong>t</strong></code> integer C type
- (<code>int</code>, <code>long</code>,
- <code>size_t</code>, ...)
- </li>
- <li><code class="arg"><strong>n</strong></code> field name</li>
- <li><code class="arg"><strong>e</strong></code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_integer_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_integer_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Standard integer, displayed in base 16</p>
- <ul>
- <li><code class="arg"><strong>t</strong></code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td><code class="no-bg">ctf_integer_oct(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></td>
- <td>
- <p>Standard integer, displayed in base 8</p>
- <ul>
- <li><code class="arg"><strong>t</strong></code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_integer_network(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_integer_network(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Integer in network byte order (big-endian),
- displayed in base 10
- </p>
- <ul>
- <li><code class="arg">t</code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_integer_network_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_integer_network_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Integer in network byte order, displayed
- in base 16</p>
- <ul>
- <li><code class="arg">t</code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_float(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_float(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_float_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_float_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Floating point number</p>
- <ul>
- <li>
- <code class="arg">t</code> floating point number
- C type (<code>float</code>, <code>double</code>)
- </li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_string(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_string(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_string_nowrite(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_user_string_nowrite(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Null-terminated string; undefined behavior if
- <code class="arg">e</code> is <code>NULL</code>
- </p>
- <ul>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_array(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_user_array(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_array_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_user_array_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Statically-sized array of integers, displayed in
- base 10
- </p>
- <ul>
- <li><code class="arg">t</code> array element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">s</code> number of elements</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_array_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_user_array_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_array_text_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_user_array_text_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Statically-sized array, printed as text; no need to
- be null-terminated
- </p>
- <ul>
- <li><code class="arg">t</code> array element C type (always <code>char</code>)</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">s</code> number of elements</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_user_sequence(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_sequence_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_user_sequence_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array of integers, displayed in
- base 10; type of
- <code class="arg">E</code> needs to be unsigned
- </p>
- <ul>
- <li><code class="arg">t</code> sequence element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array of integers, displayed in
- base 16; type of
- <code class="arg">E</code> needs to be unsigned
- </p>
- <ul>
- <li><code class="arg">t</code> sequence element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence_network(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array of integers in network byte
- order (big endian), displayed in base 10;
- type of
-
-
-
- <code class="arg">E</code> needs to be unsigned
- </p>
- <ul>
- <li><code class="arg">t</code> sequence element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_user_sequence_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_sequence_text_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_user_sequence_text_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array, displayed as text; no need to
- be null-terminated; undefined behavior if
- <code class="arg">e</code> is <code>NULL</code></p>
- <ul>
- <li><code class="arg">t</code> sequence element C type (always <code>char</code>)</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-The `_user` versions must be used when the argument expression, `e`, is
-a user space address. In the cases of `ctf_user_integer*()` and
-`ctf_user_float*()`, `&e` must be a user space address, thus `e` must
-be addressable.
-
-The `_nowrite` versions omit themselves from the session trace, but are
-otherwise identical. This means the `_nowrite` fields won't be written
-in the recorded trace. Their primary purpose is to make some
-of the event context available to the
-[event filters](#doc-enabling-disabling-events) without having to
-commit the data to sub-buffers.
+++ /dev/null
----
-id: lttng-ust-ref
----
-
-This section presents references of the LTTng-UST package.
+++ /dev/null
----
-id: liblttng-ust
----
-
-The LTTng-UST library, or `liblttng-ust`, is the main shared object
-against which user applications are linked to make LTTng user space
-tracing possible.
-
-The [C application](#doc-c-application) guide shows the complete
-process to instrument, build and run a C/C++ application using
-LTTng-UST, while this section contains a few important tables.
+++ /dev/null
----
-id: liblttng-ust-tp-fields
----
-
-The available macros to define tracepoint fields, which should be listed
-within `TP_FIELDS()` in `TRACEPOINT_EVENT()`, are:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th>Macro</th>
- <th>Description/arguments</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_integer(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_integer_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Standard integer, displayed in base 10</p>
- <ul>
- <li>
- <code class="arg"><strong>t</strong></code> integer C type
- (<code>int</code>, <code>long</code>,
- <code>size_t</code>, ...)
- </li>
- <li><code class="arg"><strong>n</strong></code> field name</li>
- <li><code class="arg"><strong>e</strong></code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td><code class="no-bg">ctf_integer_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></td>
- <td>
- <p>Standard integer, displayed in base 16</p>
- <ul>
- <li><code class="arg"><strong>t</strong></code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td><code class="no-bg">ctf_integer_network(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></td>
- <td>
- <p>
- Integer in network byte order (big endian),
- displayed in base 10
- </p>
- <ul>
- <li><code class="arg">t</code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td><code class="no-bg">ctf_integer_network_hex(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></td>
- <td>
- <p>
- Integer in network byte order, displayed
- in base 16</p>
- <ul>
- <li><code class="arg">t</code> integer C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_float(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_float_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Floating point number</p>
- <ul>
- <li>
- <code class="arg">t</code> floating point number
- C type (<code>float</code>, <code>double</code>)
- </li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_string(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- <li><code class="no-bg">ctf_string_nowrite(<span class="arg">n</span>, <span class="arg">e</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Null-terminated string; undefined behavior if
- <code class="arg">e</code> is <code>NULL</code>
- </p>
- <ul>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_array(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_array_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>Statically-sized array of integers</p>
- <ul>
- <li><code class="arg">t</code> array element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">s</code> number of elements</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_array_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- <li><code class="no-bg">ctf_array_nowrite_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">s</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Statically-sized array, printed as text; no need to be
- null-terminated
- </p>
- <ul>
- <li><code class="arg">t</code> array element C type (always <code>char</code>)</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">s</code> number of elements</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_sequence_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array of integers; type of
- <code class="arg">E</code> needs to be unsigned
- </p>
- <ul>
- <li><code class="arg">t</code> sequence element C type</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <ul>
- <li><code class="no-bg">ctf_sequence_text(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- <li><code class="no-bg">ctf_sequence_text_nowrite(<span class="arg">t</span>, <span class="arg">n</span>, <span class="arg">e</span>, <span class="arg">T</span>, <span class="arg">E</span>)</code></li>
- </ul>
- </td>
- <td>
- <p>
- Dynamically-sized array, displayed as text; no need to
- be null-terminated; undefined behavior if
- <code class="arg">e</code> is <code>NULL</code></p>
- <ul>
- <li><code class="arg">t</code> sequence element C type (always <code>char</code>)</li>
- <li><code class="arg">n</code> field name</li>
- <li><code class="arg">e</code> argument expression</li>
- <li><code class="arg">T</code> length expression C type</li>
- <li><code class="arg">E</code> length expression</li>
- </ul>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-The `_nowrite` versions omit themselves from the session trace, but are
-otherwise identical. This means the `_nowrite` fields won't be written
-in the recorded trace. Their primary purpose is to make some
-of the event context available to the
-[event filters](#doc-enabling-disabling-events) without having to
-commit the data to sub-buffers.
+++ /dev/null
----
-id: liblttng-ust-tracepoint-loglevel
----
-
-The following table shows the available log level values for the
-`TRACEPOINT_LOGLEVEL()` macro:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th>Enum label</th>
- <th>Enum value</th>
- <th>Description</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td><code class="no-bg">TRACE_EMERG</code></td>
- <td>0</td>
- <td>System is unusable</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_ALERT</code></td>
- <td>1</td>
- <td>Action must be taken immediately</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_CRIT</code></td>
- <td>2</td>
- <td>Critical conditions</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_ERR</code></td>
- <td>3</td>
- <td>Error conditions</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_WARNING</code></td>
- <td>4</td>
- <td>Warning conditions</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_NOTICE</code></td>
- <td>5</td>
- <td>Normal, but significant, condition</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_INFO</code></td>
- <td>6</td>
- <td>Informational message</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_SYSTEM</code></td>
- <td>7</td>
- <td>Debug information with system-level scope (set of programs)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_PROGRAM</code></td>
- <td>8</td>
- <td>Debug information with program-level scope (set of processes)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_PROCESS</code></td>
- <td>9</td>
- <td>Debug information with process-level scope (set of modules)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_MODULE</code></td>
- <td>10</td>
- <td>Debug information with module (executable/library) scope (set of units)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_UNIT</code></td>
- <td>11</td>
- <td>Debug information with compilation unit scope (set of functions)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_FUNCTION</code></td>
- <td>12</td>
- <td>Debug information with function-level scope</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG_LINE</code></td>
- <td>13</td>
- <td>Debug information with line-level scope (<code>TRACEPOINT_EVENT</code> default)</td>
- </tr>
- <tr>
- <td><code class="no-bg">TRACE_DEBUG</code></td>
- <td>14</td>
- <td>Debug-level message</td>
- </tr>
- </tbody>
-</table>
-</div>
-
-Higher log level numbers imply the most verbosity (expect higher tracing
-throughput). Log levels 0 through 6 and log level 14 match
-<a href="http://man7.org/linux/man-pages/man3/syslog.3.html" class="ext">syslog</a>
-level semantics. Log levels 7 through 13 offer more fine-grained
-selection of debug information.
+++ /dev/null
----
-id: online-lttng-manpages
----
-
-LTTng packages currently install the following [man pages](/man), available
-online using the links below:
-
- * **LTTng-tools**
- * <a href="/man/1/lttng/v2.7" class="ext">`lttng`</a>
- * <a href="/man/1/lttng-crash/v2.7" class="ext">`lttng-crash`</a>
- * <a href="/man/8/lttng-sessiond/v2.7" class="ext">`lttng-sessiond`</a>
- * <a href="/man/8/lttng-relayd/v2.7" class="ext">`lttng-relayd`</a>
- * **LTTng-UST**
- * <a href="/man/1/lttng-gen-tp/v2.7" class="ext">`lttng-gen-tp`</a>
- * <a href="/man/3/lttng-ust/v2.7" class="ext">`lttng-ust`</a>
- * <a href="/man/3/lttng-ust-cyg-profile/v2.7" class="ext">`lttng-ust-cyg-profile`</a>
- * <a href="/man/3/lttng-ust-dl/v2.7" class="ext">`lttng-ust-dl`</a>
+++ /dev/null
----
-id: channel-buffering-schemes
----
-
-In the user space tracing domain, two **buffering schemes** are
-available when creating a channel:
-
- * **Per-PID buffering**: keep one ring buffer per process.
- * **Per-UID buffering**: keep one ring buffer for all processes of
- a single user.
-
-The per-PID buffering scheme consumes more memory than the per-UID
-option if more than one process is instrumented for LTTng-UST. However,
-per-PID buffering ensures that one process having a high event
-throughput won't fill all the shared sub-buffers, only its own.
-
-The Linux kernel tracing domain only has one available buffering scheme
-which is to use a single ring buffer for the whole system.
+++ /dev/null
----
-id: channel-overwrite-mode-vs-discard-mode
----
-
-As previously mentioned, a channel's ring buffer is divided into many
-equally sized sub-buffers.
-
-As events occur, they are serialized as trace data into a specific
-sub-buffer (yellow arc in the following animation) until it is full:
-when this happens, the sub-buffer is marked as consumable (red) and
-another, _empty_ (white) sub-buffer starts receiving the following
-events. The marked sub-buffer is eventually consumed by a consumer
-daemon (returns to white).
-
-<script type="text/javascript">
- document.write('<div class="anim img img-50" id="docsvg-channel-subbuf-anim"></div>');
-
- $(document).ready(function() {
- var doc = SVG('docsvg-channel-subbuf-anim');
-
- doc.viewbox(0, 0, 2, 2);
-
- var stdRb = rbBuildStdAnimated(doc, {
- div: 5,
- oR: 0.97,
- evDur: 300,
- evPerSubBuf: 6,
- consumerAfter: 10
- });
-
- stdRb.rb.getGroup().move(1, 1);
- rbSetParentPlayIcon(doc, function() {
- rbStdStart(stdRb);
- });
- });
-</script>
-
-<noscript>
- <div class="err">
- <p>
- <span class="t">Oops!</span>JavaScript must be enabled in
- order to view animations.
- </p>
- </div>
-</noscript>
-
-In an ideal world, sub-buffers are consumed faster than filled, like it
-is the case above. In the real world, however, all sub-buffers could be
-full at some point, leaving no space to record the following events. By
-design, LTTng is a _non-blocking_ tracer: when no empty sub-buffer
-exists, losing events is acceptable when the alternative would be to
-cause substantial delays in the instrumented application's execution.
-LTTng privileges performance over integrity, aiming at perturbing the
-traced system as little as possible in order to make tracing of subtle
-race conditions and rare interrupt cascades possible.
-
-When it comes to losing events because no empty sub-buffer is available,
-the channel's _event loss mode_ determines what to do amongst:
-
- * **Discard**: drop the newest events until a sub-buffer is released.
- * **Overwrite**: clear the sub-buffer containing the oldest recorded
- events and start recording the newest events there. This mode is
- sometimes called _flight recorder mode_ because it behaves like a
- flight recorder: always keep a fixed amount of the latest data.
-
-Which mechanism you should choose depends on your context: prioritize
-the newest or the oldest events in the ring buffer?
-
-Beware that, in overwrite mode, a whole sub-buffer is abandoned as soon
-as a new event doesn't find an empty sub-buffer, whereas in discard
-mode, only the event that doesn't fit is discarded.
-
-Also note that a count of lost events is incremented and saved in
-the trace itself when an event is lost in discard mode, whereas no
-information is kept when a sub-buffer gets overwritten before being
-committed.
-
-There are known ways to decrease your probability of losing events. The
-next section shows how tuning the sub-buffers count and size can be
-used to virtually stop losing events.
+++ /dev/null
----
-id: channel-subbuf-size-vs-subbuf-count
----
-
-For each channel, an LTTng user may set its number of sub-buffers and
-their size.
-
-Note that there is a noticeable tracer's CPU overhead introduced when
-switching sub-buffers (marking a full one as consumable and switching
-to an empty one for the following events to be recorded). Knowing this,
-the following list presents a few practical situations along with how
-to configure sub-buffers for them:
-
- * **High event throughput**: in general, prefer bigger sub-buffers to
- lower the risk of losing events. Having bigger sub-buffers
- also ensures a lower sub-buffer switching frequency. The number of
- sub-buffers is only meaningful if the channel is enabled in
- overwrite mode: in this case, if a sub-buffer overwrite happens, the
- other sub-buffers are left unaltered.
- * **Low event throughput**: in general, prefer smaller sub-buffers
- since the risk of losing events is already low. Since events
- happen less frequently, the sub-buffer switching frequency should
- remain low and thus the tracer's overhead should not be a problem.
- * **Low memory system**: if your target system has a low memory
- limit, prefer fewer first, then smaller sub-buffers. Even if the
- system is limited in memory, you want to keep the sub-buffers as
- big as possible to avoid a high sub-buffer switching frequency.
-
-You should know that LTTng uses CTF as its trace format, which means
-event data is very compact. For example, the average LTTng Linux kernel
-event weights about 32 bytes. A sub-buffer size of 1 MiB is
-thus considered big.
-
-The previous situations highlight the major trade-off between a few big
-sub-buffers and more, smaller sub-buffers: sub-buffer switching
-frequency vs. how much data is lost in overwrite mode. Assuming a
-constant event throughput and using the overwrite mode, the two
-following configurations have the same ring buffer total size:
-
-<script type="text/javascript">
- document.write('<div class="anim img img-100" id="docsvg-channel-subbuf-size-vs-count-anim"></div>');
-
- $(document).ready(function() {
- var doc = SVG('docsvg-channel-subbuf-size-vs-count-anim');
-
- doc.viewbox(0, 0, 4.25, 2);
-
- var stdRb2 = rbBuildStdAnimated(doc, {
- div: 2,
- oR: 0.97,
- evDur: 300,
- evPerSubBuf: 17,
- consumerAfter: 25
- });
- var stdRb16 = rbBuildStdAnimated(doc, {
- div: 8,
- oR: 0.97,
- evDur: 300,
- evPerSubBuf: 4,
- consumerAfter: 6
- });
-
- stdRb2.rb.getGroup().move(1, 1);
- stdRb16.rb.getGroup().move(3.25, 1);
- rbSetParentPlayIcon(doc, function() {
- rbStdStart(stdRb2);
- rbStdStart(stdRb16);
- });
- });
-</script>
-
-<noscript>
- <div class="err">
- <p>
- <span class="t">Oops!</span>JavaScript must be enabled in
- order to view animations.
- </p>
- </div>
-</noscript>
-
- * **2 sub-buffers of 4 MiB each** lead to a very low sub-buffer
- switching frequency, but if a sub-buffer overwrite happens, half of
- the recorded events so far (4 MiB) are definitely lost.
- * **8 sub-buffers of 1 MiB each** lead to 4 times the tracer's
- overhead as the previous configuration, but if a sub-buffer
- overwrite happens, only the eighth of events recorded so far are
- definitely lost.
-
-In discard mode, the sub-buffers count parameter is pointless: use two
-sub-buffers and set their size according to the requirements of your
-situation.
+++ /dev/null
----
-id: channel-switch-timer
----
-
-The _switch timer_ period is another important configurable feature of
-channels to ensure periodic sub-buffer flushing.
-
-When the _switch timer_ fires, a sub-buffer switch happens. This timer
-may be used to ensure that event data is consumed and committed to
-trace files periodically in case of a low event throughput:
-
-<script type="text/javascript">
- document.write('<div class="anim img img-50" id="docsvg-channel-switch-timer"></div>');
-
- $(document).ready(function() {
- var doc = SVG('docsvg-channel-switch-timer');
-
- doc.viewbox(0, 0, 2, 2);
-
- var div = 4;
- var evDur = 1000;
- var rb = rbBuildStd(doc, div, 0.97);
- var switchText = doc.text('Switch!');
-
- switchText.font({
- 'size': 0.1,
- 'weight': 'bold'
- });
- switchText.center(1, 1);
- switchText.attr({
- 'opacity': 0,
- 'fill': '#b02b2c'
- });
-
- var curSubBuf = 0;
- var totalEvents = 0;
- var onEventAdded = function() {
- totalEvents++;
-
- var curSubBufEvCount = rb.getSubBufEvCount(curSubBuf % div);
-
- if (totalEvents >= 4) {
- // switch timer fires
- switchText.attr({
- 'opacity': 1
- });
- switchText.animate(500, '<>', 1000).attr({
- 'opacity': 0
- });
- rb.markSubBuf(curSubBuf % div, 'full');
-
- var lastFullSubBuf = curSubBuf;
-
- setTimeout(function() {
- rb.consumeSubBuf(lastFullSubBuf % div);
- }, 3000);
- totalEvents = 0;
- curSubBuf++;
- rb.markSubBuf(curSubBuf % div, 'cur');
- }
-
- rb.addEvent(curSubBuf % div, evDur, onEventAdded);
- };
-
- rb.markSubBuf(0, 'cur');
- rb.getGroup().move(1, 1);
- rbSetParentPlayIcon(doc, function() {
- rb.addEvent(0, evDur, onEventAdded);
- });
- });
-</script>
-
-<noscript>
- <div class="err">
- <p>
- <span class="t">Oops!</span>JavaScript must be enabled in
- order to view animations.
- </p>
- </div>
-</noscript>
-
-It's also convenient when big sub-buffers are used to cope with
-sporadic high event throughput, even if the throughput is normally
-lower.
+++ /dev/null
----
-id: channel
----
-
-A _channel_ is a set of events with specific parameters and potential
-added context information. Channels have unique names per domain within
-a tracing session. A given event is always registered to at least one
-channel; having the same enabled event in two channels makes
-this event being recorded twice everytime it occurs.
-
-Channels may be individually enabled or disabled. Occurring events of
-a disabled channel never make it to recorded events.
-
-The fundamental role of a channel is to keep a shared ring buffer, where
-events are eventually recorded by the tracer and consumed by a consumer
-daemon. This internal ring buffer is divided into many sub-buffers of
-equal size.
-
-Channels, when created, may be fine-tuned thanks to a few parameters,
-many of them related to sub-buffers. The following subsections explain
-what those parameters are and in which situations you should manually
-adjust them.
+++ /dev/null
----
-id: domain
----
-
-A tracing _domain_ is the official term the LTTng project uses to
-designate a tracer category.
-
-There are currently five known domains:
-
- * Linux kernel
- * user space
- * `java.util.logging` (JUL)
- * log4j
- * Python (2 and 3 are supported)
-
-Different tracers expose common features in their own interfaces, but,
-from a user's perspective, you still need to target a specific type of
-tracer to perform some actions. For example, since both kernel and user
-space tracers support named tracepoints (probes manually inserted in
-source code), you need to specify which one is concerned when enabling
-an event because both domains could have existing events with the same
-name.
-
-Some features are reserved for some domains. Dynamic function entry/return
-instrumentation, for example, is currently only supported in the kernel
-domain, but support for other domains could be added in the future.
+++ /dev/null
----
-id: event
----
-
-An _event_, in LTTng's realm, is a term often used metonymically,
-having multiple definitions depending on the context:
-
- 1. When tracing, an event is a _point in space-time_. Space, in a
- tracing context, is the set of all executable positions of a
- compiled application by a logical processor. When a program is
- executed by a processor and some instrumentation point, or
- _probe_, is encountered, an event occurs. This event is accompanied
- by some contextual payload (values of specific variables at this
- point of execution) which may or may not be recorded.
- 2. In the context of a recorded trace file, the term _event_ implies
- a _recorded event_.
- 3. When configuring a tracing session, _enabled events_ refer to
- specific rules which could lead to the transfer of actual
- occurring events (1) to recorded events (2).
-
-The whole [Core concepts](#doc-core-concepts) section focuses on the
-third definition. An event is always registered to _one or more_
-channels and may be enabled or disabled at will per channel. A disabled
-event never leads to a recorded event, even if its channel is enabled.
-
-An event (3) is enabled with a few conditions that must _all_ be met
-when an event (1) happens in order to generate a recorded event (2):
-
- 1. A _probe_ or group of probes in the traced application must be
- executed.
- 2. **Optionally**, the probe must have a log level matching a
- log level range specified when enabling the event.
- 3. **Optionally**, the occurring event must satisfy a custom
- expression, or _filter_, specified when enabling the event.
+++ /dev/null
----
-id: core-concepts
----
-
-This section explains the various elementary concepts a user has to deal
-with when using LTTng. They are:
-
- * [tracing session](#doc-tracing-session)
- * [domain](#doc-domain)
- * [channel](#doc-channel)
- * [event](#doc-event)
-
+++ /dev/null
----
-id: tracing-session
----
-
-A _tracing session_ is—like any session—a container of
-state. Anything that is done when tracing using LTTng happens in the
-scope of a tracing session. In this regard, it is analogous to a bank
-website's session: you can't interact online with your bank account
-unless you are logged in a session, except for reading a few static
-webpages (LTTng, too, can report some static information that does not
-need a created tracing session).
-
-A tracing session holds the following attributes and objects (some of
-which are described in the following sections):
-
- * a name
- * the tracing state (tracing started or stopped)
- * the trace data output path/URL (local path or sent over the network)
- * a mode (normal, snapshot or live)
- * the snapshot output paths/URLs (if applicable)
- * for each [domain](#doc-domain), a list of [channels](#doc-channel)
- * for each channel:
- * a name
- * the channel state (enabled or disabled)
- * its parameters (event loss mode, sub-buffers size and count,
- timer periods, output type, trace files size and count, and the rest)
- * a list of added context information
- * a list of [events](#doc-event)
- * for each event:
- * its state (enabled or disabled)
- * a list of instrumentation points (tracepoints, system calls,
- dynamic probes, other types of probes)
- * associated log levels
- * a filter expression
-
-All this information is completely isolated between tracing sessions.
-As you can see in the list above, even the tracing state
-is a per-tracing session attribute, so that you may trace your target
-system/application in a given tracing session with a specific
-configuration while another one stays inactive.
-
-<figure class="img img-100">
-<img src="/images/docs26/concepts.png" alt="LTTng concepts">
-<figcaption>
- A <em>tracing session</em> is a container of domains,
- channels, and events.
-</figcaption>
-</figure>
-
-Conceptually, a tracing session is a per-user object; the
-[Plumbing](#doc-plumbing) section shows how this is actually
-implemented. Any user may create as many concurrent tracing sessions
-as desired.
-
-<figure class="img img-100">
-<img src="/images/docs26/many-sessions.png" alt="Tracing sessions per user">
-<figcaption>
- Each user may create as many tracing sessions as desired.
-</figcaption>
-</figure>
-
-The trace data generated in a tracing session may be either saved
-to disk, sent over the network or not saved at all (in which case
-snapshots may still be saved to disk or sent to a remote machine).
+++ /dev/null
----
-id: understanding-lttng
----
-
-If you're going to use LTTng in any serious way, it is fundamental that
-you become familiar with its core concepts. Technical terms like
-_tracing sessions_, _domains_, _channels_ and _events_ are used over
-and over in the [Using LTTng](#doc-using-lttng) chapter,
-and it is assumed that you understand what they mean when reading it.
-
-LTTng, as you already know, is a _toolkit_. It would be wrong
-to call it a simple _tool_ since it is composed of multiple interacting
-components. This chapter also describes the latter, providing details
-about their respective roles and how they connect together to form
-the current LTTng ecosystem.
+++ /dev/null
----
-id: plumbing
----
-
-The previous section described the concepts at the heart of LTTng.
-This section summarizes LTTng's implementation: how those objects are
-managed by different applications and libraries working together to
-form the toolkit.
+++ /dev/null
----
-id: liblttng-ctl-lttng
----
-
-The LTTng control library, `liblttng-ctl`, can be used to communicate
-with the session daemon using a C API that hides the underlying
-protocol's details. `liblttng-ctl` is part of LTTng-tools.
-
-`liblttng-ctl` may be used by including its "master" header:
-
-~~~ c
-#include <lttng/lttng.h>
-~~~
-
-Some objects are referred by name (C string), such as tracing sessions,
-but most of them require creating a handle first using
-`lttng_create_handle()`. The best available developer documentation for
-`liblttng-ctl` is, for the moment, its installed header files as such.
-Every function/structure is thoroughly documented.
-
-The `lttng` program is the _de facto_ standard user interface to
-control LTTng tracing sessions. `lttng` uses `liblttng-ctl` to
-communicate with session daemons behind the scenes.
-<a href="/man/1/lttng/v2.7" class="ext">Its man page</a> is exhaustive, as
-well as its command line help (<code>lttng <em>cmd</em> --help</code>,
-where <code><em>cmd</em></code> is the command name).
-
-The [Controlling tracing](#doc-controlling-tracing) section is a feature
-tour of the `lttng` tool.
+++ /dev/null
----
-id: lttng-consumerd
----
-
-The _consumer daemon_, or `lttng-consumerd`, is a program sharing some
-ring buffers with user applications or the LTTng kernel modules to
-collect trace data and output it at some place (on disk or sent over
-the network to an LTTng relay daemon).
-
-Consumer daemons are created by a session daemon as soon as events are
-enabled within a tracing session, well before tracing is activated
-for the latter. Entirely managed by session daemons,
-consumer daemons survive session destruction to be reused later,
-should a new tracing session be created. Consumer daemons are always
-owned by the same user as their session daemon. When its owner session
-daemon is killed, the consumer daemon also exits. This is because
-the consumer daemon is always the child process of a session daemon.
-Consumer daemons should never be started manually. For this reason,
-they are not installed in one of the usual locations listed in the
-`PATH` environment variable. `lttng-sessiond` has, however, a
-<a href="/man/8/lttng-sessiond/v2.7" class="ext">bunch of options</a> to
-specify custom consumer daemon paths if, for some reason, a consumer
-daemon other than the default installed one is needed.
-
-There are up to two running consumer daemons per user, whereas only one
-session daemon may run per user. This is because each process has
-independent bitness: if the target system runs a mixture of 32-bit and
-64-bit processes, it is more efficient to have separate corresponding
-32-bit and 64-bit consumer daemons. The `root` user is an exception: it
-may have up to _three_ running consumer daemons: 32-bit and 64-bit
-instances for its user space applications and one more reserved for
-collecting kernel trace data.
-
-As new tracing domains are added to LTTng, the development community's
-intent is to minimize the need for additionnal consumer daemon instances
-dedicated to them. For instance, the `java.util.logging` (JUL) domain
-events are in fact mapped to the user space domain, thus tracing this
-particular domain is handled by existing user space domain consumer
-daemons.
+++ /dev/null
----
-id: lttng-modules
----
-
-The LTTng Linux kernel modules provide everything needed to trace the
-Linux kernel: various probes, a ring buffer implementation for a
-consumer daemon to read trace data and the tracer itself.
-
-Only in exceptional circumstances should you ever need to load the
-LTTng kernel modules manually: it is normally the responsability of
-`root`'s session daemon to do so. Even if you were to develop your
-own LTTng probe module—for tracing a custom kernel or some kernel
-module (this topic is covered in the
-[Linux kernel](#doc-instrumenting-linux-kernel) instrumenting guide of
-the [Using LTTng](#doc-using-lttng) chapter)—you
-should use the `--extra-kmod-probes` option of the session daemon to
-append your probe to the default list. The session and consumer daemons
-of regular users do not interact with the LTTng kernel modules at all.
-
-LTTng kernel modules are installed, by default, in
-<code>/usr/lib/modules/<em>release</em>/extra</code>, where
-<code><em>release</em></code> is the kernel release
-(see `uname --kernel-release`).
+++ /dev/null
----
-id: lttng-relayd
----
-
-When a tracing session is configured to send its trace data over the
-network, an LTTng _relay daemon_ must be used at the other end to
-receive trace packets and serialize them to trace files. This setup
-makes it possible to trace a target system without ever committing trace
-data to its local storage, a feature which is useful for embedded
-systems, amongst others. The command implementing the relay daemon
-is `lttng-relayd`.
-
-The basic use case of `lttng-relayd` is to transfer trace data received
-over the network to trace files on the local file system. The relay
-daemon must listen on two TCP ports to achieve this: one control port,
-used by the target session daemon, and one data port, used by the
-target consumer daemon. The relay and session daemons agree on common
-default ports when custom ones are not specified.
-
-Since the communication transport protocol for both ports is standard
-TCP, the relay daemon may be started either remotely or locally (on the
-target system).
-
-While two instances of consumer daemons (32-bit and 64-bit) may run
-concurrently for a given user, `lttng-relayd` needs only be of its
-host operating system's bitness.
-
-The other important feature of LTTng's relay daemon is the support of
-_LTTng live_. LTTng live is an application protocol to view events as
-they arrive. The relay daemon still records events in trace files,
-but a _tee_ allows to inspect incoming events.
-
-<figure class="img img-100">
-<img src="/images/docs26/lttng-live.png" alt="LTTng live">
-<figcaption>
- The relay daemon creates a <em>tee</em>, forwarding the trace data
- to both trace files and a live viewer.
-</figcaption>
-</figure>
-
-Using LTTng live
-locally thus requires to run a local relay daemon.
+++ /dev/null
----
-id: lttng-sessiond
----
-
-At the heart of LTTng's plumbing is the _session daemon_, often called
-by its command name, `lttng-sessiond`.
-
-The session daemon is responsible for managing tracing sessions and
-what they logically contain (channel properties, enabled/disabled
-events, and the rest). By communicating locally with instrumented
-applications (using LTTng-UST) and with the LTTng Linux kernel modules
-(LTTng-modules), it oversees all tracing activities.
-
-One of the many things that `lttng-sessiond` does is to keep
-track of the available event types. User space applications and
-libraries actively connect and register to the session daemon when they
-start. By contrast, `lttng-sessiond` seeks out and loads the appropriate
-LTTng kernel modules as part of its own initialization. Kernel event
-types are _pulled_ by `lttng-sessiond`, whereas user space event types
-are _pushed_ to it by the various user space tracepoint providers.
-
-Using a specific inter-process communication protocol with Linux kernel
-and user space tracers, the session daemon can send channel information
-so that they are initialized, enable/disable specific probes based on
-enabled/disabled events by the user, send event filters information to
-LTTng tracers so that filtering actually happens at the tracer site,
-start/stop tracing a specific application or the Linux kernel, and more.
-
-The session daemon is not useful without some user controlling it,
-because it's only a sophisticated control interchange and thus
-doesn't make any decision on its own. `lttng-sessiond` opens a local
-socket for controlling it, albeit the preferred way to control it is
-using `liblttng-ctl`, an installed C library hiding the communication
-protocol behind an easy-to-use API. The `lttng` tool makes use of
-`liblttng-ctl` to implement a user-friendly command line interface.
-
-`lttng-sessiond` does not receive any trace data from instrumented
-applications; the _consumer daemons_ are the programs responsible for
-collecting trace data using shared ring buffers. However, the session
-daemon is the one that must spawn a consumer daemon and establish
-a control communication with it.
-
-Session daemons run on a per-user basis. Knowing this, multiple
-instances of `lttng-sessiond` may run simultaneously, each belonging
-to a different user and each operating independently of the others.
-Only `root`'s session daemon, however, may control LTTng kernel modules
-(that is, the kernel tracer). With that in mind, if a user has no root
-access on the target system, he cannot trace the system's kernel, but
-should still be able to trace its own instrumented applications.
-
-It has to be noted that, although only `root`'s session daemon may
-control the kernel tracer, the `lttng-sessiond` command has a `--group`
-option which may be used to specify the name of a special user group
-allowed to communicate with `root`'s session daemon and thus record
-kernel traces. By default, this group is named `tracing`.
-
-If not done yet, the `lttng` tool, by default, automatically starts a
-session daemon. `lttng-sessiond` may also be started manually:
-
-<pre class="term">
-lttng-sessiond
-</pre>
-
-This starts the session daemon in foreground. Use
-
-<pre class="term">
-lttng-sessiond --daemonize
-</pre>
-
-to start it as a true daemon.
-
-To kill the current user's session daemon, `pkill` may be used:
-
-<pre class="term">
-pkill lttng-sessiond
-</pre>
-
-The default `SIGTERM` signal terminates it cleanly.
-
-Several other options are available and described in
-<a href="/man/8/lttng-sessiond/v2.7" class="ext"><code>lttng-sessiond</code>'s
-man page</a> or by running `lttng-sessiond --help`.
+++ /dev/null
----
-id: lttng-ust
----
-
-The user space tracing part of LTTng is possible thanks to the user
-space tracing library, `liblttng-ust`, which is part of the LTTng-UST
-package.
-
-`liblttng-ust` provides header files containing macros used to define
-tracepoints and create tracepoint providers, as well as a shared object
-that must be linked to individual applications to connect to and
-communicate with a session daemon and a consumer daemon as soon as the
-application starts.
-
-The exact mechanism by which an application is registered to the
-session daemon is beyond the scope of this documentation. The only thing
-you need to know is that, since the library constructor does this job
-automatically, tracepoints may be safely inserted anywhere in the source
-code without prior manual initialization of `liblttng-ust`.
-
-The `liblttng-ust`-session daemon collaboration also provides an
-interesting feature: user space events may be enabled _before_
-applications actually start. By doing this and starting tracing before
-launching the instrumented application, you make sure that even the
-earliest occurring events can be recorded.
-
-The [C application](#doc-c-application) instrumenting guide of the
-[Using LTTng](#doc-using-lttng) chapter focuses on using `liblttng-ust`:
-instrumenting, building/linking and running a user application.
+++ /dev/null
----
-id: plumbing-overview
----
-
-As [mentioned previously](#doc-installing-lttng), the whole LTTng suite
-is made of the LTTng-tools, LTTng-UST, and
-LTTng-modules packages. Together, they provide different daemons, libraries,
-kernel modules and command line interfaces. The following tree shows
-which usable component belongs to which package:
-
- * **LTTng-tools**:
- * session daemon (`lttng-sessiond`)
- * consumer daemon (`lttng-consumerd`)
- * relay daemon (`lttng-relayd`)
- * tracing control library (`liblttng-ctl`)
- * tracing control command line tool (`lttng`)
- * **LTTng-UST**:
- * user space tracing library (`liblttng-ust`) and its headers
- * preloadable user space tracing helpers
- (`liblttng-ust-libc-wrapper`, `liblttng-ust-pthread-wrapper`,
- `liblttng-ust-cyg-profile`, `liblttng-ust-cyg-profile-fast`
- and `liblttng-ust-dl`)
- * user space tracepoint code generator command line tool
- (`lttng-gen-tp`)
- * `java.util.logging`/log4j tracepoint providers
- (`liblttng-ust-jul-jni` and `liblttng-ust-log4j-jni`) and JAR
- file (`liblttng-ust-agent.jar`)
- * **LTTng-modules**:
- * LTTng Linux kernel tracer module
- * tracing ring buffer kernel modules
- * many LTTng probe kernel modules
-
-The following diagram shows how the most important LTTng components
-interact. Plain purple arrows represent trace data paths while dashed
-red arrows indicate control communications. The LTTng relay daemon is
-shown running on a remote system, although it could as well run on the
-target (monitored) system.
-
-<figure class="img img-100">
-<img src="/images/docs27/plumbing-27.png" alt="LTTng plumbing">
-<figcaption>
- Control and data paths between LTTng components
-</figcaption>
-</figure>
-
-Each component is described in the following subsections.
+++ /dev/null
----
-id: adding-context
----
-
-If you read all the sections of
-[Controlling tracing](#doc-controlling-tracing) so far, you should be
-able to create tracing sessions, create and enable channels and events
-within them and start/stop the LTTng tracers. Event fields recorded in
-trace files provide important information about occurring events, but
-sometimes external context may help you solve a problem faster. This
-section discusses how to add context information to events of a
-specific channel using the `lttng` tool.
-
-There are various available context values which can accompany events
-recorded by LTTng, for example:
-
- * **process information**:
- * identifier (PID)
- * name
- * priority
- * scheduling priority (niceness)
- * thread identifier (TID)
- * the **hostname** of the system on which the event occurred
- * plenty of **performance counters** using perf, for example:
- * CPU cycles, stalled cycles, idle cycles, and the other cycle types
- * cache misses
- * branch instructions, misses, loads
- * CPU faults
-
-The full list is available in the output of `lttng add-context --help`.
-Some of them are reserved for a specific domain (kernel or
-user space) while others are available for both.
-
-To add context information to one or all channels of a given tracing
-session, use the `add-context` command:
-
-<pre class="term">
-lttng add-context --userspace --type vpid --type perf:thread:cpu-cycles
-</pre>
-
-The above example adds the virtual process identifier and per-thread
-CPU cycles count values to all recorded user space domain events of the
-current tracing session. Use the `--channel` option to select a specific
-channel:
-
-<pre class="term">
-lttng add-context --kernel --channel my-channel --type tid
-</pre>
-
-adds the thread identifier value to all recorded kernel domain events
-in the channel `my-channel` of the current tracing session.
-
-Beware that context information cannot be removed from channels once
-it's added for a given tracing session.
+++ /dev/null
----
-id: basic-tracing-session-control
----
-
-Once you have
-[created a tracing session](#doc-creating-destroying-tracing-sessions)
-and [enabled one or more events](#doc-enabling-disabling-events),
-you may activate the LTTng tracers for the current tracing session at
-any time:
-
-<pre class="term">
-lttng start
-</pre>
-
-Subsequently, you may stop the tracers:
-
-<pre class="term">
-lttng stop
-</pre>
-
-LTTng is very flexible: user space applications may be launched before
-or after the tracers are started. Events are only recorded if they
-are properly enabled and if they occur while tracers are active.
-
-A tracing session name may be passed to both the `start` and `stop`
-commands to start/stop tracing a session other than the current one.
+++ /dev/null
----
-id: creating-destroying-tracing-sessions
----
-
-Whatever you want to do with `lttng`, it has to happen inside a
-**tracing session**, created beforehand. A session, in general, is a
-per-user container of state. A tracing session is no different; it
-keeps a specific state of stuff like:
-
- * session name
- * enabled/disabled channels with associated parameters
- * enabled/disabled events with associated log levels and filters
- * context information added to channels
- * tracing activity (started or stopped)
-
-and more.
-
-A single user may have many active tracing sessions. LTTng session
-daemons are the ultimate owners and managers of tracing sessions. For
-user space tracing, each user has its own session daemon. Since Linux
-kernel tracing requires root privileges, only `root`'s session daemon
-may enable and trace kernel events. However, `lttng` has a `--group`
-option (which is passed to `lttng-sessiond` when starting it) to
-specify the name of a _tracing group_ which selected users may be part
-of to be allowed to communicate with `root`'s session daemon. By
-default, the tracing group name is `tracing`.
-
-To create a tracing session, do:
-
-<pre class="term">
-lttng create my-session
-</pre>
-
-This creates a new tracing session named `my-session` and make it
-the current one. If you don't specify a name (running only
-`lttng create`), your tracing session is named `auto` followed by the
-current date and time. Traces
-are written in <code>~/lttng-traces/<em>session</em>-</code> followed
-by the tracing session's creation date/time by default, where
-<code><em>session</em></code> is the tracing session name. To save them
-at a different location, use the `--output` option:
-
-<pre class="term">
-lttng create --output /tmp/some-directory my-session
-</pre>
-
-You may create as many tracing sessions as you wish:
-
-<pre class="term">
-lttng create other-session
-lttng create yet-another-session
-</pre>
-
-You may view all existing tracing sessions using the `list` command:
-
-<pre class="term">
-lttng list
-</pre>
-
-The state of a _current tracing session_ is kept in `~/.lttngrc`. Each
-invocation of `lttng` reads this file to set its current tracing
-session name so that you don't have to specify a session name for each
-command. You could edit this file manually, but the preferred way to
-set the current tracing session is to use the `set-session` command:
-
-<pre class="term">
-lttng set-session other-session
-</pre>
-
-Most `lttng` commands accept a `--session` option to specify the name
-of the target tracing session.
-
-Any existing tracing session may be destroyed using the `destroy`
-command:
-
-<pre class="term">
-lttng destroy my-session
-</pre>
-
-Providing no argument to `lttng destroy` destroys the current
-tracing session. Destroying a tracing session stops any tracing
-running within the latter. Destroying a tracing session frees resources
-acquired by the session daemon and tracer side, making sure to flush
-all trace data.
-
-You can't do much with LTTng using only the `create`, `set-session`
-and `destroy` commands of `lttng`, but it is essential to know them in
-order to control LTTng tracing, which always happen within the scope of
-a tracing session.
+++ /dev/null
----
-id: fine-tuning-channels
----
-
-There are various parameters that may be fine-tuned with the
-`enable-channel` command. The latter are well documented in
-<a href="/man/1/lttng/v2.7" class="ext">the man page of `lttng`</a>
-and in the [Channel](#doc-channel) section of the
-[Understanding LTTng](#doc-understanding-lttng) chapter. For basic
-tracing needs, their default values should be just fine, but here are a
-few examples to break the ice.
-
-As the frequency of recorded events increases—either because the
-event throughput is actually higher or because you enabled more events
-than usual—_event loss_ might be experienced. Since LTTng never
-waits, by design, for sub-buffer space availability (non-blocking
-tracer), when a sub-buffer is full and no empty sub-buffers are left,
-there are two possible outcomes: either the new events that do not fit
-are rejected, or they start replacing the oldest recorded events.
-The choice of which algorithm to use is a per-channel parameter, the
-default being discarding the newest events until there is some space
-left. If your situation always needs the latest events at the expense
-of writing over the oldest ones, create a channel with the `--overwrite`
-option:
-
-<pre class="term">
-lttng enable-channel --kernel --overwrite my-channel
-</pre>
-
-When an event is lost, it means no space was available in any
-sub-buffer to accommodate it. Thus, if you want to cope with sporadic
-high event throughput situations and avoid losing events, you need to
-allocate more room for storing them in memory. This can be done by
-either increasing the size of sub-buffers or by adding sub-buffers.
-The following example creates a user space domain channel with
-16 sub-buffers of 512 kiB each:
-
-<pre class="term">
-lttng enable-channel --userspace --num-subbuf 16 --subbuf-size 512k big-channel
-</pre>
-
-Both values need to be powers of two, otherwise they are rounded up
-to the next one.
-
-Two other interesting available parameters of `enable-channel` are
-`--tracefile-size` and `--tracefile-count`, which respectively limit
-the size of each trace file and the their count for a given channel.
-When the number of written trace files reaches its limit for a given
-channel-CPU pair, the next trace file overwrites the very first
-one. The following example creates a kernel domain channel with a
-maximum of three trace files of 1 MiB each:
-
-<pre class="term">
-lttng enable-channel --kernel --tracefile-size 1M --tracefile-count 3 my-channel
-</pre>
-
-An efficient way to make sure lots of events are generated is enabling
-all kernel events in this channel and starting the tracer:
-
-<pre class="term">
-lttng enable-event --kernel --all --channel my-channel
-lttng start
-</pre>
-
-After a few seconds, look at trace files in your tracing session
-output directory. For two CPUs, it should look like:
-
-~~~ text
-my-channel_0_0 my-channel_1_0
-my-channel_0_1 my-channel_1_1
-my-channel_0_2 my-channel_1_2
-~~~
-
-Amongst the files above, you might see one in each group with a size
-lower than 1 MiB: they are the files currently being written.
-
-Since all those small files are valid LTTng trace files, LTTng trace
-viewers may read them. It is the viewer's responsibility to properly
-merge the streams so as to present an ordered list to the user.
-<a href="http://diamon.org/babeltrace" class="ext">Babeltrace</a>
-merges LTTng trace files correctly and is fast at doing it.
+++ /dev/null
----
-id: enabling-disabling-channels
----
-
-[As mentioned](#doc-event) in the
-[Understanding LTTng](#doc-understanding-lttng) chapter, enabled
-events are contained in a specific channel, itself contained in a
-specific tracing session. A channel is a group of events with
-tunable parameters (event loss mode, sub-buffer size, number of
-sub-buffers, trace file sizes and count, to name a few). A given channel
-may only be responsible for enabled events belonging to one domain:
-either kernel or user space.
-
-If you only used the `create`, `enable-event` and `start`/`stop`
-commands of the `lttng` tool so far, one or two channels were
-automatically created for you (one for the kernel domain and/or one
-for the user space domain). The default channels are both named
-`channel0`; channels from different domains may have the same name.
-
-The current channels of a given tracing session can be viewed with
-
-<pre class="term">
-lttng list some-session
-</pre>
-
-where `some-session` is the name of the desired tracing session.
-
-To create and enable a channel, use the `enable-channel` command:
-
-<pre class="term">
-lttng enable-channel --kernel my-channel
-</pre>
-
-This creates a kernel domain channel named `my-channel` with
-default parameters in the current tracing session.
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>Because of a current limitation, all
- channels must be <em>created</em> prior to beginning tracing in a
- given tracing session, that is before the first time you do
- <code>lttng start</code>.
-</p>
-<p>
- Since a channel is automatically created by
- <code>enable-event</code> only for the specified domain, you cannot,
- for example, enable a kernel domain event, start tracing and then
- enable a user space domain event because no user space channel
- exists yet and it's too late to create one.
-</p>
-<p>
- For this reason, make sure to configure your channels properly
- before starting the tracers for the first time!
-</p>
-</div>
-
-Here's another example:
-
-<pre class="term">
-lttng enable-channel --userspace --session other-session --overwrite \
- --tracefile-size 1048576 1mib-channel
-</pre>
-
-This creates a user space domain channel named `1mib-channel` in
-the tracing session named `other-session` that loses new events by
-overwriting previously recorded events (instead of the default mode of
-discarding newer ones) and saves trace files with a maximum size of
-1 MiB each.
-
-Note that channels may also be created using the `--channel` option of
-the `enable-event` command when the provided channel name doesn't exist
-for the specified domain:
-
-<pre class="term">
-lttng enable-event --kernel --channel some-channel sched_switch
-</pre>
-
-If no kernel domain channel named `some-channel` existed before calling
-the above command, it would be created with default parameters.
-
-You may enable the same event in two different channels:
-
-<pre class="term">
-lttng enable-event --userspace --channel my-channel app:tp
-lttng enable-event --userspace --channel other-channel app:tp
-</pre>
-
-If both channels are enabled, the occurring `app:tp` event
-generates two recorded events, one for each channel.
-
-Disabling a channel is done with the `disable-event` command:
-
-<pre class="term">
-lttng disable-event --kernel some-channel
-</pre>
-
-The state of a channel precedes the individual states of events within
-it: events belonging to a disabled channel, even if they are
-enabled, won't be recorded.
-
+++ /dev/null
----
-id: enabling-disabling-events
----
-
-Inside a tracing session, individual events may be enabled or disabled
-so that tracing them may or may not generate trace data.
-
-We sometimes use the term _event_ metonymically throughout this text to
-refer to a specific condition, or _rule_, that could lead, when
-satisfied, to an actual occurring event (a point at a specific position
-in source code/binary program, logical processor and time capturing
-some payload) being recorded as trace data. This specific condition is
-composed of:
-
- 1. A **domain** (kernel, user space, `java.util.logging`, or log4j)
- (required).
- 2. One or many **instrumentation points** in source code or binary
- program (tracepoint name, address, symbol name, function name,
- logger name, amongst other types of probes) to be executed
- (required).
- 3. A **log level** (each instrumentation point declares its own log
- level) or log level range to match (optional; only valid for user
- space domain).
- 4. A **custom user expression**, or **filter**, that must evaluate to
- _true_ when a tracepoint is executed (optional).
-
-All conditions are specified using arguments passed to the
-`enable-event` command of the `lttng` tool.
-
-Condition 1 is specified using either `--kernel`/`-k` (kernel),
-`--userspace`/`-u` (user space), `--jul`/`-j`
-(<abbr title="java.util.logging">JUL</abbr>), or `--log4j`/`-l` (log4j).
-Exactly one of those four arguments must be specified.
-
-Condition 2 is specified using one of:
-
- * `--tracepoint`: **tracepoint**
- * `--probe`: **dynamic probe** (address, symbol name or combination
- of both in binary program; only valid for kernel domain)
- * `--function`: **function entry/exit** (address, symbol name or
- combination of both in binary program; only valid for kernel domain)
- * `--syscall`: **system call entry/exit** (only valid for kernel
- domain)
-
-When none of the above is specified, `enable-event` defaults to
-using `--tracepoint`.
-
-Condition 3 is specified using one of:
-
- * `--loglevel`: log level range from 0 to a specific log level
- * `--loglevel-only`: specific log level
-
-See `lttng enable-event --help` for the complete list of log level
-names.
-
-Condition 4 is specified using the `--filter` option. This filter is
-a C-like expression, potentially reading real-time values of event
-fields, that has to evaluate to _true_ for the condition to be satisfied.
-Event fields are read using plain identifiers while context fields
-must be prefixed with `$ctx.`. See `lttng enable-event --help` for
-all usage details.
-
-The aforementioned arguments are combined to create and enable events.
-Each unique combination of arguments leads to a different
-_enabled event_. The log level and filter arguments are optional, their
-default values being respectively all log levels and a filter which
-always returns _true_.
-
-Here are a few examples (you must
-[create a tracing session](#doc-creating-destroying-tracing-sessions)
-first):
-
-<pre class="term">
-lttng enable-event -u --tracepoint my_app:hello_world
-lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_WARNING
-lttng enable-event -u --tracepoint 'my_other_app:*'
-lttng enable-event -u --tracepoint my_app:foo_bar \
- --filter 'some_field <= 23 && !other_field'
-lttng enable-event -k --tracepoint sched_switch
-lttng enable-event -k --tracepoint gpio_value
-lttng enable-event -k --function usb_probe_device usb_probe_device
-lttng enable-event -k --syscall --all
-lttng enable-event -k --tracepoint irq_handler_entry \
- --filter 'irq == 28 || irq == 17'
-</pre>
-
-The wildcard symbol, `*`, matches _anything_ and may only be used at
-the end of the string when specifying a _tracepoint_. Make sure to
-use it between single quotes in your favorite shell to avoid
-undesired shell expansion.
-
-System call events can be enabled individually, too:
-
-<pre class="term" style="position: relative">
-<div class="since">Since 2.6</div>lttng enable-event -k --syscall open
-lttng enable-event -k --syscall read
-lttng enable-event -k --syscall fork,chdir,pipe
-</pre>
-
-The complete list of available system call events can be
-obtained using
-
-<pre class="term">
-lttng list --kernel --syscall
-</pre>
-
-You can see a list of events (enabled or disabled) using
-
-<pre class="term">
-lttng list some-session
-</pre>
-
-where `some-session` is the name of the desired tracing session.
-
-What you're actually doing when enabling events with specific conditions
-is creating a **whitelist** of traceable events for a given channel.
-Thus, the following case presents redundancy:
-
-<pre class="term">
-lttng enable-event -u --tracepoint my_app:hello_you
-lttng enable-event -u --tracepoint my_app:hello_you --loglevel TRACE_DEBUG
-</pre>
-
-The second command, matching a log level range, is useless since the first
-command enables all tracepoints matching the same name,
-`my_app:hello_you`.
-
-Disabling an event is simpler: you only need to provide the event
-name to the `disable-event` command:
-
-<pre class="term">
-lttng disable-event --userspace my_app:hello_you
-</pre>
-
-This name has to match a name previously given to `enable-event` (it
-has to be listed in the output of `lttng list some-session`).
-The `*` wildcard is supported, as long as you also used it in a
-previous `enable-event` invocation.
-
-Disabling an event does not add it to some blacklist: it simply removes
-it from its channel's whitelist. This is why you cannot disable an event
-which wasn't previously enabled.
-
-A disabled event doesn't generate any trace data, even if all its
-specified conditions are met.
-
-Events may be enabled and disabled at will, either when LTTng tracers
-are active or not. Events may be enabled before a user space application
-is even started.
+++ /dev/null
----
-id: controlling-tracing
----
-
-Once you're in possession of a software that is properly
-[instrumented](#doc-instrumenting) for LTTng tracing, be it thanks to
-the built-in LTTng probes for the Linux kernel, a custom user
-application, or a custom Linux kernel, all that is left is actually
-tracing it.
-
-You control LTTng tracing using a single command
-line tool: `lttng`. LTTng [session daemons](#doc-lttng-sessiond) may
-either be started manually (`lttng-sessiond`) or automatically by the
-`lttng` command when a sessions is created. Trace data may be forwarded
-to the network and used elsewhere using an LTTng
-[relay daemon](#doc-lttng-relayd) (`lttng-relayd`).
-
-The man pages of `lttng`, `lttng-sessiond`, and `lttng-relayd` are pretty
-complete, thus this section is not an online copy of the latter (we
-leave this contents for the
-[Online LTTng man pages](#doc-online-lttng-manpages) section).
-This section is rather a tour of LTTng
-features through practical examples and tips.
-
-If not already done, make sure you understand the core concepts
-and how LTTng components connect together by reading the
-[Understanding LTTng](#doc-understanding-lttng) chapter; this section
-assumes you are familiar with them.
+++ /dev/null
----
-id: lttng-live
-since: 2.4
----
-
-We have seen how trace files may be produced by LTTng out of generated
-application and Linux kernel events. We have seen that those trace files
-may be either recorded locally by consumer daemons or remotely using
-a relay daemon. And we have seen that the maximum size and count of
-trace files is configurable for each channel. With all those features,
-it's still not possible to read a trace file as it is being written
-because it could be incomplete and appear corrupted to the viewer.
-There is a way to view events as they arrive, however: using
-_LTTng live_.
-
-LTTng live is implemented, in LTTng, solely on the relay daemon side.
-As trace data is sent over the network to a relay daemon by a (possibly
-remote) consumer daemon, a _tee_ is created: trace data is recorded to
-trace files _as well as_ being transmitted to a connected live viewer:
-
-<figure class="img img-100">
-<img src="/images/docs26/lttng-live.png" alt="LTTng live">
-<figcaption>
- The relay daemon creates a <em>tee</em>, forwarding the trace data
- to both trace files and a live viewer.
-</figcaption>
-</figure>
-
-In order to use this feature, a tracing session must created in live
-mode on the target system:
-
-<pre class="term">
-lttng create --live
-</pre>
-
-An optional parameter may be passed to `--live` to set the period
-(in microseconds) between flushes to the network
-(1 second is the default). With:
-
-<pre class="term">
-lttng create --live 100000
-</pre>
-
-the daemons flush their data every 100 ms.
-
-If no network output is specified to the `create` command, a local
-relay daemon is spawned. In this very common case, viewing a live
-trace is easy: enable events and start tracing as usual, then use
-`lttng view` to start the default live viewer:
-
-<pre class="term">
-lttng view
-</pre>
-
-The correct arguments are passed to the live viewer so that it
-may connect to the local relay daemon and start reading live events.
-
-You may also wish to use a live viewer not running on the target
-system. In this case, you should specify a network output when using
-the `create` command (`--set-url` or `--ctrl-url`/`--data-url` options).
-A distant LTTng relay daemon should also be started to receive control
-and trace data. By default, `lttng-relayd` listens on 127.0.0.1:5344
-for an LTTng live connection. Otherwise, the desired URL may be
-specified using its `--live-port` option.
-
-The
-<a href="http://diamon.org/babeltrace" class="ext">`babeltrace`</a>
-viewer supports LTTng live as one of its input formats. `babeltrace` is
-the default viewer when using `lttng view`. To use it manually, first
-list active tracing sessions by doing the following (assuming the relay
-daemon to connect to runs on the same host):
-
-<pre class="term">
-babeltrace --input-format lttng-live net://localhost
-</pre>
-
-Then, choose a tracing session and start viewing events as they arrive
-using LTTng live:
-
-<pre class="term">
-babeltrace --input-format lttng-live net://localhost/host/hostname/my-session
-</pre>
+++ /dev/null
----
-id: mi
-since: 2.6
----
-
-The `lttng` tool aims at providing a command output as human-readable as
-possible. While this output is easy to parse by a human being, machines
-have a hard time.
-
-This is why the `lttng` tool provides the general `--mi` option, which
-must specify a machine interface output format. As of the latest
-LTTng stable release, only the `xml` format is supported. A schema
-definition (XSD) is made
-<a href="https://github.com/lttng/lttng-tools/blob/master/src/common/mi_lttng.xsd" class="ext">available</a>
-to ease the integration with external tools as much as possible.
-
-The `--mi` option can be used in conjunction with all `lttng` commands.
-Here are some examples:
-
-<pre class="term">
-lttng --mi xml create some-session
-lttng --mi xml list some-session
-lttng --mi xml list --kernel
-lttng --mi xml enable-event --kernel --syscall open
-lttng --mi xml start
-</pre>
+++ /dev/null
----
-id: persistent-memory-file-systems
-since: 2.7
----
-
-<a href="https://en.wikipedia.org/wiki/Non-volatile_random-access_memory" class="ext">Non-volatile random-access memory</a>
-(NVRAM) is random-access memory that retains its information when power is turned off (non-volatile).
-Systems with such memory can store data structures in RAM
-and retrieve them after a reboot, without flushing to typical _storage_.
-
-Linux supports NVRAM file systems thanks to either
-<a href="http://pramfs.sourceforge.net/" class="ext">PRAMFS</a> or
-<a href="https://www.kernel.org/doc/Documentation/filesystems/dax.txt" class="ext">DAX</a> + <a href="http://lkml.iu.edu/hypermail/linux/kernel/1504.1/03463.html" class="ext">pmem</a>
-(requires Linux 4.1+).
-
-This documentation does not describe how to operate such file systems;
-it is assumed that you have a working persistent memory file system.
-
-When creating an LTTng tracing session, you can use the `--shm-path`
-option to specify the path of the shared memory holding the ring
-buffers. Specifying a location on an NVRAM file system makes it possible
-to retrieve the latest recorded trace data when the system reboots
-after a crash.
-
-Example:
-
-<pre class="term">
-lttng create <strong>--shm-path /path/to/shm</strong>
-</pre>
-
-The binary layout of the ring buffer files is not exactly the same as
-the trace files layout. To view the events of ring buffer files after
-a system crash, use the `lttng-crash` utility:
-
-<pre class="term">
-lttng-crash /path/to/shm
-</pre>
-
-This extracts the trace data behind the scenes and runs
-<a href="http://diamon.org/babeltrace" class="ext"><code>babeltrace</code></a>
-to view the events. To extract the trace data to an LTTng trace without
-viewing the events, use the `--extract` option:
-
-<pre class="term">
-lttng-crash <strong>--extract /path/to/trace</strong> /path/to/shm
-</pre>
-
-See the <a href="/man/1/lttng-crash/v2.7" class="ext"><code>lttng-crash</code> man page</a>
-for the complete list of options.
+++ /dev/null
----
-id: pid-tracking
-since: 2.7
----
-
-It's often useful to allow only specific process IDs (PIDs) to emit
-enabled events. For example, you may wish to record all the system
-calls made by a given process (à la
-<a href="http://linux.die.net/man/1/strace" class="ext">strace</a>).
-
-The `lttng track` and `lttng untrack` commands serve this purpose. Both
-commands operate on a whitelist of process IDs. The `track` command
-adds entries to this whitelist while the `untrack` command removes
-entries. Any process having one of the PIDs in the whitelist is allowed
-to emit [enabled](#doc-enabling-disabling-events) LTTng events.
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>The PID tracker tracks the
- <em>numeric process IDs</em>. Should a process with a given tracked
- ID exit and another process be given this ID, then the latter would
- also be allowed to emit events.
-</p>
-</div>
-
-For the sake of the following examples, assume the target system has 16
-possible PIDs. When a [tracing session](#doc-creating-destroying-tracing-sessions)
-is created, the whitelist contains all the possible PIDs:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-all.png" alt="All PIDs are tracked">
-<figcaption>All PIDs are tracked</figcaption>
-</figure>
-
-When the whitelist is full and the `track` command is executed to specify
-some PIDs to track, the whitelist is first cleared, then the specific
-PIDs are tracked. For example, after
-
-<pre class="term">
-lttng track --pid 3,4,7,10,13
-</pre>
-
-the whitelist is:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-3-4-7-10-13.png" alt="PIDs 3, 4, 7, 10, and 13 are tracked">
-<figcaption>PIDs 3, 4, 7, 10, and 13 are tracked</figcaption>
-</figure>
-
-More PIDs can be added to the whitelist afterwards:
-
-<pre class="term">
-lttng track --pid 1,15,16
-</pre>
-
-gives:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-1-3-4-7-10-13-15-16.png" alt="PIDs 1, 15, and 16 are added to the whitelist">
-<figcaption>PIDs 1, 15, and 16 are added to the whitelist</figcaption>
-</figure>
-
-The `untrack` command removes entries from the PID tracker's whitelist.
-Given the last example, the following command:
-
-<pre class="term">
-lttng untrack --pid 3,7,10,13
-</pre>
-
-leads to this whitelist:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-1-4-15-16.png" alt="PIDs 3, 7, 10, and 13 are removed from the whitelist">
-<figcaption>PIDs 3, 7, 10, and 13 are removed from the whitelist</figcaption>
-</figure>
-
-All possible PIDs can be tracked again using the `--all` option of
-`lttng track`:
-
-<pre class="term">
-lttng track --pid --all
-</pre>
-
-gives:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-all.png" alt="All PIDs are tracked">
-<figcaption>All PIDs are tracked</figcaption>
-</figure>
-
-A very typical use case with PID tracking is starting with an empty
-whitelist, then [starting the tracers](#doc-basic-tracing-session-control),
-and then adding PIDs manually while tracing is active. This can be
-accomplished by using the `--all` option of the `untrack` command
-to clear the whitelist after a tracing session is created:
-
-<pre class="term">
-lttng untrack --pid --all
-</pre>
-
-gives:
-
-<figure class="img img-100">
-<img src="/images/docs27/untrack-all.png" alt="No PIDs are tracked">
-<figcaption>No PIDs are tracked</figcaption>
-</figure>
-
-Tracing with this whitelist configuration does not produce any event
-because no processes are tracked. The `track` command can be used
-as usual to track specific PIDs, for example:
-
-<pre class="term">
-lttng track --pid 6,11
-</pre>
-
-results in:
-
-<figure class="img img-100">
-<img src="/images/docs27/track-6-11.png" alt="PIDs 6 and 11 are tracked">
-<figcaption>PIDs 6 and 11 are tracked</figcaption>
-</figure>
+++ /dev/null
----
-id: saving-loading-tracing-session
-since: 2.5
----
-
-Configuring a tracing session may be long: creating and enabling
-channels with specific parameters, enabling kernel and user space
-domain events with specific log levels and filters, and adding context
-to some channels are just a few of the many possible operations using
-the `lttng` command line tool. If you're going to use LTTng to solve real
-world problems, chances are you're going to have to record events using
-the same tracing session setup over and over, modifying a few variables
-each time in your instrumented program or environment. To avoid
-constant tracing session reconfiguration, the `lttng` tool is able to
-save and load tracing session configurations to/from XML files.
-
-To save a given tracing session configuration, do:
-
-<pre class="term">
-lttng save my-session
-</pre>
-
-where `my-session` is the name of the tracing session to save. Tracing
-session configurations are saved to `~/.lttng/sessions` by default;
-use the `--output-path` option to change this destination directory.
-
-All configuration parameters are saved:
-
- * tracing session name
- * trace data output path
- * channels with their state and all their parameters
- * context information added to channels
- * events with their state, log level and filter
- * tracing activity (started or stopped)
-
-To load a tracing session, simply do:
-
-<pre class="term">
-lttng load my-session
-</pre>
-
-or, if you used a custom path:
-
-<pre class="term">
-lttng load --input-path /path/to/my-session.lttng
-</pre>
-
-Your saved tracing session is restored as if you just configured
-it manually.
+++ /dev/null
----
-id: sending-trace-data-over-the-network
----
-
-The possibility of sending trace data over the network comes as a
-built-in feature of LTTng-tools. For this to be possible, an LTTng
-_relay daemon_ must be executed and listening on the machine where
-trace data is to be received, and the user must create a tracing
-session using appropriate options to forward trace data to the remote
-relay daemon.
-
-The relay daemon listens on two different TCP ports: one for control
-information and the other for actual trace data.
-
-Starting the relay daemon on the remote machine is easy:
-
-<pre class="term">
-lttng-relayd
-</pre>
-
-This makes it listen to its default ports: 5342 for control and
-5343 for trace data. The `--control-port` and `--data-port` options may
-be used to specify different ports.
-
-Traces written by `lttng-relayd` are written to
-<code>~/lttng-traces/<em>hostname</em>/<em>session</em></code> by
-default, where <code><em>hostname</em></code> is the host name of the
-traced (monitored) system and <code><em>session</em></code> is the
-tracing session name. Use the `--output` option to write trace data
-outside `~/lttng-traces`.
-
-On the sending side, a tracing session must be created using the
-`lttng` tool with the `--set-url` option to connect to the distant
-relay daemon:
-
-<pre class="term">
-lttng create my-session --set-url net://distant-host
-</pre>
-
-The URL format is described in the output of `lttng create --help`.
-The above example uses the default ports; the `--ctrl-url` and
-`--data-url` options may be used to set the control and data URLs
-individually.
-
-Once this basic setup is completed and the connection is established,
-you may use the `lttng` tool on the target machine as usual; everything
-you do is transparently forwarded to the remote machine if needed.
-For example, a parameter changing the maximum size of trace files
-only has an effect on the distant relay daemon actually writing
-the trace.
+++ /dev/null
----
-id: taking-a-snapshot
-since: 2.3
----
-
-The normal behavior of LTTng is to record trace data as trace files.
-This is ideal for keeping a long history of events that occurred on
-the target system and applications, but may be too much data in some
-situations. For example, you may wish to trace your application
-continuously until some critical situation happens, in which case you
-would only need the latest few recorded events to perform the desired
-analysis, not multi-gigabyte trace files.
-
-LTTng has an interesting feature called _snapshots_. When creating
-a tracing session in snapshot mode, no trace files are written; the
-tracers' sub-buffers are constantly overwriting the oldest recorded
-events with the newest. At any time, either when the tracers are started
-or stopped, you may take a snapshot of those sub-buffers.
-
-There is no difference between the format of a normal trace file and the
-format of a snapshot: viewers of LTTng traces also support LTTng
-snapshots. By default, snapshots are written to disk, but they may also
-be sent over the network.
-
-To create a tracing session in snapshot mode, do:
-
-<pre class="term">
-lttng create --snapshot my-snapshot-session
-</pre>
-
-Next, enable channels, events and add context to channels as usual.
-Once a tracing session is created in snapshot mode, channels are
-forced to use the
-[overwrite](#doc-channel-overwrite-mode-vs-discard-mode) mode
-(`--overwrite` option of the `enable-channel` command; also called
-_flight recorder mode_) and have an `mmap()` channel type
-(`--output mmap`).
-
-Start tracing. When you're ready to take a snapshot, do:
-
-<pre class="term">
-lttng snapshot record --name my-snapshot
-</pre>
-
-This records a snapshot named `my-snapshot` of all channels of
-all domains of the current tracing session. By default, snapshots files
-are recorded in the path returned by `lttng snapshot list-output`. You
-may change this path or decide to send snapshots over the network
-using either:
-
- 1. an output path/URL specified when creating the tracing session
- (`lttng create`)
- 2. an added snapshot output path/URL using
- `lttng snapshot add-output`
- 3. an output path/URL provided directly to the
- `lttng snapshot record` command
-
-Method 3 overrides method 2 which overrides method 1. When specifying
-a URL, a relay daemon must be listening on some machine (see
-[Sending trace data over the network](#doc-sending-trace-data-over-the-network)).
-
-If you need to make absolutely sure that the output file won't be
-larger than a certain limit, you can set a maximum snapshot size when
-taking it with the `--max-size` option:
-
-<pre class="term">
-lttng snapshot record --name my-snapshot --max-size 2M
-</pre>
-
-Older recorded events are discarded in order to respect this
-maximum size.
+++ /dev/null
----
-id: building-32-bit-lttng-tools
----
-
-Since the host is a 64-bit system, most 32-bit binaries and libraries of
-LTTng-tools are not needed; the host uses their 64-bit counterparts.
-The required step here is building and installing a 32-bit consumer
-daemon.
-
-Follow this:
-
-<pre class="term">
-git clone http://git.lttng.org/lttng-tools.git
-cd lttng-ust
-./bootstrap
-./configure --prefix=/usr \
- --libdir=/usr/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
- LDFLAGS=-L/usr/lib32
-make
-cd src/bin/lttng-consumerd
-sudo make install
-sudo ldconfig
-</pre>
-
-The above commands build all the LTTng-tools project as 32-bit
-applications, but only installs the 32-bit consumer daemon.
+++ /dev/null
----
-id: building-32-bit-lttng-ust
----
-
-Follow this:
-
-<pre class="term">
-git clone http://git.lttng.org/lttng-ust.git
-cd lttng-ust
-./bootstrap
-./configure --prefix=/usr \
- --libdir=/usr/lib32 \
- CFLAGS=-m32 CXXFLAGS=-m32 \
- LDFLAGS=-L/usr/lib32
-make
-sudo make install
-sudo ldconfig
-</pre>
-
-`-L/usr/lib32` is required for the build to find the 32-bit versions
-of Userspace RCU and other dependencies.
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>Depending on your Linux distribution,
- 32-bit libraries could be installed at a different location than
- <code>/usr/lib32</code>. For example, Debian is known to install
- some 32-bit libraries in <code>/usr/lib/i386-linux-gnu</code>.
-</p>
-<p>
- In this case, make sure to set <code>LDFLAGS</code> to all the
- relevant 32-bit library paths, for example,
- <code>LDFLAGS="-L/usr/lib32 -L/usr/lib/i386-linux-gnu"</code>.
-</p>
-</div>
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>You may add options to
- <code>./configure</code> if you need them (for
- Java and SystemTap support, for example). Look at
- <code>./configure --help</code> for more information.
-</p>
-</div>
+++ /dev/null
----
-id: building-32-bit-userspace-rcu
----
-
-Follow this:
-
-<pre class="term">
-git clone git://git.urcu.so/urcu.git
-cd urcu
-./bootstrap
-./configure --libdir=/usr/lib32 CFLAGS=-m32
-make
-sudo make install
-sudo ldconfig
-</pre>
-
-The `-m32` C compiler flag creates 32-bit object files and `--libdir`
-indicates where to install the resulting libraries.
+++ /dev/null
----
-id: building-64-bit-lttng-tools
----
-
-Finally, you need to build a 64-bit version of LTTng-tools which is
-aware of the 32-bit consumer daemon previously built and installed:
-
-<pre class="term">
-make clean
-./bootstrap
-./configure --prefix=/usr \
- --with-consumerd32-libdir=/usr/lib32 \
- --with-consumerd32-bin=/usr/lib32/lttng/libexec/lttng-consumerd
-make
-sudo make install
-sudo ldconfig
-</pre>
-
-Henceforth, the 64-bit session daemon automatically finds the
-32-bit consumer daemon if required.
+++ /dev/null
----
-id: building-instrumented-32-bit-c-application
----
-
-Let us reuse the _Hello world_ example of
-[Tracing your own user application](#doc-tracing-your-own-user-application)
-([Getting started](#doc-getting-started) chapter).
-
-The instrumentation process is unaltered.
-
-First, a typical 64-bit build (assuming you're running a 64-bit system):
-
-<pre class="term">
-gcc -o hello64 -I. hello.c hello-tp.c -ldl -llttng-ust
-</pre>
-
-Now, a 32-bit build:
-
-<pre class="term">
-gcc -o hello32 -I. <strong>-m32</strong> hello.c hello-tp.c <strong>-L/usr/lib32</strong> \
- -ldl -llttng-ust <strong>-Wl,-rpath,/usr/lib32</strong>
-</pre>
-
-The `-rpath` option, passed to the linker, makes the dynamic loader
-check for libraries in `/usr/lib32` before looking in its default paths,
-where it should find the 32-bit version of `liblttng-ust`.
+++ /dev/null
----
-id: instrumenting-32-bit-app-on-64-bit-system
----
-
-In order to trace a 32-bit application running on a 64-bit system,
-LTTng must use a dedicated 32-bit
-[consumer daemon](#doc-lttng-consumerd). This section discusses how to
-build that daemon (which is _not_ part of the default 64-bit LTTng
-build) and the LTTng 32-bit tracing libraries, and how to instrument
-a 32-bit application in that context.
-
-Make sure you install all 32-bit versions of LTTng dependencies.
-Their names can be found in the `README.md` files of each LTTng package
-source. How to find and install them depends on your target's
-Linux distribution. `gcc-multilib` is a common package name for the
-multilib version of GCC, which you also need.
-
-The following packages will be built for 32-bit support on a 64-bit
-system: <a href="http://urcu.so/" class="ext">Userspace RCU</a>,
-LTTng-UST and LTTng-tools.
+++ /dev/null
----
-id: running-32-bit-and-64-bit-c-applications
----
-
-Now, both 32-bit and 64-bit versions of the _Hello world_ example above
-can be traced in the same tracing session. Use the `lttng` tool as usual
-to create a tracing session and start tracing:
-
-<pre class="term">
-lttng create session-3264
-lttng enable-event -u -a
-./hello32
-./hello64
-lttng stop
-</pre>
-
-Use `lttng view` to verify both processes were
-successfully traced.
+++ /dev/null
----
-id: advanced-instrumenting-techniques
----
-
-This section presents some advanced techniques related to
-LTTng instrumenting.
+++ /dev/null
----
-id: assigning-log-levels
----
-
-Optionally, a log level can be assigned to a defined tracepoint.
-Assigning different levels of importance to tracepoints can be useful;
-when controlling tracing sessions,
-[you can choose](#doc-controlling-tracing) to only enable tracepoints
-falling into a specific log level range.
-
-Log levels are assigned to defined tracepoints using the
-`TRACEPOINT_LOGLEVEL()` macro. The latter must be used _after_ having
-used `TRACEPOINT_EVENT()` for a given tracepoint. The
-`TRACEPOINT_LOGLEVEL()` macro has the following construct:
-
-~~~ c
-TRACEPOINT_LOGLEVEL(<provider name>, <tracepoint name>, <log level>)
-~~~
-
-where the first two arguments are the same as the first two arguments
-of `TRACEPOINT_EVENT()` and `<log level>` is one
-of the values given in the
-[LTTng-UST library reference](#doc-liblttng-ust-tracepoint-loglevel)
-section.
-
-As an example, let's assign a `TRACE_DEBUG_UNIT` log level to our
-previous tracepoint definition:
-
-~~~ c
-TRACEPOINT_LOGLEVEL(my_provider, my_tracepoint, TRACE_DEBUG_UNIT)
-~~~
+++ /dev/null
----
-id: dynamic-linking
----
-
-The second approach to package the tracepoint providers is to use
-dynamic linking: the library and its member functions are explicitly
-sought, loaded and unloaded at runtime using `libdl`.
-
-It has to be noted that, for a variety of reasons, the created shared
-library is be dynamically _loaded_, as opposed to dynamically
-_linked_. The tracepoint provider shared object is, however, linked
-with `liblttng-ust`, so that `liblttng-ust` is guaranteed to be loaded
-as soon as the tracepoint provider is. If the tracepoint provider is
-not loaded, since the application itself is not linked with
-`liblttng-ust`, the latter is not loaded at all and the tracepoint calls
-become inert.
-
-The process to create the tracepoint provider shared object is pretty
-much the same as the static library method, except that:
-
- * since the tracepoint provider is not part of the application
- anymore, `TRACEPOINT_DEFINE` _must_ be defined, for each tracepoint
- provider, in exactly one translation unit (C source file) of the
- _application_;
- * `TRACEPOINT_PROBE_DYNAMIC_LINKAGE` must be defined next to
- `TRACEPOINT_DEFINE`.
-
-Regarding `TRACEPOINT_DEFINE` and `TRACEPOINT_PROBE_DYNAMIC_LINKAGE`,
-the recommended practice is to use a separate C source file in your
-application to define them, then include the tracepoint provider
-header files afterwards. For example:
-
-~~~ c
-#define TRACEPOINT_DEFINE
-#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
-
-/* include the header files of one or more tracepoint providers below */
-#include "tp1.h"
-#include "tp2.h"
-#include "tp3.h"
-~~~
-
-`TRACEPOINT_PROBE_DYNAMIC_LINKAGE` makes the macros included afterwards
-(by including the tracepoint provider header, which itself includes
-LTTng-UST headers) aware that the tracepoint provider is to be loaded
-dynamically and not part of the application's executable.
-
-The tracepoint provider object file used to create the shared library
-is built like it is using the static library method, only with the
-`-fpic` option added:
-
-<pre class="term">
-gcc -c <strong>-fpic</strong> -I. tp.c
-</pre>
-
-It is then linked as a shared library like this:
-
-<pre class="term">
-gcc <strong>-shared -Wl,--no-as-needed -o tp.so -llttng-ust</strong> tp.o
-</pre>
-
-As previously stated, this tracepoint provider shared object isn't
-linked with the user application: it's loaded manually. This is
-why the application is built with no mention of this tracepoint
-provider, but still needs `libdl`:
-
-<pre class="term">
-gcc -o app other.o files.o of.o your.o app.o <strong>-ldl</strong>
-</pre>
-
-Now, to make LTTng-UST tracing available to the application, the
-`LD_PRELOAD` environment variable is used to preload the tracepoint
-provider shared library _before_ the application actually starts:
-
-<pre class="term">
-<strong>LD_PRELOAD=/path/to/tp.so</strong> ./app
-</pre>
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>It is not safe to use
- <code>dlclose()</code> on a tracepoint provider shared object that
- is being actively used for tracing, due to a lack of reference
- counting from LTTng-UST to the shared object.
-</p>
-
-<p>
- For example, statically linking a tracepoint provider to a
- shared object which is to be dynamically loaded by an application
- (a plugin, for example) is not safe: the shared object, which
- contains the tracepoint provider, could be dynamically closed
- (<code>dlclose()</code>) at any time by the application.
-</p>
-
-<p>
- To instrument a shared object, either:
-</p>
-
-<ol>
- <li>
- Statically link the tracepoint provider to the
- <em>application</em>, or
- </li>
- <li>
- Build the tracepoint provider as a shared object (following
- the procedure shown in this section), and preload it when
- tracing is needed using the <code>LD_PRELOAD</code>
- environment variable.
- </li>
-</ol>
-</div>
-
-Your application will still work without this preloading, albeit without
-LTTng-UST tracing support:
-
-<pre class="term">
-./app
-</pre>
-
+++ /dev/null
----
-id: building-tracepoint-providers-and-user-application
----
-
-This section explains the final step of using LTTng-UST for tracing
-a user space C application (beside running the application): building and
-linking tracepoint providers and the application itself.
-
-As discussed above, the macros used by the user-written tracepoint provider
-header file are useless until actually used to create probes code
-(global data structures and functions) in a translation unit (C source file).
-This is accomplished by defining `TRACEPOINT_CREATE_PROBES` in a translation
-unit and then including the tracepoint provider header file.
-When `TRACEPOINT_CREATE_PROBES` is defined, macros used and included by
-the tracepoint provider header produce actual source code needed by any
-application using the defined tracepoints. Defining
-`TRACEPOINT_CREATE_PROBES` produces code used when registering
-tracepoint providers when the tracepoint provider package loads.
-
-The other important definition is `TRACEPOINT_DEFINE`. This one creates
-global, per-tracepoint structures referencing the tracepoint providers
-data. Those structures are required by the actual functions inserted
-where `tracepoint()` macros are placed and need to be defined by the
-instrumented application.
-
-Both `TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` need to be defined
-at some places in order to trace a user space C application using LTTng.
-Although explaining their exact mechanism is beyond the scope of this
-document, the reason they both exist separately is to allow the trace
-providers to be packaged as a shared object (dynamically loaded library).
-
-There are two ways to compile and link the tracepoint providers
-with the application: _[statically](#doc-static-linking)_ or
-_[dynamically](#doc-dynamic-linking)_. Both methods are covered in the
-following subsections.
+++ /dev/null
----
-id: lttng-ust-pkg-config
----
-
-On some distributions, LTTng-UST is shipped with a pkg-config metadata
-file, so that you may use the `pkg-config` tool:
-
-<pre class="term">
-pkg-config --libs lttng-ust
-</pre>
-
-This prints `-llttng-ust -ldl` on Linux systems.
-
-You may also check the LTTng-UST version using `pkg-config`:
-
-<pre class="term">
-pkg-config --modversion lttng-ust
-</pre>
-
-For more information about pkg-config, see
-<a href="http://linux.die.net/man/1/pkg-config" class="ext">its man page</a>.
+++ /dev/null
----
-id: static-linking
----
-
-With the static linking method, compiled tracepoint providers are copied
-into the target application. There are three ways to do this:
-
- 1. Use one of your **existing C source files** to create probes.
- 2. Create probes in a separate C source file and build it as an
- **object file** to be linked with the application (more decoupled).
- 3. Create probes in a separate C source file, build it as an
- object file and archive it to create a **static library**
- (more decoupled, more portable).
-
-The first approach is to define `TRACEPOINT_CREATE_PROBES` and include
-your tracepoint provider(s) header file(s) directly into an existing C
-source file. Here's an example:
-
-~~~ c
-#include <stdlib.h>
-#include <stdio.h>
-/* ... */
-
-#define TRACEPOINT_CREATE_PROBES
-#define TRACEPOINT_DEFINE
-#include "tp.h"
-
-/* ... */
-
-int my_func(int a, const char* b)
-{
- /* ... */
-
- tracepoint(my_provider, my_tracepoint, buf, sz, limit, &tt)
-
- /* ... */
-}
-
-/* ... */
-~~~
-
-Again, before including a given tracepoint provider header file,
-`TRACEPOINT_CREATE_PROBES` and `TRACEPOINT_DEFINE` must be defined in
-one, **and only one**, translation unit. Other C source files of the
-same application may include `tp.h` to use tracepoints with
-the `tracepoint()` macro, but must not define
-`TRACEPOINT_CREATE_PROBES`/`TRACEPOINT_DEFINE` again.
-
-This translation unit may be built as an object file by making sure to
-add `.` to the include path:
-
-<pre class="term">
-gcc -c <strong>-I.</strong> file.c
-</pre>
-
-The second approach is to isolate the tracepoint provider code into a
-separate object file by using a dedicated C source file to create probes:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-
-#include "tp.h"
-~~~
-
-`TRACEPOINT_DEFINE` must be defined by a translation unit of the
-application. Since we're talking about static linking here, it could as
-well be defined directly in the file above, before `#include "tp.h"`:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-#define TRACEPOINT_DEFINE
-
-#include "tp.h"
-~~~
-
-This is actually what [`lttng-gen-tp`](#doc-lttng-gen-tp) does, and is
-the recommended practice.
-
-Build the tracepoint provider:
-
-<pre class="term">
-gcc -c -I. tp.c
-</pre>
-
-Finally, the resulting object file may be archived to create a
-more portable tracepoint provider static library:
-
-<pre class="term">
-ar rc tp.a tp.o
-</pre>
-
-Using a static library does have the advantage of centralising the
-tracepoint providers objects so they can be shared between multiple
-applications. This way, when the tracepoint provider is modified, the
-source code changes don't have to be patched into each application's source
-code tree. The applications need to be relinked after each change, but need
-not to be otherwise recompiled (unless the tracepoint provider's API
-changes).
-
-Regardless of which method you choose, you end up with an object file
-(potentially archived) containing the trace providers assembled code.
-To link this code with the rest of your application, you must also link
-with `liblttng-ust` and `libdl`:
-
-<pre class="term">
-gcc -o app <strong>tp.o</strong> other.o files.o of.o your.o app.o <strong>-llttng-ust -ldl</strong>
-</pre>
-
-or
-
-<pre class="term">
-gcc -o app <strong>tp.a</strong> other.o files.o of.o your.o app.o -llttng-ust -ldl
-</pre>
-
-If you're using a <abbr title="Berkeley Software Distribution">BSD</abbr>
-system, replace `-ldl` with `-lc`:
-
-<pre class="term">
-gcc -o app tp.a other.o files.o of.o your.o app.o -llttng-ust <strong>-lc</strong>
-</pre>
-
-The application can be started as usual, for example:
-
-<pre class="term">
-./app
-</pre>
-
-The `lttng` command line tool can be used to
-[control tracing](#doc-controlling-tracing).
+++ /dev/null
----
-id: using-lttng-ust-with-daemons
----
-
-Some extra care is needed when using `liblttng-ust` with daemon
-applications that call `fork()`, `clone()` or BSD's `rfork()` without
-a following `exec()` family system call. The `liblttng-ust-fork`
-library must be preloaded for the application.
-
-Example:
-
-<pre class="term">
-<strong>LD_PRELOAD=liblttng-ust-fork.so</strong> ./app
-</pre>
-
-Or, if you're using a tracepoint provider shared library:
-
-<pre class="term">
-<strong>LD_PRELOAD="liblttng-ust-fork.so /path/to/tp.so"</strong> ./app
-</pre>
+++ /dev/null
----
-id: defining-tracepoints
----
-
-As written in [Tracepoint provider](#doc-tracepoint-provider),
-tracepoints are defined using the
-`TRACEPOINT_EVENT()` macro. Each tracepoint, when called using the
-`tracepoint()` macro in the actual application's source code, generates
-a specific event type with its own fields.
-
-Let's have another look at the example above, with a few added comments:
-
-~~~ c
-TRACEPOINT_EVENT(
- /* tracepoint provider name */
- my_provider,
-
- /* tracepoint/event name */
- my_first_tracepoint,
-
- /* list of tracepoint arguments */
- TP_ARGS(
- int, my_integer_arg,
- char*, my_string_arg
- ),
-
- /* list of fields of eventual event */
- TP_FIELDS(
- ctf_string(my_string_field, my_string_arg)
- ctf_integer(int, my_integer_field, my_integer_arg)
- )
-)
-~~~
-
-The tracepoint provider name must match the name of the tracepoint
-provider in which this tracepoint is defined
-(see [Tracepoint provider](#doc-tracepoint-provider)). In other words,
-always use the same string as the value of `TRACEPOINT_PROVIDER` above.
-
-The tracepoint name becomes the event name once events are recorded
-by the LTTng-UST tracer. It must follow the tracepoint provider name
-syntax: start with a letter and contain either letters, numbers or
-underscores. Two tracepoints under the same provider cannot have the
-same name. In other words, you cannot overload a tracepoint like you
-would overload functions and methods in C++/Java.
-
-<div class="tip">
-<p><span class="t">Note:</span>The concatenation of the tracepoint
-provider name and the tracepoint name cannot exceed 254 characters. If
-it does, the instrumented application compiles and runs, but LTTng
-issues multiple warnings and you could experience serious problems.</p>
-</div>
-
-The list of tracepoint arguments gives this tracepoint its signature:
-see it like the declaration of a C function. The format of `TP_ARGS()`
-arguments is: C type, then argument name; repeat as needed, up to ten
-times. For example, if we were to replicate the signature of C standard
-library's `fseek()`, the `TP_ARGS()` part would look like:
-
-~~~ c
- TP_ARGS(
- FILE*, stream,
- long int, offset,
- int, origin
- ),
-~~~
-
-Of course, you need to include appropriate header files before
-the `TRACEPOINT_EVENT()` macro calls if any argument has a complex type.
-
-`TP_ARGS()` may not be omitted, but may be empty. `TP_ARGS(void)` is
-also accepted.
-
-The list of fields is where the fun really begins. The fields defined
-in this list are the fields of the events generated by the execution
-of this tracepoint. Each tracepoint field definition has a C
-_argument expression_ which is evaluated when the execution reaches
-the tracepoint. Tracepoint arguments _may be_ used freely in those
-argument expressions, but they _don't_ have to.
-
-There are several types of tracepoint fields available. The macros to
-define them are given and explained in the
-[LTTng-UST library reference](#doc-liblttng-ust-tp-fields) section.
-
-Field names must follow the standard C identifier syntax: letter, then
-optional sequence of letters, numbers or underscores. Each field must have
-a different name.
-
-Those `ctf_*()` macros are added to the `TP_FIELDS()` part of
-`TRACEPOINT_EVENT()`. Note that they are not delimited by commas.
-`TP_FIELDS()` may be empty, but the `TP_FIELDS(void)` form is _not_
-accepted.
-
-The following snippet shows how argument expressions may be used in
-tracepoint fields and how they may refer freely to tracepoint arguments.
-
-~~~ c
-/* for struct stat */
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <unistd.h>
-
-TRACEPOINT_EVENT(
- my_provider,
- my_tracepoint,
- TP_ARGS(
- int, my_int_arg,
- char*, my_str_arg,
- struct stat*, st
- ),
- TP_FIELDS(
- /* simple integer field with constant value */
- ctf_integer(
- int, /* field C type */
- my_constant_field, /* field name */
- 23 + 17 /* argument expression */
- )
-
- /* my_int_arg tracepoint argument */
- ctf_integer(
- int,
- my_int_arg_field,
- my_int_arg
- )
-
- /* my_int_arg squared */
- ctf_integer(
- int,
- my_int_arg_field2,
- my_int_arg * my_int_arg
- )
-
- /* sum of first 4 characters of my_str_arg */
- ctf_integer(
- int,
- sum4,
- my_str_arg[0] + my_str_arg[1] +
- my_str_arg[2] + my_str_arg[3]
- )
-
- /* my_str_arg as string field */
- ctf_string(
- my_str_arg_field, /* field name */
- my_str_arg /* argument expression */
- )
-
- /* st_size member of st tracepoint argument, hexadecimal */
- ctf_integer_hex(
- off_t, /* field C type */
- size_field, /* field name */
- st->st_size /* argument expression */
- )
-
- /* st_size member of st tracepoint argument, as double */
- ctf_float(
- double, /* field C type */
- size_dbl_field, /* field name */
- (double) st->st_size /* argument expression */
- )
-
- /* half of my_str_arg string as text sequence */
- ctf_sequence_text(
- char, /* element C type */
- half_my_str_arg_field, /* field name */
- my_str_arg, /* argument expression */
- size_t, /* length expression C type */
- strlen(my_str_arg) / 2 /* length expression */
- )
- )
-)
-~~~
-
-As you can see, having a custom argument expression for each field
-makes tracepoints very flexible for tracing a user space C application.
-This tracepoint definition is reused later in this guide, when
-actually using tracepoints in a user space application.
+++ /dev/null
----
-id: c-application
----
-
-Instrumenting a C (or C++) application, be it an executable program or
-a library, implies using LTTng-UST, the
-user space tracing component of LTTng. For C/C++ applications, the
-LTTng-UST package includes a dynamically loaded library
-(`liblttng-ust`), C headers and the `lttng-gen-tp` command line utility.
-
-Since C and C++ are the base languages of virtually all other
-programming languages
-(Java virtual machine, Python, Perl, PHP and Node.js interpreters, to
-name a few), implementing user space tracing for an unsupported language
-is just a matter of using the LTTng-UST C API at the right places.
-
-The usual work flow to instrument a user space C application with
-LTTng-UST is:
-
- 1. Define tracepoints (actual probes)
- 2. Write tracepoint providers
- 3. Insert tracepoints into target source code
- 4. Package (build) tracepoint providers
- 5. Build user application and link it with tracepoint providers
-
-The steps above are discussed in greater detail in the following
-subsections.
+++ /dev/null
----
-id: lttng-gen-tp
----
-
-LTTng-UST ships with `lttng-gen-tp`, a handy command line utility for
-generating most of the stuff discussed above. It takes a _template file_,
-with a name usually ending with the `.tp` extension, containing only
-tracepoint definitions, and outputs a tracepoint provider (either a C
-source file or a precompiled object file) with its header file.
-
-`lttng-gen-tp` should suffice in [static linking](#doc-static-linking)
-situations. When using it, write a template file containing a list of
-`TRACEPOINT_EVENT()` macro calls. The tool finds the provider names
-used and generate the appropriate files which are going to look a lot
-like `tp.h` and `tp.c` above.
-
-Just call `lttng-gen-tp` like this:
-
-<pre class="term">
-lttng-gen-tp my-template.tp
-</pre>
-
-`my-template.c`, `my-template.o` and `my-template.h` are created
-in the same directory.
-
-You may specify custom C flags passed to the compiler invoked by
-`lttng-gen-tp` using the `CFLAGS` environment variable:
-
-<pre class="term">
-CFLAGS=-I/custom/include/path lttng-gen-tp my-template.tp
-</pre>
-
-For more information on `lttng-gen-tp`, see
-<a href="/man/1/lttng-gen-tp/v2.7" class="ext">its man page</a>.
+++ /dev/null
----
-id: lttng-ust-environment-variables-compiler-flags
----
-
-A few special environment variables and compile flags may affect the
-behavior of LTTng-UST.
-
-LTTng-UST's debugging can be activated by setting the environment
-variable `LTTNG_UST_DEBUG` to `1` when launching the application. It
-can also be enabled at compile time by defining `LTTNG_UST_DEBUG` when
-compiling LTTng-UST (using the `-DLTTNG_UST_DEBUG` compiler option).
-
-The environment variable `LTTNG_UST_REGISTER_TIMEOUT` can be used to
-specify how long the application should wait for the
-[session daemon](#doc-lttng-sessiond)'s _registration done_ command
-before proceeding to execute the main program. The timeout value is
-specified in milliseconds. 0 means _don't wait_. -1 means
-_wait forever_. Setting this environment variable to 0 is recommended
-for applications with time contraints on the process startup time.
-
-The default value of `LTTNG_UST_REGISTER_TIMEOUT` (when not defined)
-is **3000 ms**.
-
-The compilation definition `LTTNG_UST_DEBUG_VALGRIND` should be enabled
-at build time (`-DLTTNG_UST_DEBUG_VALGRIND`) to allow `liblttng-ust`
-to be used with <a href="http://valgrind.org/" class="ext">Valgrind</a>.
-The side effect of defining `LTTNG_UST_DEBUG_VALGRIND` is that per-CPU
-buffering is disabled.
+++ /dev/null
----
-id: probing-the-application-source-code
----
-
-Once tracepoints are properly defined within a tracepoint provider,
-they may be inserted into the user application to be instrumented
-using the `tracepoint()` macro. Its first argument is the tracepoint
-provider name and its second is the tracepoint name. The next, optional
-arguments are defined by the `TP_ARGS()` part of the definition of
-the tracepoint to use.
-
-As an example, let us again take the following tracepoint definition:
-
-~~~ c
-TRACEPOINT_EVENT(
- /* tracepoint provider name */
- my_provider,
-
- /* tracepoint/event name */
- my_first_tracepoint,
-
- /* list of tracepoint arguments */
- TP_ARGS(
- int, my_integer_arg,
- char*, my_string_arg
- ),
-
- /* list of fields of eventual event */
- TP_FIELDS(
- ctf_string(my_string_field, my_string_arg)
- ctf_integer(int, my_integer_field, my_integer_arg)
- )
-)
-~~~
-
-Assuming this is part of a file named `tp.h` which defines the tracepoint
-provider and which is included by `tp.c`, here's a complete C application
-calling this tracepoint (multiple times):
-
-~~~ c
-#define TRACEPOINT_DEFINE
-#include "tp.h"
-
-int main(int argc, char* argv[])
-{
- int i;
-
- tracepoint(my_provider, my_first_tracepoint, 23, "Hello, World!");
-
- for (i = 0; i < argc; ++i) {
- tracepoint(my_provider, my_first_tracepoint, i, argv[i]);
- }
-
- return 0;
-}
-~~~
-
-For each tracepoint provider, `TRACEPOINT_DEFINE` must be defined into
-exactly one translation unit (C source file) of the user application,
-before including the tracepoint provider header file. In other words,
-for a given tracepoint provider, you cannot define `TRACEPOINT_DEFINE`,
-and then include its header file in two separate C source files of
-the same application. `TRACEPOINT_DEFINE` is discussed further in
-[Building/linking tracepoint providers and the user application](#doc-building-tracepoint-providers-and-user-application).
-
-As another example, remember this definition we wrote in a previous
-section (comments are stripped):
-
-~~~ c
-/* for struct stat */
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <unistd.h>
-
-TRACEPOINT_EVENT(
- my_provider,
- my_tracepoint,
- TP_ARGS(
- int, my_int_arg,
- char*, my_str_arg,
- struct stat*, st
- ),
- TP_FIELDS(
- ctf_integer(int, my_constant_field, 23 + 17)
- ctf_integer(int, my_int_arg_field, my_int_arg)
- ctf_integer(int, my_int_arg_field2, my_int_arg * my_int_arg)
- ctf_integer(int, sum4_field, my_str_arg[0] + my_str_arg[1] +
- my_str_arg[2] + my_str_arg[3])
- ctf_string(my_str_arg_field, my_str_arg)
- ctf_integer_hex(off_t, size_field, st->st_size)
- ctf_float(double, size_dbl_field, (double) st->st_size)
- ctf_sequence_text(char, half_my_str_arg_field, my_str_arg,
- size_t, strlen(my_str_arg) / 2)
- )
-)
-~~~
-
-Here's an example of calling it:
-
-~~~ c
-#define TRACEPOINT_DEFINE
-#include "tp.h"
-
-int main(void)
-{
- struct stat s;
-
- stat("/etc/fstab", &s);
-
- tracepoint(my_provider, my_tracepoint, 23, "Hello, World!", &s);
-
- return 0;
-}
-~~~
-
-When viewing the trace, assuming the file size of `/etc/fstab` is
-301 bytes, the event generated by the execution of this tracepoint
-should have the following fields, in this order:
-
-~~~ text
-my_constant_field 40
-my_int_arg_field 23
-my_int_arg_field2 529
-sum4_field 389
-my_str_arg_field "Hello, World!"
-size_field 0x12d
-size_dbl_field 301.0
-half_my_str_arg_field "Hello,"
-~~~
+++ /dev/null
----
-id: tracef
-since: 2.5
----
-
-`tracef()` is a small LTTng-UST API to avoid defining your own
-tracepoints and tracepoint providers. The signature of `tracef()` is
-the same as `printf()`'s.
-
-The `tracef()` utility function was developed to make user space tracing
-super simple, albeit with notable disadvantages compared to custom,
-full-fledged tracepoint providers:
-
- * All generated events have the same provider/event names, respectively
- `lttng_ust_tracef` and `event`.
- * There's no static type checking.
- * The only event field you actually get, named `msg`, is a string
- potentially containing the values you passed to the function
- using your own format. This also means that you cannot use filtering
- using a custom expression at runtime because there are no isolated
- fields.
- * Since `tracef()` uses C standard library's `vasprintf()` function
- in the background to format the strings at runtime, its
- expected performance is lower than using custom tracepoint providers
- with typed fields, which do not require a conversion to a string.
-
-Thus, `tracef()` is useful for quick prototyping and debugging, but
-should not be considered for any permanent/serious application
-instrumentation.
-
-To use `tracef()`, first include `<lttng/tracef.h>` in the C source file
-where you need to insert probes:
-
-~~~ c
-#include <lttng/tracef.h>
-~~~
-
-Use `tracef()` like you would use `printf()` in your source code, for
-example:
-
-~~~ c
- /* ... */
-
- tracef("my message, my integer: %d", my_integer);
-
- /* ... */
-~~~
-
-Link your application with `liblttng-ust`:
-
-<pre class="term">
-gcc -o app app.c <strong>-llttng-ust</strong>
-</pre>
-
-Execute the application as usual:
-
-<pre class="term">
-./app
-</pre>
-
-Voilà! Use the `lttng` command line tool to
-[control tracing](#doc-controlling-tracing). You can enable `tracef()`
-events like this:
-
-<pre class="term">
-lttng enable-event --userspace 'lttng_ust_tracef:*'
-</pre>
+++ /dev/null
----
-id: tracelog
-since: 2.7
----
-
-The `tracelog()` API is very similar to [`tracef()`](#doc-tracef). The
-only difference is that it accepts an additional log level parameter.
-
-The goal of `tracelog()` is to ease the migration from logging to
-tracing.
-
-Here's an example:
-
-~~~ c
-#include <lttng/tracelog.h>
-
-void my_function(int my_integer) {
- /* ... */
-
- tracelog(TRACE_INFO, "my message, my integer: %d", my_integer);
-
- /* ... */
-}
-~~~
-
-See [LTTng-UST library reference](#doc-liblttng-ust-tracepoint-loglevel)
-for the list of available log level names.
-
-Link your application with `liblttng-ust`:
-
-<pre class="term">
-gcc -o app app.c <strong>-llttng-ust</strong>
-</pre>
-
-Execute the application as usual:
-
-<pre class="term">
-./app
-</pre>
-
-The events produced by `tracelog()` calls are prefixed with
-`lttng_ust_tracelog:`. To enable `tracelog()` events matching a range
-of log levels, do:
-
-<pre class="term">
-lttng enable-event --userspace 'lttng_ust_tracelog:*' \
- --loglevel TRACE_INFO
-</pre>
-
-This enables all `tracelog()` events with a log level at least as important
-as `TRACE_INFO`.
-
-To enable `tracelog()` events matching a specific log level, do:
-
-<pre class="term">
-lttng enable-event --userspace 'lttng_ust_tracelog:*' \
- --loglevel-only TRACE_WARNING
-</pre>
-
-See [Enabling and disabling events](#doc-enabling-disabling-events) for
-more options.
+++ /dev/null
----
-id: tracepoint-provider
----
-
-Before jumping into defining tracepoints and inserting
-them into the application source code, you must understand what a
-_tracepoint provider_ is.
-
-For the sake of this guide, consider the following two files:
-
-`tp.h`:
-
-~~~ c
-#undef TRACEPOINT_PROVIDER
-#define TRACEPOINT_PROVIDER my_provider
-
-#undef TRACEPOINT_INCLUDE
-#define TRACEPOINT_INCLUDE "./tp.h"
-
-#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
-#define _TP_H
-
-#include <lttng/tracepoint.h>
-
-TRACEPOINT_EVENT(
- my_provider,
- my_first_tracepoint,
- TP_ARGS(
- int, my_integer_arg,
- char*, my_string_arg
- ),
- TP_FIELDS(
- ctf_string(my_string_field, my_string_arg)
- ctf_integer(int, my_integer_field, my_integer_arg)
- )
-)
-
-TRACEPOINT_EVENT(
- my_provider,
- my_other_tracepoint,
- TP_ARGS(
- int, my_int
- ),
- TP_FIELDS(
- ctf_integer(int, some_field, my_int)
- )
-)
-
-#endif /* _TP_H */
-
-#include <lttng/tracepoint-event.h>
-~~~
-
-`tp.c`:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-
-#include "tp.h"
-~~~
-
-The two files above are defining a _tracepoint provider_. A tracepoint
-provider is some sort of namespace for _tracepoint definitions_. Tracepoint
-definitions are written above with the `TRACEPOINT_EVENT()` macro, and allow
-eventual `tracepoint()` calls respecting their definitions to be inserted
-into the user application's C source code (we explore this in a
-later section).
-
-Many tracepoint definitions may be part of the same tracepoint provider
-and many tracepoint providers may coexist in a user space application. A
-tracepoint provider is packaged either:
-
- * directly into an existing user application's C source file
- * as an object file
- * as a static library
- * as a shared library
-
-The two files above, `tp.h` and `tp.c`, show a typical template for
-writing a tracepoint provider. LTTng-UST was designed so that two
-tracepoint providers should not be defined in the same header file.
-
-We will now go through the various parts of the above files and
-give them a meaning. As you may have noticed, the LTTng-UST API for
-C/C++ applications is some preprocessor sorcery. The LTTng-UST macros
-used in your application and those in the LTTng-UST headers are
-combined to produce actual source code needed to make tracing possible
-using LTTng.
-
-Let's start with the header file, `tp.h`. It begins with
-
-~~~ c
-#undef TRACEPOINT_PROVIDER
-#define TRACEPOINT_PROVIDER my_provider
-~~~
-
-`TRACEPOINT_PROVIDER` defines the name of the provider to which the
-following tracepoint definitions belong. It is used internally by
-LTTng-UST headers and _must_ be defined. Since `TRACEPOINT_PROVIDER`
-could have been defined by another header file also included by the same
-C source file, the best practice is to undefine it first.
-
-<div class="tip">
-<p><span class="t">Note:</span>Names in LTTng-UST follow the C
-<em>identifier</em> syntax (starting with a letter and containing either
-letters, numbers or underscores); they are <em>not</em> C strings
-(not surrounded by double quotes). This is because LTTng-UST macros
-use those identifier-like strings to create symbols (named types and
-variables).</p>
-</div>
-
-The tracepoint provider is a group of tracepoint definitions; its chosen
-name should reflect this. A hierarchy like Java packages is recommended,
-using underscores instead of dots, for example,
-`org_company_project_component`.
-
-Next is `TRACEPOINT_INCLUDE`:
-
-~~~ c
-#undef TRACEPOINT_INCLUDE
-#define TRACEPOINT_INCLUDE "./tp.h"
-~~~
-
-This little bit of instrospection is needed by LTTng-UST to include
-your header at various predefined places.
-
-Include guard follows:
-
-~~~ c
-#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
-#define _TP_H
-~~~
-
-Add these precompiler conditionals to ensure the tracepoint event
-generation can include this file more than once.
-
-The `TRACEPOINT_EVENT()` macro is defined in a LTTng-UST header file which
-must be included:
-
-~~~ c
-#include <lttng/tracepoint.h>
-~~~
-
-This also allows the application to use the `tracepoint()` macro.
-
-Next is a list of `TRACEPOINT_EVENT()` macro calls which create the
-actual tracepoint definitions. We skip this for the moment and
-come back to how to use `TRACEPOINT_EVENT()`
-[in a later section](#doc-defining-tracepoints). Just pay attention to
-the first argument: it's always the name of the tracepoint provider
-being defined in this header file.
-
-End of include guard:
-
-~~~ c
-#endif /* _TP_H */
-~~~
-
-Finally, include `<lttng/tracepoint-event.h>` to expand the macros:
-
-~~~ c
-#include <lttng/tracepoint-event.h>
-~~~
-
-That's it for `tp.h`. Of course, this is only a header file; it must be
-included in some C source file to actually use it. This is the job of
-`tp.c`:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-
-#include "tp.h"
-~~~
-
-When `TRACEPOINT_CREATE_PROBES` is defined, the macros used in `tp.h`,
-which is included just after, actually create the source code for
-LTTng-UST probes (global data structures and functions) out of your
-tracepoint definitions. How exactly this is done is out of this text's scope.
-`TRACEPOINT_CREATE_PROBES` is discussed further
-in [Building/linking tracepoint providers and the user application](#doc-building-tracepoint-providers-and-user-application).
-
-You could include other header files like `tp.h` here to create the probes
-of different tracepoint providers, for example:
-
-~~~ c
-#define TRACEPOINT_CREATE_PROBES
-
-#include "tp1.h"
-#include "tp2.h"
-~~~
-
-The rule is: probes of a given tracepoint provider
-must be created in exactly one source file. This source file could be one
-of your project's; it doesn't have to be on its own like `tp.c`, although
-[a later section](#doc-building-tracepoint-providers-and-user-application)
-shows that doing so allows packaging the tracepoint providers
-independently and keep them out of your application, also making it
-possible to reuse them between projects.
-
-The following sections explain how to define tracepoints, how to use the
-`tracepoint()` macro to instrument your user space C application and how
-to build/link tracepoint providers and your application with LTTng-UST
-support.
+++ /dev/null
----
-id: using-tracepoint-classes
----
-
-In LTTng-UST, a _tracepoint class_ is a class of tracepoints sharing the
-same field types and names. A _tracepoint instance_ is one instance of
-such a declared tracepoint class, with its own event name and tracepoint
-provider name.
-
-What is documented in [Defining tracepoints](#doc-defining-tracepoints)
-is actually how to declare a _tracepoint class_ and define a
-_tracepoint instance_ at the same time. Without revealing the internals
-of LTTng-UST too much, it has to be noted that one serialization
-function is created for each tracepoint class. A serialization
-function is responsible for serializing the fields of a tracepoint
-into a sub-buffer when tracing. For various performance reasons, when
-your situation requires multiple tracepoints with different names, but
-with the same fields layout, the best practice is to manually create
-a tracepoint class and instantiate as many tracepoint instances as
-needed. One positive effect of such a design, amongst other advantages,
-is that all tracepoint instances of the same tracepoint class
-reuse the same serialization function, thus reducing cache pollution.
-
-As an example, here are three tracepoint definitions as we know them:
-
-~~~ c
-TRACEPOINT_EVENT(
- my_app,
- get_account,
- TP_ARGS(
- int, userid,
- size_t, len
- ),
- TP_FIELDS(
- ctf_integer(int, userid, userid)
- ctf_integer(size_t, len, len)
- )
-)
-
-TRACEPOINT_EVENT(
- my_app,
- get_settings,
- TP_ARGS(
- int, userid,
- size_t, len
- ),
- TP_FIELDS(
- ctf_integer(int, userid, userid)
- ctf_integer(size_t, len, len)
- )
-)
-
-TRACEPOINT_EVENT(
- my_app,
- get_transaction,
- TP_ARGS(
- int, userid,
- size_t, len
- ),
- TP_FIELDS(
- ctf_integer(int, userid, userid)
- ctf_integer(size_t, len, len)
- )
-)
-~~~
-
-In this case, three tracepoint classes are created, with one tracepoint
-instance for each of them: `get_account`, `get_settings` and
-`get_transaction`. However, they all share the same field names and
-types. Declaring one tracepoint class and three tracepoint instances of
-the latter is a better design choice:
-
-~~~ c
-/* the tracepoint class */
-TRACEPOINT_EVENT_CLASS(
- /* tracepoint provider name */
- my_app,
-
- /* tracepoint class name */
- my_class,
-
- /* arguments */
- TP_ARGS(
- int, userid,
- size_t, len
- ),
-
- /* fields */
- TP_FIELDS(
- ctf_integer(int, userid, userid)
- ctf_integer(size_t, len, len)
- )
-)
-
-/* the tracepoint instances */
-TRACEPOINT_EVENT_INSTANCE(
- /* tracepoint provider name */
- my_app,
-
- /* tracepoint class name */
- my_class,
-
- /* tracepoint/event name */
- get_account,
-
- /* arguments */
- TP_ARGS(
- int, userid,
- size_t, len
- )
-)
-TRACEPOINT_EVENT_INSTANCE(
- my_app,
- my_class,
- get_settings,
- TP_ARGS(
- int, userid,
- size_t, len
- )
-)
-TRACEPOINT_EVENT_INSTANCE(
- my_app,
- my_class,
- get_transaction,
- TP_ARGS(
- int, userid,
- size_t, len
- )
-)
-~~~
-
-Of course, all those names and `TP_ARGS()` invocations are redundant,
-but some C preprocessor magic can solve this:
-
-~~~ c
-#define MY_TRACEPOINT_ARGS \
- TP_ARGS( \
- int, userid, \
- size_t, len \
- )
-
-TRACEPOINT_EVENT_CLASS(
- my_app,
- my_class,
- MY_TRACEPOINT_ARGS,
- TP_FIELDS(
- ctf_integer(int, userid, userid)
- ctf_integer(size_t, len, len)
- )
-)
-
-#define MY_APP_TRACEPOINT_INSTANCE(name) \
- TRACEPOINT_EVENT_INSTANCE( \
- my_app, \
- my_class, \
- name, \
- MY_TRACEPOINT_ARGS \
- )
-
-MY_APP_TRACEPOINT_INSTANCE(get_account)
-MY_APP_TRACEPOINT_INSTANCE(get_settings)
-MY_APP_TRACEPOINT_INSTANCE(get_transaction)
-~~~
+++ /dev/null
----
-id: cxx-application
----
-
-Because of C++'s cross-compatibility with the C language, C++
-applications can be readily instrumented with the LTTng-UST C API.
-
-Follow the [C application](#doc-c-application) user guide above. It
-should be noted that, in this case, tracepoint providers should have
-the typical `.cpp`, `.cxx` or `.cc` extension and be built with `g++`
-instead of `gcc`. This is the easiest way of avoiding linking errors
-due to symbol name mangling incompatibilities between both languages.
+++ /dev/null
----
-id: instrumenting-linux-kernel-tracing
----
-
-The [Controlling tracing](#doc-controlling-tracing) section explains
-how to use the `lttng` tool to create and control tracing sessions.
-Although the `lttng` tool loads the appropriate _known_ LTTng kernel
-modules when needed (by launching `root`'s session daemon), it won't
-load your custom `lttng-probe-hello` module by default. You need to
-manually start an LTTng session daemon as `root` and use the
-`--extra-kmod-probes` option to append your custom probe module to the
-default list:
-
-<pre class="term">
-sudo pkill -u root lttng-sessiond
-sudo lttng-sessiond --extra-kmod-probes=hello
-</pre>
-
-The first command makes sure any existing instance is killed. If
-you're not interested in using the default probes, or if you only
-want to use a few of them, you could use `--kmod-probes` instead,
-which specifies an absolute list:
-
-<pre class="term">
-sudo lttng-sessiond --kmod-probes=hello,ext4,net,block,signal,sched
-</pre>
-
-Confirm the custom probe module is loaded:
-
-<pre class="term">
-lsmod | grep lttng_probe_hello
-</pre>
-
-The `hello_world` event should appear in the list when doing
-
-<pre class="term">
-lttng list --kernel | grep hello
-</pre>
-
-You may now create an LTTng tracing session, enable the `hello_world`
-kernel event (and others if you wish) and start tracing:
-
-<pre class="term">
-sudo lttng create my-session
-sudo lttng enable-event --kernel hello_world
-sudo lttng start
-</pre>
-
-Plug a few USB devices, then stop tracing and inspect the trace (if
-<a href="http://diamon.org/babeltrace" class="ext">Babeltrace</a>
-is installed):
-
-<pre class="term">
-sudo lttng stop
-sudo lttng view
-</pre>
-
-Here's a sample output:
-
-~~~ text
-[15:30:34.835895035] (+?.?????????) hostname hello_world: { cpu_id = 1 }, { my_int = 8, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
-[15:30:42.262781421] (+7.426886386) hostname hello_world: { cpu_id = 1 }, { my_int = 9, char0 = 80, char1 = 97, product = "Patriot Memory" }
-[15:30:48.175621778] (+5.912840357) hostname hello_world: { cpu_id = 1 }, { my_int = 10, char0 = 68, char1 = 97, product = "DataTraveler 2.0" }
-~~~
-
-Two USB flash drives were used for this test.
-
-You may change your LTTng custom probe, rebuild it and reload it at
-any time when not tracing. Make sure you remove the old module
-(either by killing the root LTTng session daemon which loaded the
-module in the first place, or by using `modprobe --remove` directly)
-before loading the updated one.
+++ /dev/null
----
-id: instrumenting-linux-kernel-itself
----
-
-This section explains strictly how to add custom LTTng
-instrumentation to the Linux kernel. It does not explain how the
-macros actually work and the internal mechanics of the tracer.
-
-You should have a Linux kernel source code tree to work with.
-Throughout this section, all file paths are relative to the root of
-this tree unless otherwise stated.
-
-You need a copy of the LTTng-modules Git repository:
-
-<pre class="term">
-git clone git://git.lttng.org/lttng-modules.git
-</pre>
-
-The steps to add custom LTTng instrumentation to a Linux kernel
-involves defining and using the mainline `TRACE_EVENT()` tracepoints
-first, then writing and using the LTTng adaptation layer.
+++ /dev/null
----
-id: lttng-adaptation-layer
----
-
-The steps to write the LTTng adaptation layer are, in your
-LTTng-modules copy's source code tree:
-
- 1. In `instrumentation/events/lttng-module`,
- add a header <code><em>subsys</em>.h</code> for your custom
- subsystem <code><em>subsys</em></code> and write your
- tracepoint definitions using LTTng-modules macros in it.
- Those macros look like the mainline kernel equivalents,
- but they present subtle, yet important differences.
- 2. In `probes`, create the C source file of the LTTng probe kernel
- module for your subsystem. It should be named
- <code>lttng-probe-<em>subsys</em>.c</code>.
- 3. Edit `probes/Makefile` so that the LTTng-modules project
- builds your custom LTTng probe kernel module.
- 4. Build and install LTTng kernel modules.
-
-Following our `hello_world` event example, here's the content of
-`instrumentation/events/lttng-module/hello.h`:
-
-~~~ c
-#undef TRACE_SYSTEM
-#define TRACE_SYSTEM hello
-
-#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
-#define _TRACE_HELLO_H
-
-#include "../../../probes/lttng-tracepoint-event.h"
-#include <linux/tracepoint.h>
-
-LTTNG_TRACEPOINT_EVENT(
- /* format identical to mainline version for those */
- hello_world,
- TP_PROTO(int foo, const char* bar),
- TP_ARGS(foo, bar),
-
- /* possible differences */
- TP_STRUCT__entry(
- __field(int, my_int)
- __field(char, char0)
- __field(char, char1)
- __string(product, bar)
- ),
-
- /* notice the use of tp_assign()/tp_strcpy() and no semicolons */
- TP_fast_assign(
- tp_assign(my_int, foo)
- tp_assign(char0, bar[0])
- tp_assign(char1, bar[1])
- tp_strcpy(product, bar)
- ),
-
- /* This one is actually not used by LTTng either, but must be
- * present for the moment.
- */
- TP_printk("", 0)
-
-/* no semicolon after this either */
-)
-
-#endif
-
-/* other difference: do NOT include <trace/define_trace.h> */
-#include "../../../probes/define_trace.h"
-~~~
-
-Some possible entries for `TP_STRUCT__entry()` and `TP_fast_assign()`,
-in the case of LTTng-modules, are shown in the
-[LTTng-modules reference](#doc-lttng-modules-ref) section.
-
-You may also be interested in using the
-[`LTTNG_TRACEPOINT_EVENT_CODE()` macro](#doc-lttng-tracepoint-event-code),
-instead of using `LTTNG_TRACEPOINT_EVENT()`, which allows custom local
-variables and C code to be executed before the event fields are recorded.
-
-The best way to learn how to use the above macros is to inspect
-existing LTTng tracepoint definitions in `instrumentation/events/lttng-module`
-header files. Compare them with the Linux kernel mainline versions
-in `include/trace/events`.
-
-The next step is writing the LTTng probe kernel module C source file.
-This one is named <code>lttng-probe-<em>subsys</em>.c</code>
-in `probes`. You may always use the following template:
-
-~~~ c
-#include <linux/module.h>
-#include "../lttng-tracer.h"
-
-/* Build time verification of mismatch between mainline TRACE_EVENT()
- * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
- */
-#include <trace/events/hello.h>
-
-/* create LTTng tracepoint probes */
-#define LTTNG_PACKAGE_BUILD
-#define CREATE_TRACE_POINTS
-#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module
-
-#include "../instrumentation/events/lttng-module/hello.h"
-
-MODULE_LICENSE("GPL and additional rights");
-MODULE_AUTHOR("Your name <your-email>");
-MODULE_DESCRIPTION("LTTng hello probes");
-MODULE_VERSION(__stringify(LTTNG_MODULES_MAJOR_VERSION) "."
- __stringify(LTTNG_MODULES_MINOR_VERSION) "."
- __stringify(LTTNG_MODULES_PATCHLEVEL_VERSION)
- LTTNG_MODULES_EXTRAVERSION);
-~~~
-
-Just replace `hello` with your subsystem name. In this example,
-`<trace/events/hello.h>`, which is the original mainline tracepoint
-definition header, is included for verification purposes: the
-LTTng-modules build system is able to emit an error at build time when
-the arguments of the mainline `TRACE_EVENT()` definitions do not match
-the ones of the LTTng-modules adaptation layer
-(`LTTNG_TRACEPOINT_EVENT()`).
-
-Edit `probes/Makefile` and add your new kernel module object
-next to existing ones:
-
-~~~ makefile
-# ...
-
-obj-m += lttng-probe-module.o
-obj-m += lttng-probe-power.o
-
-obj-m += lttng-probe-hello.o
-
-# ...
-~~~
-
-Time to build! Point to your custom Linux kernel source tree using
-the `KERNELDIR` variable:
-
-<pre class="term">
-make <strong>KERNELDIR=/path/to/custom/linux</strong>
-</pre>
-
-Finally, install modules:
-
-<pre class="term">
-sudo make modules_install
-</pre>
+++ /dev/null
----
-id: lttng-tracepoint-event-code
-since: 2.7
----
-
-Although it is recommended to always use the
-[`LTTNG_TRACEPOINT_EVENT()`](#doc-lttng-adaptation-layer)
-macro to describe the arguments and fields of an LTTng tracepoint when
-possible, sometimes a more complex process is needed to access the data
-to be recorded as tracepoint fields. In other words, local variables
-and multiple C statements are required instead of simple argument-based
-expressions passed to the
-[`ctf_*()` macros of `TP_FIELDS()`](#doc-lttng-modules-tp-fields).
-
-The `LTTNG_TRACEPOINT_EVENT_CODE()` macro can be used instead of
-`LTTNG_TRACEPOINT_EVENT()` to declare custom local variables and
-define a block of C code to be executed before the fields are
-recorded. The structure of this macro is:
-
-~~~ c
-LTTNG_TRACEPOINT_EVENT_CODE(
- /* format identical to LTTNG_TRACEPOINT_EVENT() version for those */
- hello_world,
- TP_PROTO(int foo, const char *bar),
- TP_ARGS(foo, bar),
-
- /* declarations of custom local variables */
- TP_locvar(
- int a = 0;
- unsigned long b = 0;
- const char *name = "(undefined)";
- struct my_struct *my_struct;
- ),
-
- /*
- * Custom code using which use both tracepoint arguments
- * (in TP_ARGS()) and local variables (in TP_locvar()).
- *
- * Local variables are actually members of a structure pointed
- * to by the special variable tp_locvar.
- */
- TP_code(
- if (foo) {
- tp_locvar->a = foo + 17;
- tp_locvar->my_struct = get_my_struct_at(tp_locvar->a);
- tp_locvar->b = my_struct_compute_b(tp_locvar->my_struct);
- tp_locvar->name = my_struct_get_name(tp_locvar->my_struct);
- put_my_struct(tp_locvar->my_struct);
-
- if (tp_locvar->b) {
- tp_locvar->a = 1;
- }
- }
- ),
-
- /*
- * Format identical to LTTNG_TRACEPOINT_EVENT() version for this,
- * except that tp_locvar members can be used in the argument
- * expression parameters of the ctf_*() macros.
- */
- TP_FIELDS(
- ctf_integer(unsigned long, my_struct_b, tp_locvar->b)
- ctf_integer(int, my_struct_a, tp_locvar->a)
- ctf_string(bar_field, bar)
- ctf_string(my_struct_name, tp_locvar->name)
- )
-)
-~~~
-
-Make sure that the C code defined in `TP_code()` has no side effects
-when executed. In particular, the code should not allocate memory or get
-resources without deallocating this memory or putting those resources
-afterwards.
+++ /dev/null
----
-id: mainline-trace-event
----
-
-The first step is to define tracepoints using the mainline Linux
-`TRACE_EVENT()` macro and insert tracepoints where you want them.
-Your tracepoint definitions reside in a header file in
-`include/trace/events`. If you're adding tracepoints to an existing
-subsystem, edit its appropriate header file.
-
-As an example, the following header file (let's call it
-`include/trace/events/hello.h`) defines one tracepoint using
-`TRACE_EVENT()`:
-
-~~~ c
-/* subsystem name is "hello" */
-#undef TRACE_SYSTEM
-#define TRACE_SYSTEM hello
-
-#if !defined(_TRACE_HELLO_H) || defined(TRACE_HEADER_MULTI_READ)
-#define _TRACE_HELLO_H
-
-#include <linux/tracepoint.h>
-
-TRACE_EVENT(
- /* "hello" is the subsystem name, "world" is the event name */
- hello_world,
-
- /* tracepoint function prototype */
- TP_PROTO(int foo, const char* bar),
-
- /* arguments for this tracepoint */
- TP_ARGS(foo, bar),
-
- /* LTTng doesn't need those */
- TP_STRUCT__entry(),
- TP_fast_assign(),
- TP_printk("", 0)
-);
-
-#endif
-
-/* this part must be outside protection */
-#include <trace/define_trace.h>
-~~~
-
-Notice that we don't use any of the last three arguments: they
-are left empty here because LTTng doesn't need them. You would only fill
-`TP_STRUCT__entry()`, `TP_fast_assign()` and `TP_printk()` if you were
-to also use this tracepoint for ftrace/perf.
-
-Once this is done, you may place calls to `trace_hello_world()`
-wherever you want in the Linux source code. As an example, let us place
-such a tracepoint in the `usb_probe_device()` static function
-(`drivers/usb/core/driver.c`):
-
-~~~ c
-/* called from driver core with dev locked */
-static int usb_probe_device(struct device *dev)
-{
- struct usb_device_driver *udriver = to_usb_device_driver(dev->driver);
- struct usb_device *udev = to_usb_device(dev);
- int error = 0;
-
- trace_hello_world(udev->devnum, udev->product);
-
- /* ... */
-}
-~~~
-
-This tracepoint should fire every time a USB device is plugged in.
-
-At the top of `driver.c`, we need to include our actual tracepoint
-definition and, in this case (one place per subsystem), define
-`CREATE_TRACE_POINTS`, which creates our tracepoint:
-
-~~~ c
-/* ... */
-
-#include "usb.h"
-
-#define CREATE_TRACE_POINTS
-#include <trace/events/hello.h>
-
-/* ... */
-~~~
-
-Build your custom Linux kernel. In order to use LTTng, make sure the
-following kernel configuration options are enabled:
-
- * `CONFIG_MODULES` (loadable module support)
- * `CONFIG_KALLSYMS` (load all symbols for debugging/kksymoops)
- * `CONFIG_HIGH_RES_TIMERS` (high resolution timer support)
- * `CONFIG_TRACEPOINTS` (kernel tracepoint instrumentation)
-
-Boot the custom kernel. The directory
-`/sys/kernel/debug/tracing/events/hello` should exist if everything
-went right, with a `hello_world` subdirectory.
+++ /dev/null
----
-id: instrumenting-out-of-tree-linux-kernel
----
-
-Instrumenting a custom Linux kernel module for LTTng follows the exact
-same steps as
-[adding instrumentation to the Linux kernel itself](#doc-instrumenting-linux-kernel-itself),
-the only difference being that your mainline tracepoint definition
-header doesn't reside in the mainline source tree, but in your
-kernel module source tree.
-
-The only reference to this mainline header is in the LTTng custom
-probe's source code (`probes/lttng-probe-hello.c` in our example), for
-build time verification:
-
-~~~ c
-/* ... */
-
-/* Build time verification of mismatch between mainline TRACE_EVENT()
- * arguments and LTTng adaptation layer LTTNG_TRACEPOINT_EVENT() arguments.
- */
-#include <trace/events/hello.h>
-
-/* ... */
-~~~
-
-The preferred, flexible way to include your module's mainline
-tracepoint definition header is to put it in a specific directory
-relative to your module's root (`tracepoints`, for example) and include it
-relative to your module's root directory in the LTTng custom probe's
-source:
-
-~~~ c
-#include <tracepoints/hello.h>
-~~~
-
-You may then build LTTng-modules by adding your module's root
-directory as an include path to the extra C flags:
-
-<pre class="term">
-make <strong>ccflags-y=-I/path/to/kernel/module</strong> KERNELDIR=/path/to/custom/linux
-</pre>
-
-Using `ccflags-y` allows you to move your kernel module to another
-directory and rebuild the LTTng-modules project with no change to
-source files.
+++ /dev/null
----
-id: instrumenting-linux-kernel
----
-
-The Linux kernel can be instrumented for LTTng tracing, either its core
-source code or a kernel module. It has to be noted that Linux is
-readily traceable using LTTng since many parts of its source code are
-already instrumented: this is the job of the upstream
-<a href="http://git.lttng.org/?p=lttng-modules.git" class="ext">LTTng-modules</a>
-package. This section presents how to add LTTng instrumentation where it
-does not currently exist and how to instrument custom kernel modules.
-
-All LTTng instrumentation in the Linux kernel is based on an existing
-infrastructure which bears the name of its main macro, `TRACE_EVENT()`.
-This macro is used to define tracepoints,
-each tracepoint having a name, usually with the
-<code><em>subsys</em>_<em>name</em></code> format,
-<code><em>subsys</em></code> being the subsystem name and
-<code><em>name</em></code> the specific event name.
-
-Tracepoints defined with `TRACE_EVENT()` may be inserted anywhere in
-the Linux kernel source code, after what callbacks, called _probes_,
-may be registered to execute some action when a tracepoint is
-executed. This mechanism is directly used by ftrace and perf,
-but cannot be used as is by LTTng: an adaptation layer is added to
-satisfy LTTng's specific needs.
-
-With that in mind, this documentation does not cover the `TRACE_EVENT()`
-format and how to use it, but it is mandatory to understand it and use
-it to instrument Linux for LTTng. A series of
-<abbr title="Linux Weekly News">LWN</abbr> articles explain
-`TRACE_EVENT()` in details:
-<a href="http://lwn.net/Articles/379903/" class="ext">part 1</a>,
-<a href="http://lwn.net/Articles/381064/" class="ext">part 2</a>, and
-<a href="http://lwn.net/Articles/383362/" class="ext">part 3</a>.
-Once you master `TRACE_EVENT()` enough for your use case, continue
-reading this section so that you can add the LTTng adaptation layer of
-instrumentation.
-
-This section first discusses the general method of instrumenting the
-Linux kernel for LTTng. This method is then reused for the specific
-case of instrumenting a kernel module.
+++ /dev/null
----
-id: instrumenting
----
-
-There are many examples of tracing and monitoring in our everyday life.
-You have access to real-time and historical weather reports and forecasts
-thanks to weather stations installed around the country. You know your
-possibly hospitalized friends' and family's hearts are safe thanks to
-electrocardiography. You make sure not to drive your car too fast
-and have enough fuel to reach your destination thanks to gauges visible
-on your dashboard.
-
-All the previous examples have something in common: they rely on
-**probes**. Without electrodes attached to the surface of a body's
-skin, cardiac monitoring would be futile.
-
-LTTng, as a tracer, is no different from the real life examples above.
-If you're about to trace a software system or, put in other words, record its
-history of execution, you better have probes in the subject you're
-tracing: the actual software. Various ways were developed to do this.
-The most straightforward one is to manually place probes, called
-_tracepoints_, in the software's source code. The Linux kernel tracing
-domain also allows probes added dynamically.
-
-If you're only interested in tracing the Linux kernel, it may very well
-be that your tracing needs are already appropriately covered by LTTng's
-built-in Linux kernel tracepoints and other probes. Or you may be in
-possession of a user space application which has already been
-instrumented. In such cases, the work resides entirely in the design
-and execution of tracing sessions, allowing you to jump to
-[Controlling tracing](#doc-controlling-tracing) right now.
-
-This section focuses on the following use cases of instrumentation:
-
- * [C](#doc-c-application) and [C++](#doc-cxx-application) applications
- * [prebuilt user space tracing helpers](#doc-prebuilt-ust-helpers)
- * [Java application](#doc-java-application)
- * [Linux kernel](#doc-instrumenting-linux-kernel) module or the
- kernel itself
- * the [`/proc/lttng-logger` <abbr title="Application Binary Interface">ABI</abbr>](#doc-proc-lttng-logger-abi)
-
-Some [advanced techniques](#doc-advanced-instrumenting-techniques) are
-also presented at the very end.
+++ /dev/null
----
-id: java-application
-since: 2.4
----
-
-LTTng-UST provides a _logging_ back-end for Java applications using
-either
-<a href="http://docs.oracle.com/javase/7/docs/api/java/util/logging/Logger.html" class="ext"><code>java.util.logging</code></a>
-(JUL) or
-<a href="http://logging.apache.org/log4j/1.2/" class="ext">Apache log4j 1.2</a>.
-This back-end is called the _LTTng-UST Java agent_, and it is responsible
-for the communications with an LTTng session daemon.
-
-From the user's point of view, once the LTTng-UST Java agent has been
-initialized, JUL and log4j loggers may be created and used as usual.
-The agent adds its own handler to the _root logger_, so that all
-loggers may generate LTTng events with no effort.
-
-Common JUL/log4j features are supported using the `lttng` tool
-(see [Controlling tracing](#doc-controlling-tracing)):
-
- * listing all logger names
- * enabling/disabling events per logger name
- * JUL/log4j log levels
+++ /dev/null
----
-id: jul
-since: 2.4
----
-
-Here's an example of tracing a Java application which is using
-**`java.util.logging`**:
-
-~~~ java
-import java.util.logging.Logger;
-import org.lttng.ust.agent.LTTngAgent;
-
-public class Test
-{
- private static final int answer = 42;
-
- public static void main(String[] argv) throws Exception
- {
- // create a logger
- Logger logger = Logger.getLogger("jello");
-
- // call this as soon as possible (before logging)
- LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
-
- // log at will!
- logger.info("some info");
- logger.warning("some warning");
- Thread.sleep(500);
- logger.finer("finer information; the answer is " + answer);
- Thread.sleep(123);
- logger.severe("error!");
-
- // not mandatory, but cleaner
- lttngAgent.dispose();
- }
-}
-~~~
-
-The LTTng-UST Java agent is packaged in a JAR file named
-`liblttng-ust-agent.jar` It is typically located in
-`/usr/lib/lttng/java`. To compile the snippet above
-(saved as `Test.java`), do:
-
-<pre class="term">
-javac -cp /usr/lib/lttng/java/liblttng-ust-agent.jar Test.java
-</pre>
-
-You can run the resulting compiled class like this:
-
-<pre class="term">
-java -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:. Test
-</pre>
-
-<div class="tip">
-<p>
- <span class="t">Note:</span><a href="http://openjdk.java.net/" class="ext">OpenJDK</a> 7
- is used for development and continuous integration, thus this
- version is directly supported. However, the LTTng-UST Java agent has
- also been tested with OpenJDK 6.
-</p>
-</div>
+++ /dev/null
----
-id: log4j
-since: 2.6
----
-
-LTTng features an Apache log4j 1.2 agent, which means your existing
-Java applications using log4j 1.2 for logging can record events to
-LTTng traces with just a minor source code modification.
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>This version of LTTng does not
- support Log4j 2.
-</p>
-</div>
-
-Here's an example:
-
-~~~ java
-import org.apache.log4j.Logger;
-import org.apache.log4j.BasicConfigurator;
-import org.lttng.ust.agent.LTTngAgent;
-
-public class Test
-{
- private static final int answer = 42;
-
- public static void main(String[] argv) throws Exception
- {
- // create and configure a logger
- Logger logger = Logger.getLogger(Test.class);
- BasicConfigurator.configure();
-
- // call this as soon as possible (before logging)
- LTTngAgent lttngAgent = LTTngAgent.getLTTngAgent();
-
- // log at will!
- logger.info("some info");
- logger.warn("some warning");
- Thread.sleep(500);
- logger.debug("debug information; the answer is " + answer);
- Thread.sleep(123);
- logger.error("error!");
- logger.fatal("fatal error!");
-
- // not mandatory, but cleaner
- lttngAgent.dispose();
- }
-}
-~~~
-
-To compile the snippet above, do:
-
-<pre class="term">
-javac -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:$LOG4JCP Test.java
-</pre>
-
-where `$LOG4JCP` is the log4j 1.2 JAR file path.
-
-You can run the resulting compiled class like this:
-
-<pre class="term">
-java -cp /usr/lib/lttng/java/liblttng-ust-agent.jar:$LOG4JCP:. Test
-</pre>
+++ /dev/null
----
-id: prebuilt-ust-helpers
----
-
-The LTTng-UST package provides a few helpers that one may find
-useful in some situations. They all work the same way: you must
-preload the appropriate shared object before running the user
-application (using the `LD_PRELOAD` environment variable).
-
-The shared objects are normally found in `/usr/lib`.
-
-The current installed helpers are:
-
- * `liblttng-ust-libc-wrapper.so` and
- `liblttng-ust-pthread-wrapper.so`:
- [C standard library and POSIX threads tracing](#doc-liblttng-ust-libc-pthread-wrapper)
- * `liblttng-ust-cyg-profile.so` and
- `liblttng-ust-cyg-profile-fast.so`:
- [function tracing](#doc-liblttng-ust-cyg-profile)
- * `liblttng-ust-dl.so`:
- [dynamic linker tracing](#doc-liblttng-ust-dl)
-
-The following subsections document what helpers instrument exactly
-and how to use them.
+++ /dev/null
----
-id: liblttng-ust-cyg-profile
----
-
-Function tracing is the recording of which functions are entered and
-left during the execution of an application. Like with any LTTng event,
-the precise time at which this happens is also kept.
-
-GCC and clang have an option named
-<a href="https://gcc.gnu.org/onlinedocs/gcc-4.9.1/gcc/Code-Gen-Options.html" class="ext"><code>-finstrument-functions</code></a>
-which generates instrumentation calls for entry and exit to functions.
-The LTTng-UST function tracing helpers, `liblttng-ust-cyg-profile.so`
-and `liblttng-ust-cyg-profile-fast.so`, take advantage of this feature
-to add instrumentation to the two generated functions (which contain
-`cyg_profile` in their names, hence the shared object's name).
-
-In order to use LTTng-UST function tracing, the translation units to
-instrument must be built using the `-finstrument-functions` compiler
-flag.
-
-LTTng-UST function tracing comes in two flavors, each providing
-different trade-offs: `liblttng-ust-cyg-profile-fast.so` and
-`liblttng-ust-cyg-profile.so`.
-
-**`liblttng-ust-cyg-profile-fast.so`** is a lightweight variant that
-should only be used where it can be _guaranteed_ that the complete event
-stream is recorded without any missing events. Any kind of duplicate
-information is left out. This version registers the following
-tracepoints:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th><abbr title="Tracepoint">TP</abbr> provider name</th>
- <th><abbr title="Tracepoint">TP</abbr> name</th>
- <th>Description/fields</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td rowspan="2">
- <code class="no-bg">lttng_ust_cyg_profile_fast</code>
- </td>
- <td>
- <code class="no-bg">func_entry</code>
- </td>
- <td>
- <p>Function entry</p>
-
- <ul>
- <li>
- <code class="arg">addr</code> address of the
- called function
- </li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">func_exit</code>
- </td>
- <td>
- <p>Function exit</p>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-Assuming no event is lost, having only the function addresses on entry
-is enough for creating a call graph (remember that a recorded event
-always contains the ID of the CPU that generated it). A tool like
-<a href="https://sourceware.org/binutils/docs/binutils/addr2line.html" class="ext"><code>addr2line</code></a>
-may be used to convert function addresses back to source files names
-and line numbers.
-
-The other helper,
-**`liblttng-ust-cyg-profile.so`**,
-is a more robust variant which also works for use cases where
-events might get discarded or not recorded from application startup.
-In these cases, the trace analyzer needs extra information to be
-able to reconstruct the program flow. This version registers the
-following tracepoints:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th><abbr title="Tracepoint">TP</abbr> provider name</th>
- <th><abbr title="Tracepoint">TP</abbr> name</th>
- <th>Description/fields</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td rowspan="2">
- <code class="no-bg">lttng_ust_cyg_profile</code>
- </td>
- <td>
- <code class="no-bg">func_entry</code>
- </td>
- <td>
- <p>Function entry</p>
-
- <ul>
- <li>
- <code class="arg">addr</code> address of the
- called function
- </li>
- <li>
- <code class="arg">call_site</code> call site
- address
- </li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">func_exit</code>
- </td>
- <td>
- <p>Function exit</p>
-
- <ul>
- <li>
- <code class="arg">addr</code> address of the
- called function
- </li>
- <li>
- <code class="arg">call_site</code> call site
- address
- </li>
- </ul>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-To use one or the other variant with any user application, assuming at
-least one translation unit of the latter is compiled with the
-`-finstrument-functions` option, do:
-
-<pre class="term">
-LD_PRELOAD=liblttng-ust-cyg-profile-fast.so my-app
-</pre>
-
-or
-
-<pre class="term">
-LD_PRELOAD=liblttng-ust-cyg-profile.so my-app
-</pre>
-
-It might be necessary to limit the number of source files where
-`-finstrument-functions` is used to prevent excessive amount of trace
-data to be generated at runtime.
-
-<div class="tip">
-<p>
- <span class="t">Tip:</span> When using GCC, at least, you may use
- the
- <code>-finstrument-functions-exclude-function-list</code>
- option to avoid instrumenting entries and exits of specific
- symbol names.
-</p>
-</div>
-
-All events generated from LTTng-UST function tracing are provided on
-log level `TRACE_DEBUG_FUNCTION`, which is useful to easily enable
-function tracing events in your tracing session using the
-`--loglevel-only` option of `lttng enable-event`
-(see [Controlling tracing](#doc-controlling-tracing)).
+++ /dev/null
----
-id: liblttng-ust-dl
-since: 2.4
----
-
-This LTTng-UST helper causes all calls to `dlopen()` and `dlclose()`
-in the target application to be traced with LTTng.
-
-The helper's shared object, `liblttng-ust-dl.so`, registers the
-following tracepoints when preloaded:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th><abbr title="Tracepoint">TP</abbr> provider name</th>
- <th><abbr title="Tracepoint">TP</abbr> name</th>
- <th>Description/fields</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td rowspan="2">
- <code class="no-bg">lttng_ust_dl</code>
- </td>
- <td>
- <code class="no-bg">dlopen</code>
- </td>
- <td>
- <p><code>dlopen()</code> call</p>
-
- <ul>
- <li>
- <code class="arg">baddr</code> memory
- base address
- (where the dynamic linker placed the shared
- object)
- </li>
- <li>
- <code class="arg">sopath</code> file system
- path to the loaded shared object
- </li>
- <li>
- <code class="arg">size</code> file size
- of the the loaded shared object
- </li>
- <li>
- <code class="arg">mtime</code> last
- modification time (seconds since Epoch time)
- of the loaded shared object
- </li>
- </ul>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">dlclose</code>
- </td>
- <td>
- <p><code>dlclose()</code> call</p>
-
- <ul>
- <li>
- <code class="arg">baddr</code> memory
- base address
- </li>
- </ul>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-To use this LTTng-UST helper with any user application, independently of
-how the latter is built, do:
-
-<pre class="term">
-LD_PRELOAD=liblttng-ust-dl.so my-app
-</pre>
-
-Of course, like any other tracepoint, the ones above need to be enabled
-in order for LTTng-UST to generate events. This is done using the
-`lttng` command line tool
-(see [Controlling tracing](#doc-controlling-tracing)).
+++ /dev/null
----
-id: liblttng-ust-libc-pthread-wrapper
-since: 2.3
----
-
-`liblttng-ust-libc-wrapper.so` and `liblttng-ust-pthread-wrapper.so`
-can add instrumentation to respectively some C standard library and
-POSIX threads functions.
-
-The following functions are traceable by `liblttng-ust-libc-wrapper.so`:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th><abbr title="Tracepoint">TP</abbr> provider name</th>
- <th><abbr title="Tracepoint">TP</abbr> name</th>
- <th>Instrumented function</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td rowspan="6">
- <code class="no-bg">lttng_ust_libc</code>
- </td>
- <td>
- <code class="no-bg">malloc</code>
- </td>
- <td>
- <code class="no-bg">malloc()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">calloc</code>
- </td>
- <td>
- <code class="no-bg">calloc()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">realloc</code>
- </td>
- <td>
- <code class="no-bg">realloc()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">free</code>
- </td>
- <td>
- <code class="no-bg">free()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">memalign</code>
- </td>
- <td>
- <code class="no-bg">memalign()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">posix_memalign</code>
- </td>
- <td>
- <code class="no-bg">posix_memalign()</code>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-The following functions are traceable by
-`liblttng-ust-pthread-wrapper.so`:
-
-<div class="table">
-<table class="func-desc">
- <thead>
- <tr>
- <th><abbr title="Tracepoint">TP</abbr> provider name</th>
- <th><abbr title="Tracepoint">TP</abbr> name</th>
- <th>Instrumented function</th>
- </tr>
- </thead>
- <tbody>
- <tr>
- <td rowspan="4">
- <code class="no-bg">lttng_ust_pthread</code>
- </td>
- <td>
- <code class="no-bg">pthread_mutex_lock_req</code>
- </td>
- <td>
- <code class="no-bg">pthread_mutex_lock()</code> (request time)
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">pthread_mutex_lock_acq</code>
- </td>
- <td>
- <code class="no-bg">pthread_mutex_lock()</code> (acquire time)
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">pthread_mutex_trylock</code>
- </td>
- <td>
- <code class="no-bg">pthread_mutex_trylock()</code>
- </td>
- </tr>
- <tr>
- <td>
- <code class="no-bg">pthread_mutex_unlock</code>
- </td>
- <td>
- <code class="no-bg">pthread_mutex_unlock()</code>
- </td>
- </tr>
- </tbody>
-</table>
-</div>
-
-All tracepoints have fields corresponding to the arguments of the
-function they instrument.
-
-To use one or the other with any user application, independently of
-how the latter is built, do:
-
-<pre class="term">
-LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
-</pre>
-
-or
-
-<pre class="term">
-LD_PRELOAD=liblttng-ust-pthread-wrapper.so my-app
-</pre>
-
-To use both, do:
-
-<pre class="term">
-LD_PRELOAD="liblttng-ust-libc-wrapper.so liblttng-ust-pthread-wrapper.so" my-app
-</pre>
-
-When the shared object is preloaded, it effectively replaces the
-functions listed in the above tables by wrappers which add tracepoints
-and call the replaced functions.
-
-Of course, like any other tracepoint, the ones above need to be enabled
-in order for LTTng-UST to generate events. This is done using the
-`lttng` command line tool
-(see [Controlling tracing](#doc-controlling-tracing)).
+++ /dev/null
----
-id: proc-lttng-logger-abi
-since: 2.5
----
-
-The `lttng-tracer` Linux kernel module, installed by the LTTng-modules
-package, creates a special LTTng logger ABI file `/proc/lttng-logger`
-when loaded. Writing text data to this file generates an LTTng kernel
-domain event named `lttng_logger`.
-
-Unlike other kernel domain events, `lttng_logger` may be enabled by
-any user, not only root users or members of the tracing group.
-
-To use the LTTng logger ABI, simply write a string to
-`/proc/lttng-logger`:
-
-<pre class="term">
-echo -n 'Hello, World!' > /proc/lttng-logger
-</pre>
-
-The `msg` field of the `lttng_logger` event contains the recorded
-message.
-
-<div class="tip">
-<p>
- <span class="t">Note:</span>Messages are split in chunks of
- 1024 bytes.
-</p>
-</div>
-
-The LTTng logger ABI is a quick and easy way to trace some events from
-user space through the kernel tracer. However, it is much more basic
-than LTTng-UST: it's slower (involves system call round-trip to the
-kernel and only supports logging strings). The LTTng logger ABI is
-particularly useful for recording logs as LTTng traces from shell
-scripts, potentially combining them with other Linux kernel/user space
-events.
+++ /dev/null
----
-id: python-application
-since: 2.7
----
-
-Python 2 and Python 3 applications using the standard
-<a href="https://docs.python.org/3/howto/logging.html" class="ext"><code>logging</code> module</a>
-can be traced by LTTng using the LTTng-UST Python agent.
-
-Import the `lttngust` package in your Python application. For example:
-
-~~~ python
-import lttngust
-import logging
-import time
-
-
-def example():
- logging.basicConfig()
- logger = logging.getLogger('my-logger')
-
- while True:
- logger.debug('debug message')
- logger.info('info message')
- logger.warn('warn message')
- logger.error('error message')
- logger.critical('critical message')
- time.sleep(1)
-
-
-if __name__ == '__main__':
- example()
-~~~
-
-Importing `lttngust` adds a logging handler which emits LTTng-UST
-events. You do not need to get a special logger for tracing to work.
-
-Note that `logging.basicConfig()`, which adds to the root logger a basic
-logging handler which prints to the standard error stream, is not
-strictly required for LTTng-UST tracing to work, but in versions of
-Python preceding 3.2, a warning message could be seen indicating that no
-handler exists for the logger `my-logger`.
-
-Use the `--python` option of the `lttng enable-event`,
-`lttng disable-event`, and `lttng list` commands to target
-Python applications. For example, here's how to enable the events
-produced by the Python logger above:
-
-<pre class="term">
-lttng enable-event --python my-logger
-</pre>
-
-Standard Python log levels are supported using the `PYTHON_` prefix.
-For example, here's how to enable the warning (and more important)
-events produced by the Python logger above:
-
-<pre class="term">
-lttng enable-event --python my-logger --loglevel PYTHON_WARNING
-</pre>
-
-See [Enabling and disabling events](#doc-enabling-disabling-events) for
-more options.
-
-When loading, the LTTng-UST Python agent tries to register to the
-[session daemon](#doc-lttng-sessiond). Note that the session daemon
-needs to be started _before_ the Python application is started. If a
-session daemon is found, the agent tries to register to it during
-5 seconds, after which the application continues without LTTng
-tracing support. This timeout value is overriden by the
-`LTTNG_UST_PYTHON_REGISTER_TIMEOUT` environment variable (milliseconds).
-
-If the session daemon stops while a registered Python application is
-registered, the application retries to connect and register to a session
-daemon every 3 seconds. This timeout value is overridden by the
-`LTTNG_UST_PYTHON_REGISTER_RETRY_DELAY` environment variable.
+++ /dev/null
----
-id: using-lttng
----
-
-Using LTTng involves two main activities: **instrumenting** and
-**controlling tracing**.
-
-_[Instrumenting](#doc-instrumenting)_ is the process of inserting probes
-into some source code. It can be done manually, by writing tracepoint
-calls at specific locations in the source code of the program to trace,
-or more automatically using dynamic probes (address in assembled code,
-symbol name, function entry/return, and others).
-
-It has to be noted that, as an LTTng user, you may not have to worry
-about the instrumentation process. Indeed, you may want to trace a
-program already instrumented. As an example, the Linux kernel is
-thoroughly instrumented, which is why you can trace it without caring
-about adding probes.
-
-_[Controlling tracing](#doc-controlling-tracing)_ is everything
-that can be done by the LTTng session daemon, which is controlled using
-`liblttng-ctl` or its command line utility, `lttng`: creating tracing
-sessions, listing tracing sessions and events, enabling/disabling
-events, starting/stopping the tracers, taking snapshots, amongst many
-other commands.
-
-This chapter is a complete user guide of both activities,
-with common use cases of LTTng exposed throughout the text. It is
-assumed that you are familiar with LTTng's concepts (events, channels,
-domains, tracing sessions) and that you understand the roles of its
-components (daemons, libraries, command line tools); if not, we invite
-you to read the [Understanding LTTng](#doc-understanding-lttng) chapter
-before you begin reading this one.
-
-If you're new to LTTng, we suggest that you rather start with the
-[Getting started](#doc-getting-started) small guide first, then come
-back here to broaden your knowledge.
-
-If you're only interested in tracing the Linux kernel with its current
-instrumentation, you may skip the
-[Instrumenting](#doc-instrumenting) section.
+++ /dev/null
----
-id: whats-new
----
-
-LTTng 2.7 ships with a generous list of new features, with essential
-additions to all the project's components.
-
-Dynamic filtering of user space tracepoints has been available for
-quite some time now
-(see [Enabling and disabling events](#doc-enabling-disabling-events)).
-LTTng 2.7 adds filtering support to kernel events as well. For example:
-
-<pre class="term">
-lttng enable-event --kernel irq_handler_entry --filter 'irq == 28'
-</pre>
-
-LTTng 2.7 adds wildcard support for kernel event names:
-
-<pre class="term">
-lttng enable-event --kernel 'sched_*'
-</pre>
-
-On the user space tracing side, the new [`tracelog()`](#doc-tracelog)
-facility allows users to easily migrate from logging to tracing.
-`tracelog()` is similar to [`tracef()`](#doc-tracef), but accepts
-an additional log level parameter.
-
-The new `--shm-path` option of `lttng create` can be used to specify the
-path where the shared memory holding the ring buffers are
-created. This feature is useful when used with persistent memory file
-systems to extract the latest recorded trace data in the event of a
-crash requiring a reboot. The new `lttng-crash` command line
-utility can extract trace data from such a file (see
-[Recording trace data on persistent memory file systems](#doc-persistent-memory-file-systems)).
-
-LTTng-UST 2.7 can rely on a user plugin to provide a custom clock source
-to its tracer. LTTng-UST can also load a user plugin to retrieve the
-current CPU number. This feature exists for very advanced use cases. See
-the <a href="https://github.com/lttng/lttng-ust/tree/master/doc/examples/clock-override" class="ext">clock-override</a>
-and <a href="https://github.com/lttng/lttng-ust/tree/master/doc/examples/getcpu-override" class="ext">getcpu-override</a>
-examples for more details.
-
-Python developers can now benefit from the new
-[LTTng-UST Python agent](#doc-python-application),
-a Python 2/3-compatible package which allows standard Python logging
-using the `logging` module to output log entries to an LTTng trace.
-
-Last but not least, the new `lttng track` and `lttng untrack` commands
-make [<abbr title="process ID">PID</abbr> tracking](#doc-pid-tracking)
-super-fast for both the kernel and the user space domains. When one or
-more PIDs are tracked, only the processes having those PIDs are allowed
-to emit enabled events.
-
-Moreover, LTTng 2.7 boasts great stability, benifiting from piles of
-bug fixes and more-than-welcome internal refactorings.
-
-To learn more about the new features of LTTng 2.7, see
-<a href="https://lttng.org/blog/2015/10/14/lttng-2.7-released/" class="ext">the release announcement</a>.
+++ /dev/null
-title: The <span class="reset-text-transform">LTTng</span> Documentation
-cats:
- - id: whats-new
- title: What's new in LTTng 2.7?
- - id: nuts-and-bolts
- title: Nuts and bolts
- cats:
- - id: what-is-tracing
- title: What is tracing?
- - id: lttng-alternatives
- title: Alternatives to LTTng
- - id: installing-lttng
- title: Installing <span class="reset-text-transform">LTTng</span>
- cats:
- - id: desktop-distributions
- title: Desktop distributions
- cats:
- - id: ubuntu
- title: Ubuntu
- cats:
- - id: ubuntu-official-repositories
- title: Official repositories
- - id: ubuntu-ppa
- title: PPA
- - id: debian
- title: Debian
- - id: opensuse
- title: openSUSE/RPM
- - id: archlinux
- title: Arch Linux
- - id: enterprise-distributions
- title: "Enterprise distributions (<abbr title=\"Red Hat Enterprise Linux\">RHEL</abbr>, <abbr title=\"SUSE Linux Enterprise Server\">SLES</abbr>)"
- - id: building-from-source
- title: Building from source
- - id: getting-started
- title: Getting started with <span class="reset-text-transform">LTTng</span>
- cats:
- - id: tracing-the-linux-kernel
- title: Tracing the Linux kernel
- - id: tracing-your-own-user-application
- title: Tracing your own user application
- - id: viewing-and-analyzing-your-traces
- title: Viewing and analyzing your traces
- - id: understanding-lttng
- title: Understanding <span class="reset-text-transform">LTTng</span>
- cats:
- - id: core-concepts
- title: Core concepts
- cats:
- - id: tracing-session
- title: Tracing session
- - id: domain
- title: Domain
- - id: channel
- title: Channel
- cats:
- - id: channel-overwrite-mode-vs-discard-mode
- title: Overwrite and discard event loss modes
- - id: channel-subbuf-size-vs-subbuf-count
- title: Sub-buffers count and size
- - id: channel-switch-timer
- title: Switch timer
- - id: channel-buffering-schemes
- title: Buffering schemes
- - id: event
- title: Event
- - id: plumbing
- title: Plumbing
- cats:
- - id: plumbing-overview
- title: Overview
- - id: lttng-sessiond
- title: Session daemon
- - id: lttng-consumerd
- title: Consumer daemon
- - id: lttng-relayd
- title: Relay daemon
- - id: liblttng-ctl-lttng
- title: Control library and command line interface
- - id: lttng-ust
- title: User space tracing library
- - id: lttng-modules
- title: <span class="reset-text-transform">LTTng</span> kernel modules
- - id: using-lttng
- title: Using <span class="reset-text-transform">LTTng</span>
- cats:
- - id: instrumenting
- title: Instrumenting
- cats:
- - id: c-application
- title: C application
- cats:
- - id: tracepoint-provider
- title: Tracepoint provider
- - id: lttng-gen-tp
- title: Using <code>lttng-gen-tp</code>
- - id: defining-tracepoints
- title: Defining tracepoints
- - id: using-tracepoint-classes
- title: Using tracepoint classes
- - id: assigning-log-levels
- title: Assigning log levels to tracepoints
- - id: probing-the-application-source-code
- title: Probing the application's source code
- - id: building-tracepoint-providers-and-user-application
- title: Building/linking tracepoint providers and the user application
- cats:
- - id: static-linking
- title: Static linking
- - id: dynamic-linking
- title: Dynamic linking
- - id: using-lttng-ust-with-daemons
- title: Using <span class="reset-text-transform">LTTng</span>-UST with daemons
- - id: lttng-ust-pkg-config
- title: <span class="reset-text-transform">pkg-config</span>
- - id: tracef
- title: Using <code>tracef()</code>
- - id: tracelog
- title: Using <code>tracelog()</code>
- - id: lttng-ust-environment-variables-compiler-flags
- title: <span class="reset-text-transform">LTTng</span>-UST environment variables and special compilation flags
- - id: cxx-application
- title: C++ application
- - id: prebuilt-ust-helpers
- title: Prebuilt user space tracing helpers
- cats:
- - id: liblttng-ust-libc-pthread-wrapper
- title: C standard library and POSIX threads tracing
- - id: liblttng-ust-cyg-profile
- title: Function tracing
- - id: liblttng-ust-dl
- title: Dynamic linker tracing
- - id: java-application
- title: Java application
- cats:
- - id: jul
- title: <code>java.util.logging</code>
- - id: log4j
- title: Apache log4j 1.2
- - id: python-application
- title: Python application
- - id: instrumenting-linux-kernel
- title: Linux kernel
- cats:
- - id: instrumenting-linux-kernel-itself
- title: Instrumenting the Linux kernel for <span class="reset-text-transform">LTTng</span>
- cats:
- - id: mainline-trace-event
- title: Defining/using tracepoints with mainline <code>TRACE_EVENT()</code> infrastructure
- - id: lttng-adaptation-layer
- title: Adding the <span class="reset-text-transform">LTTng</span> adaptation layer
- - id: lttng-tracepoint-event-code
- title: Using custom C code to access the data for tracepoint fields
- - id: instrumenting-linux-kernel-tracing
- title: Tracing
- - id: instrumenting-out-of-tree-linux-kernel
- title: Instrumenting an out-of-tree Linux kernel module for <span class="reset-text-transform">LTTng</span>
- - id: proc-lttng-logger-abi
- title: <span class="reset-text-transform">LTTng</span> logger ABI
- - id: advanced-instrumenting-techniques
- title: Advanced techniques
- cats:
- - id: instrumenting-32-bit-app-on-64-bit-system
- title: Instrumenting a 32-bit application on a 64-bit system
- cats:
- - id: building-32-bit-userspace-rcu
- title: Building 32-bit Userspace RCU
- - id: building-32-bit-lttng-ust
- title: Building 32-bit <span class="reset-text-transform">LTTng</span>-UST
- - id: building-32-bit-lttng-tools
- title: Building 32-bit <span class="reset-text-transform">LTTng</span>-tools
- - id: building-64-bit-lttng-tools
- title: Building 64-bit <span class="reset-text-transform">LTTng</span>-tools
- - id: building-instrumented-32-bit-c-application
- title: Building an instrumented 32-bit C application
- - id: running-32-bit-and-64-bit-c-applications
- title: Running 32-bit and 64-bit versions of an instrumented C application
- - id: controlling-tracing
- title: Controlling tracing
- cats:
- - id: creating-destroying-tracing-sessions
- title: Creating and destroying tracing sessions
- - id: enabling-disabling-events
- title: Enabling and disabling events
- - id: basic-tracing-session-control
- title: Basic tracing session control
- - id: enabling-disabling-channels
- title: Enabling and disabling channels
- cats:
- - id: fine-tuning-channels
- title: Fine-tuning channels
- - id: adding-context
- title: Adding some context to channels
- - id : pid-tracking
- title: Tracking process IDs
- - id: saving-loading-tracing-session
- title: Saving and loading tracing session configurations
- - id: sending-trace-data-over-the-network
- title: Sending trace data over the network
- - id: lttng-live
- title: Viewing events as they arrive
- - id: taking-a-snapshot
- title: Taking a snapshot
- - id: mi
- title: Machine interface
- - id: persistent-memory-file-systems
- title: Recording trace data on persistent memory file systems
- - id: reference
- title: Reference
- cats:
- - id: online-lttng-manpages
- title: Online <span class="reset-text-transform">LTTng</span> man pages
- - id: lttng-ust-ref
- title: <span class="reset-text-transform">LTTng</span>-UST
- cats:
- - id: liblttng-ust
- title: <span class="reset-text-transform">LTTng</span>-UST library (<code>liblttng‑ust</code>)
- cats:
- - id: liblttng-ust-tp-fields
- title: Tracepoint fields macros (for <code>TP_FIELDS()</code>)
- - id: liblttng-ust-tracepoint-loglevel
- title: Tracepoint log levels (for <code>TRACEPOINT_LOGLEVEL()</code>)
- - id: lttng-modules-ref
- title: <span class="reset-text-transform">LTTng</span>-modules
- cats:
- - id: lttng-modules-tp-fields
- title: Tracepoint fields macros (for <code>TP_FIELDS()</code>)
--- /dev/null
+[attributes]
+note-no-anim="note-no-anim"
+
+[macros]
+(?su)[\\]?(?P<name>man):(?P<page>\S*?)\((?P<section>.*?)\)=
+(?su)[\\]?(?P<name>path):\{(?P<path>[^}]+)\}=
+(?su)[\\]?(?P<name>dir):\{(?P<dir>[^}]+)\}=
+(?su)[\\]?(?P<name>env):(?P<var>[a-zA-Z0-9_]+)=
+(?su)[\\]?(?P<name>cmd):(?P<var>[a-zA-Z0-9_-]+)=
+
+[man-inlinemacro]
+<citerefentry>
+ <refentrytitle>{page}</refentrytitle>
+ <manvolnum>{section}</manvolnum>
+</citerefentry>
+
+[path-inlinemacro]
+<filename>{path}</filename>
+
+[dir-inlinemacro]
+<filename class="directory">{dir}</filename>
+
+[env-inlinemacro]
+<envar>{var}</envar>
+
+[cmd-inlinemacro]
+<command>{var}</command>
+
+[tabledef-default]
+style=def
+def-style=options=("header",)
--- /dev/null
+# The MIT License (MIT)
+#
+# Copyright (c) 2016 Philippe Proulx <pproulx@efficios.com>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+# THE SOFTWARE.
+
+from termcolor import colored
+import lxml.etree as etree
+import subprocess
+import argparse
+import os.path
+import sys
+import os
+
+
+def _perror(msg, exit=True):
+ print('{} {}'.format(colored('Error:', 'red'), colored(msg, 'red', attrs=['bold'])),
+ file=sys.stderr)
+
+ if exit:
+ sys.exit(1)
+
+
+def _pinfo(msg):
+ print('{} {}'.format(colored('::', 'blue'), colored(msg, 'blue', attrs=['bold'])))
+
+
+def _get_script_dir():
+ return os.path.dirname(os.path.realpath(__file__))
+
+
+class _Checker:
+ def __init__(self, infile, verbose):
+ self._infile = infile
+ self._verbose = verbose
+ self._has_error = False
+ self._set_paths()
+ self._pverbose('asciidoc -> DocBook')
+ self._build()
+ self._set_root()
+ self._check()
+
+ @property
+ def has_error(self):
+ return self._has_error
+
+ def _pverbose(self, msg):
+ if self._verbose:
+ _pinfo(msg)
+
+ def _perror(self, msg, fatal=False):
+ self._has_error = True
+ _perror(msg, fatal)
+
+ def _set_paths(self):
+ self._indir = os.path.dirname(self._infile)
+ self._imgexportdir = os.path.join(self._indir, 'images', 'export')
+ self._builddir = os.path.join(_get_script_dir(), 'check', os.path.basename(self._infile))
+ self._outfile = os.path.join(self._builddir, 'out.xml')
+
+ def _build(self):
+ conf = os.path.join(_get_script_dir(), 'asciidoc.check.conf')
+ os.makedirs(self._builddir, mode=0o755, exist_ok=True)
+ cmd = [
+ 'asciidoc',
+ '-f', conf,
+ '-b', 'docbook',
+ '-o', self._outfile,
+ ]
+
+ if self._verbose:
+ cmd.append('-v')
+
+ cmd.append(self._infile)
+ res = subprocess.run(cmd)
+
+ if res.returncode != 0:
+ self._perror('asciidoc did not finish successfully', True)
+
+ def _set_root(self):
+ tree = etree.ElementTree(file=self._outfile)
+ self._root = tree.getroot()
+
+ def _check(self):
+ self._pverbose('Checking links')
+ self._check_links()
+ self._pverbose('Checking images')
+ self._check_images()
+
+ def _check_links(self):
+ sections_anchors = self._root.findall('.//section')
+ sections_anchors += self._root.findall('.//anchor')
+ sections_anchors += self._root.findall('.//glossary')
+ links = self._root.findall('.//link')
+ end_ids = set()
+
+ for sa in sections_anchors:
+ end_id = sa.get('id')
+
+ if end_id is None:
+ self._perror('Found a section/anchor with no ID', True)
+
+ end_ids.add(end_id)
+
+ link_ends = set()
+
+ for link in links:
+ end = link.get('linkend')
+
+ if end is None:
+ self._perror('Found a link with no end', True)
+
+ link_ends.add(end)
+
+ has_error = False
+
+ for end in link_ends:
+ if end not in end_ids:
+ self._perror('Link end "{}" does not name a section/anchor ID'.format(end))
+
+ def _check_images(self):
+ image_datas = self._root.findall('.//imagedata')
+
+ for image_data in image_datas:
+ fileref = image_data.get('fileref')
+ path = os.path.join(self._imgexportdir, fileref)
+
+ if not os.path.isfile(path):
+ self._perror('Cannot find image "{}"'.format(fileref))
+
+
+def _parse_args():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('-v', '--verbose', action='store_true')
+ parser.add_argument('infile')
+ args = parser.parse_args()
+
+ if not os.path.isfile(args.infile):
+ _perror('"{}" is not an existing file'.format(args.infile))
+
+ return args
+
+
+def _main():
+ args = _parse_args()
+ checker = _Checker(args.infile, args.verbose)
+
+ if checker.has_error:
+ return 1
+
+ print(colored('All good!', 'green', attrs=['bold']))
+
+ return 0
+
+
+if __name__ == '__main__':
+ sys.exit(_main())
+++ /dev/null
-#!/usr/bin/env python3
-
-# The MIT License (MIT)
-#
-# Copyright (c) 2015 Philippe Proulx <pproulx@efficios.com>
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-# THE SOFTWARE.
-
-import re
-import os
-import sys
-import json
-from termcolor import colored
-
-
-_TOC_PATH = 'toc/docs.yml'
-_CONTENTS_ROOT_PATH = 'contents'
-
-
-class _Link:
- pass
-
-
-class _IntLink(_Link):
- def __init__(self, section):
- self._section = section
-
- @property
- def section(self):
- return self._section
-
- def __eq__(self, other):
- if type(self) != type(other):
- return False
-
- return self._section == other._section
-
- def __hash__(self):
- return hash(self._section)
-
- def to_json(self):
- return {
- 'section': self._section,
- }
-
-
-class _ExtLink(_Link):
- def __init__(self, url):
- self._url = url
-
- @property
- def url(self):
- return self._url
-
- def __eq__(self, other):
- if type(self) != type(other):
- return False
-
- return self._url == other._url
-
- def __hash__(self):
- return hash(self._url)
-
- def to_json(self):
- return {
- 'url': self._url,
- }
-
-
-class _SectionInfo:
- def __init__(self, path):
- self._path = path
- self._in_links = set()
- self._out_links = set()
-
- @property
- def path(self):
- return self._path
-
- @property
- def in_links(self):
- return self._in_links
-
- @property
- def out_links(self):
- return self._out_links
-
- def add_in_link(self, link):
- self._in_links.add(link)
-
- def add_out_link(self, link):
- self._out_links.add(link)
-
- def to_json(self):
- section_json = {
- 'path': self.path,
- }
- in_links_json = []
- out_links_json = []
-
- for in_link in self.in_links:
- in_links_json.append(in_link.to_json())
-
- for out_link in self.out_links:
- out_links_json.append(out_link.to_json())
-
- section_json['in-links'] = in_links_json
- section_json['out-links'] = out_links_json
-
- return section_json
-
-
-class _Registry:
- def __init__(self):
- self._section_infos = {}
-
- def register_section_info(self, sid, section_info):
- self._section_infos[sid] = section_info
-
- def _resolve_in_links(self):
- for sid in self._section_infos:
- section_info = self._section_infos[sid]
- for out_link in section_info.out_links:
- if type(out_link) != _IntLink:
- continue
-
- target_sid = out_link.section
- target_section_info = self._section_infos[target_sid]
- target_section_info.add_in_link(_IntLink(sid))
-
- def to_json(self):
- self._resolve_in_links()
- sections_json = {}
-
- for sid, section_info in self._section_infos.items():
- sections_json[sid] = section_info.to_json()
-
- return json.dumps(sections_json)
-
-
-def _perror(filename, msg):
- s = '{} {} {}'.format(filename, colored('Error:', 'red'),
- colored(msg, 'red', attrs=['bold']))
- print(s, file=sys.stderr)
-
-
-def _pwarn(filename, msg):
- s = '{} {} {}'.format(filename, colored('Warning:', 'yellow'),
- colored(msg, 'yellow', attrs=['bold']))
- print(s, file=sys.stderr)
-
-
-def _get_files(root):
- files = []
-
- for dirpath, dirnames, filenames in os.walk(root):
- for f in filenames:
- files.append(os.path.join(dirpath, f))
-
- return sorted(files)
-
-
-def _get_toc_ids(path):
- p = re.compile(r'id\s*:\s*(.+)$', flags=re.M)
-
- with open(path) as f:
- orig_ids = p.findall(f.read())
-
- ids = set(orig_ids)
-
- if len(ids) != len(orig_ids):
- _perror(path, 'Duplicate IDs')
- return
-
- return ids
-
-
-_id_re = re.compile(r'^\s*id:\s*([a-zA-Z0-9_-]+)\s*$', flags=re.M)
-
-
-def _get_sid_from_file(path, c):
- m = _id_re.search(c)
-
- if not m:
- _perror(path, 'No ID found')
- return
-
- return m.group(1)
-
-
-_ilink_re = re.compile(r'\[[^\]]+\]\(([^)]+)\)', flags=re.M)
-_elink_re = re.compile(r'<a(?:\s+[^>]+|\s*)>')
-_name_re = re.compile(r'name="([^"]+)"')
-_href_re = re.compile(r'href="([^"]+)"')
-_classes_re = re.compile(r'class="([^"]+)"')
-
-
-def _register_section_info(registry, toc_ids, path, c):
- sid = _get_sid_from_file(path, c)
-
- if not sid:
- return False
-
- ret = True
- ilinks = _ilink_re.findall(c)
- elinks = _elink_re.findall(c)
- section_info = _SectionInfo(path)
-
- for link in elinks:
- href = _href_re.search(link)
- name = _name_re.search(link)
- classes = _classes_re.search(link)
-
- if name and not href:
- # simple anchor
- continue
-
- if classes is None:
- _pwarn(path, 'External link has no "ext" class: "{}"'.format(link))
- classes = []
- else:
- classes = classes.group(1).split(' ')
-
- if 'int' in classes and 'ext' in classes:
- _pwarn(path, 'External link has both "ext" and "int" classes: "{}"'.format(link))
- elif 'int' not in classes and 'ext' not in classes:
- _pwarn(path, 'External link has no "ext" or "int" class: "{}"'.format(link))
-
- if href:
- href = href.group(1)
-
- if href.startswith('#') and 'int' not in classes:
- _pwarn(path, 'External link starts with #: "{}"'.format(href))
-
- if 'int' in classes:
- ilinks.append(href)
- continue
-
- section_info.add_out_link(_ExtLink(href))
- elif not name:
- _perror(path, 'External link with no "href" or "name" attribute: "{}"'.format(link))
- ret = False
-
- for link in ilinks:
- if not link.startswith('#doc-'):
- s = 'Internal link does not start with "#doc-": "{}"'.format(link)
- _perror(path, s)
- ret = False
- continue
-
- target_sid = link[5:]
-
- if target_sid not in toc_ids:
- _perror(path, 'Dead internal link: "{}"'.format(link))
- ret = False
- else:
- section_info.add_out_link(_IntLink(target_sid))
-
- registry.register_section_info(sid, section_info)
-
- return ret
-
-
-def _docs2json(toc_ids, contents_files):
- ret = True
- registry = _Registry()
-
- i = 1
-
- for path in contents_files:
- with open(path) as f:
- c = f.read()
-
- ret &= _register_section_info(registry, toc_ids, path, c)
-
- print(registry.to_json())
-
- return ret
-
-
-def _check_non_md(files):
- ret = True
-
- for f in files:
- if not f.endswith('.md'):
- _perror(f, 'Wrong, non-Markdown file: "{}"'.format(f))
- ret = False
-
- return ret
-
-
-def docs2json():
- toc_ids = _get_toc_ids(_TOC_PATH)
-
- if toc_ids is None:
- return False
-
- contents_files = _get_files(_CONTENTS_ROOT_PATH)
-
- if not _check_non_md(contents_files):
- return False
-
- if not _docs2json(toc_ids, contents_files):
- return False
-
- return True
-
-
-if __name__ == '__main__':
- sys.exit(0 if docs2json() else 1)
+++ /dev/null
-#!/usr/bin/env python3
-
-# The MIT License (MIT)
-#
-# Copyright (c) 2015 Philippe Proulx <pproulx@efficios.com>
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in
-# all copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
-# THE SOFTWARE.
-
-import re
-import os
-import sys
-import json
-import graphviz
-
-
-def _get_section_infos(json_path):
- with open(json_path) as f:
- c = f.read()
-
- return json.loads(c)
-
-
-def dotlinks():
- section_infos = _get_section_infos(sys.argv[1])
- digraph = graphviz.Digraph(format='png', engine='dot')
- digraph.attr('node', fontname='Terminus', fontsize='8')
-
- for sid, section_info in section_infos.items():
- color = ''
- style = ''
- in_links_count = len(section_info['in-links'])
- out_links_count = len(section_info['out-links'])
-
- if in_links_count == 0 and out_links_count == 0:
- color = '#e62739'
- elif in_links_count == 0:
- color = '#fae596'
- elif out_links_count == 0:
- color = '#6ed3cf'
-
- if color:
- style = 'filled'
-
- digraph.node(sid, style=style, color=color)
-
- for sid, section_info in section_infos.items():
- out_links = section_info['out-links']
-
- for out_link in out_links:
- if 'section' in out_link:
- dest = out_link['section']
- digraph.edge(sid, dest)
-
- digraph.render(filename='linkgraph')
-
- return True
-
-
-if __name__ == '__main__':
- sys.exit(0 if dotlinks() else 1)
projects / lttng-docs.git / commitdiff
