4 How to use LTTng and LTTV in a few lines :
6 This document is made of four parts : The first one explains how to install
7 LTTng and LTTV from Debian and RPM binary packages, the second one explains how
8 to install LTTng and LTTV from sources and the third one describes the steps
9 to follow to trace a system and view it. The fourth and last part explains
10 briefly how to add a new trace point to the kernel and to user space
13 What you will typically want is to read sections 2 and 3 : install LTTng from
16 These operations are made for installing the LTTng 0.6.X tracer on a
17 linux 2.6.X kernel. You will also find instructions for installation of
18 LTTV 0.8.x : the Linux Trace Toolkit Viewer.
20 To see the list of compatibilities between LTTng, ltt-control, LTTV, genevent
21 and ltt-usertrace, please refer to :
22 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
26 The following lttng patch is necessary to have the tracing hooks in the kernel.
27 The following ltt-control module controls the tracing.
29 Required programs and librairies are assumed to be automatically installed in an
30 installation with Debian or RPM packages. In the case of an installation from
31 sources, the dependencies are listed.
34 ** Current development status **
37 supported architectures :
38 Intel Pentium (UP/SMP) with TSC
39 PowerPC 32 and 64 bits
42 C2 Microsystems (variant of MIPS)
45 supported architectures :
48 PowerPC 32 and 64 bits
52 Author : Mathieu Desnoyers, September 2005
53 Last update : May 30, 2006
56 ***********************************************************
57 ** Section 1 * Installation from Debian or RPM packages **
58 ***********************************************************
60 ** NOTE : RPM and debian packages are only made once a version has been
61 thoroughly tested. If they do not exist at the moment, please install from
62 sources (see section 2 below). To see the list of compatibilities between
63 LTTng, ltt-control, LTTV, genevent and lttng-modules, please refer to
64 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
67 * Install from RPM packages on Fedora Core 4 :
71 http://ltt.polymtl.ca/packages/fedora/RPMS
75 LTTng kernel and lttng-modules RPM are available for some architectures (i586,
76 i686). Feel free to help fix the spec files to have correct lttng-modules RPM
80 * Install from Deb packages on Debian :
82 You can use the ltt.polymtl.ca apt source to get LTTV for Debian :
84 Add the following two sources to your /etc/apt/sources.list :
86 deb http://ltt.polymtl.ca/packages/debian experimental main
87 deb-src http://ltt.polymtl.ca/packages/debian experimental main
90 * Install from precompiled binary packages (LTTV compiled only for i386, and
91 LTTng only for i686 smp), perform the following :
95 apt-get install lttv lttv-doc
96 apt-get install kernel-image-2.6.12-rc4-mm2-lttng-0.4.2
97 apt-get install lttng-modules-modules-2.6.12-rc4-mm2-lttng-0.4.2
98 * note : the packages are signed by myself. I am not considered a trusted
99 Debian source yet, so warnings are normal.
101 Then, follow the section "Editing the system wide configuration" in section 2.
103 * Create custom LTTV Debian packages
105 Binary packages are only available for i386. If you want to create your own LTTV
106 packages for other platforms, do :
112 dpkg-buildpackage -rfakeroot
114 You should then have your LTTV .deb files created for your architecture.
116 * Create custom LTTng packages
118 For building LTTng Debian packages :
121 apt-get install kernel-source-2.6.12-rc4-mm2-lttng-0.4.2
123 bzip2 -cd kernel-source-2.6.12-rc4-mm2-lttng-0.4.2.tar.bz2 | tar xvof -
124 cd kernel-source-2.6.12-rc4-mm2-lttng-0.4.2
125 make menuconfig (or xconfig or config) (customize your configuration)
126 make-kpkg kernel_image
128 You will then see your freshly created .deb in /usr/src. Install it with
129 dpkg -i /usr/src/(image-name).deb
131 You will also need to create a package for the lttng-modules :
135 apt-get source lttng-modules
136 cd kernel-source-2.6.12-rc4-mm2-lttng-0.4.2
137 make-kpkg --added_modules /usr/src/lttng-modules-0.3 modules_image
139 You will then see your freshly created .deb in /usr/src. Install it with
140 dpkg -i /usr/src/lttng-modules-modules-(your version).deb
143 Then, follow the section "Editing the system wide configuration" in section 2.
146 ***********************************************************
147 ** Section 2 * Installation from sources **
148 ***********************************************************
152 Tools needed to follow the package download steps :
159 You have to install the standard development librairies and programs necessary
160 to compile a kernel :
162 (from Documentation/Changes in the Linux kernel tree)
163 o Gnu C 2.95.3 # gcc --version
164 o Gnu make 3.79.1 # make --version
165 o binutils 2.12 # ld -v
166 o util-linux 2.10o # fdformat --version
167 o module-init-tools 0.9.10 # depmod -V
169 You might also want to have libncurses5 to have the text mode kernel
170 configuration menu, but there are alternatives.
172 Prerequisites for LTTV 0.x.x installation are :
175 gtk 2.4 or better development libraries
176 (Debian : libgtk2.0, libgtk2.0-dev)
177 (Fedora : gtk2, gtk2-devel)
178 note : For Fedora users : this might require at least core 3 from Fedora,
179 or you might have to compile your own GTK2 library.
180 glib 2.4 or better development libraries
181 (Debian : libglib2.0-0, libglib2.0-dev)
182 (Fedora : glib2, glib2-devel)
183 libpopt development libraries
184 (Debian : libpopt0, libpopt-dev)
186 libpango development libraries
187 (Debian : libpango1.0, libpango1.0-dev)
188 (Fedora : pango, pango-devel)
189 libc6 development librairies
190 (Debian : libc6, libc6-dev)
191 (Fedora : glibc, glibc)
194 * Getting the LTTng packages
199 (see http://ltt.polymtl.ca/lttng for package listing)
200 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
201 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
204 * Getting LTTng kernel sources
208 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
209 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
211 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
212 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
215 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
218 * Installing a LTTng kernel
221 cd /usr/src/linux-2.6.X-lttng-0.x.xx
222 make menuconfig (or make xconfig or make config)
223 Select the < Help > button if you are not familiar with kernel
225 Items preceded by [*] means they has to be built into the kernel.
226 Items preceded by [M] means they has to be built as modules.
227 Items preceded by [ ] means they should be removed.
228 go to the "Instrumentation Support" section
229 Select the following options :
230 [*] Linux Trace Toolkit Instrumentation Support
231 <M> or <*> Linux Trace Toolkit Tracer
232 <M> or <*> Linux Trace Toolkit Relay+DebugFS Support
233 It makes no difference for the rest of the procedure whether the Tracer
234 is compiled built-in or as a module.
236 [*] Align Linux Trace Toolkit Traces
237 [*] Allow tracing from userspace
238 <M> Linux Trace Toolkit Netlink Controller
239 <M> Linux Trace Toolkit State Dump
240 your choice (see < Help >) :
241 [ ] Activate Linux Trace Toolkit Heartbeat Timer
242 You may or may not decide to compile probes. Afterward, you will have to
243 load the probe modules to enable tracing of their events. The probes
244 automatically select the appropriate facilities.
245 Static instrumentation is a more invasive type of instrumentation that gives
246 the address taking a lock or doing a printk.
252 (if necessary, create a initrd with mkinitrd or your preferate alternative)
253 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
258 Select the Linux 2.6.17-lttng-0.x.xx kernel in your boot loader.
261 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
262 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
263 cp .config /boot/config-2.6.X-lttng-0.x.xx
264 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
265 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
266 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
267 that comes first is the default kernel)
269 select the right entry at the yaboot prompt (see choices : tab, select : type
270 the kernel name followed by enter)
271 Select the Linux 2.6.17-lttng-0.x.xx kernel in your boot loader.
276 * Editing the system wide configuration
278 You must activate debugfs and specify a mount point. This is typically done in
279 fstab such that it happens at boot time.
281 If you have never used DebugFS before, these operation would do this for you :
284 cp /etc/fstab /etc/fstab.lttng.bkp
285 echo "debugfs /mnt/debugfs debugfs rw 0 0" >> /etc/fstab
287 then, rebooting or issuing the following command will activate debugfs :
291 You need to load the LTT modules to be able to control tracing from user
292 space. This is done by issuing the command :
298 modprobe ltt-probe-mm
299 modprobe ltt-probe-kernel
300 modprobe ltt-probe-i386 (or x86_64, powerpc, ppc, arm, mips)
301 modprobe ltt-probe-net
302 modprobe ltt-probe-list
303 modprobe ltt-probe-ipc
304 modprobe ltt-probe-fs
306 If you want to have complete information about the kernel state (including all
307 the process names), you need to load the ltt-statedump module. This is done by
308 issuing the command :
310 modprobe ltt-statedump
312 You can automate at boot time loading the ltt-control module by :
314 cp /etc/modules /etc/modules.bkp
315 echo ltt-control >> /etc/modules
316 echo ltt-core >> /etc/modules
317 echo ltt-relay >> /etc/modules
318 echo ltt-tracer >> /etc/modules
319 echo ltt-probe-mm >> /etc/modules
320 echo ltt-probe-kernel >> /etc/modules
321 echo ltt-probe-i386 >> /etc/modules (or x86_64, powerpc, ppc, arm, mips)
322 echo ltt-probe-net >> /etc/modules
323 echo ltt-probe-list >> /etc/modules
324 echo ltt-probe-ipc >> /etc/modules
325 echo ltt-probe-fs >> /etc/modules
326 echo ltt-statedump >> /etc/modules
328 (note : if you want to probe a marker which is within a module, make sure you
329 load the probe _after_ the module, otherwise the probe will not be able to
330 connect itself to the marker.)
333 * Getting and installing the ltt-control package (on the traced machine)
334 (note : the ltt-control package contains lttd and lttctl. Although it has the
335 same name as the ltt-control kernel module, they are *not* the same thing.)
338 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
339 gzip -cd ltt-control-0.x-xxxx2006.tar.gz | tar xvof -
340 cd ltt-control-0.x-xxxx2006
341 (refer to README to see the development libraries that must be installed on you
347 * Getting and installing the ltt-usertrace package for user space tracing
348 See http://ltt.polymtl.ca/ > USERSPACE TRACING QUICKSTART
351 * Getting and installing the LTTV package (on the visualisation machine, same or
352 different from the visualisation machine)
356 wget http://ltt.polymtl.ca/packages/LinuxTraceToolkitViewer-0.x.xx-xxxx2006.tar.gz
357 gzip -cd LinuxTraceToolkitViewer-0.x.xx-xxxx2006.tar.gz | tar xvof -
358 cd LinuxTraceToolkitViewer-0.x.xx-xxxx2006
359 (refer to README to see the development libraries that must be installed on you
368 ***********************************************************
369 ** Section 3 * Using LTTng and LTTV **
370 ***********************************************************
372 * Use graphical LTTV to control tracing and analyse traces
374 lttv-gui (or /usr/local/bin/lttv-gui)
375 - Spot the "Tracing Control" icon : click on it
376 (it's a traffic light icon)
377 - enter the root password
381 * You should now see a trace
383 * Use text mode LTTng to control tracing
385 The tracing can be controlled from a terminal by using the lttctl command (as
390 lttctl -n trace -d -l /mnt/debugfs/ltt -t /tmp/trace
392 Stop tracing and destroy trace channels :
396 see lttctl --help for details.
398 (note : to see if the buffers has been filled, look at the dmesg output after
399 lttctl -R or after stopping tracing from the GUI, it will show an event lost
400 count. If it is the case, try using larger buffers. See lttctl --help to learn
405 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
406 graphical plugins available.
408 For example, a simple trace dump in text format is available with :
410 lttv -m textDump -t /tmp/trace
412 see lttv -m textDump --help for detailed command line options of textDump.
414 It is, in the current state of the project, very useful to use "grep" on the
415 text output to filter by specific event fields. You can later copy the timestamp
416 of the events to the clipboard and paste them in the GUI by clicking on the
417 bottom right label "Current time". Support for this type of filtering should
418 be added to the filter module soon.
421 ***********************************************************
422 ** Section 4 * Adding new instrumentations with genevent **
423 ***********************************************************
425 * Getting and installing genevent
429 wget http://ltt.polymtl.ca/packages/genevent-0.xx.tar.gz
430 gzip -cd genevent-0.xx.tar.gz | tar xvof -
436 * Add new events to the kernel with genevent (deprecated in LTTng 0.9.x)
439 cd /usr/local/share/ltt-control/facilities
440 cp process.xml yourfacility.xml
441 * edit yourfacility.xml to fit your needs.
443 /usr/local/bin/genevent /usr/local/share/ltt-control/facilities/yourfacility.xml
444 cp ltt-facility-yourfacility.h ltt-facility-id-yourfacility.h \
445 /usr/src/linux-2.6.17-lttng-0.x.xx8/include/ltt
446 cp ltt-facility-loader-yourfacility.c ltt-facility-loader-yourfacility.h \
447 /usr/src/linux-2.6.17-lttng-0.x.xx/ltt/facilities
448 * edit the kernel file you want to instrument to add a marker to it. See
449 include/linux/marker.h.
450 * create a dynamically loadable probe. See ltt/probes for examples. The probe
451 will be connected to your marker and will typically call the logging
452 functions found in the header file you created with genevent.
454 * Add new kernel events
456 *Important* note : in its current state, LTTng and LTTV needs the programmer
457 to keep the marker/probe format string and the XML description of the
458 event data types in sync by hand. Failure to do so will result in errors in
461 See the markers documentation to see how to describe the marker. You will need
462 to clone probe modules found in ltt/probes to connect them to the markers so
463 that the information can be recorded in the trace.
465 * Add new events to userspace programs with genevent
466 See http://ltt.polymtl.ca/ > USERSPACE TRACING QUICKSTART
468 User-space tracing still uses genevent, which is subject to change in a near