From: compudj Date: Fri, 23 Jan 2009 22:09:50 +0000 (+0000) Subject: update manual X-Git-Tag: v0.12.20~197 X-Git-Url: http://git.lttng.org./?a=commitdiff_plain;h=55cdc3f081b518fd393f1fb11a3a45d044cbd26d;hp=a5e0a60a66139d3ba7ab0a5955039e7bdd640d37;p=lttv.git update manual git-svn-id: http://ltt.polymtl.ca/svn@3292 04897980-b3bd-0310-b5e0-8ef037075253 --- diff --git a/trunk/lttv/LTTngManual.html b/trunk/lttv/LTTngManual.html new file mode 100644 index 00000000..31c6bbae --- /dev/null +++ b/trunk/lttv/LTTngManual.html @@ -0,0 +1,620 @@ + + + + Linux Trace Toolkit Next Generation Manual + + + +

Linux Trace Toolkit Next Generation Manual

+ +Author : Mathieu Desnoyers, September 2005
+Last update : January 21st, 2009
+(originally known as the LTTng QUICKSTART guide) + +

Table of Contents

+ + + +
+ +

Introduction

+

+This document is made of five parts : the first one explains how +to install LTTng and LTTV from sources, the second one describes the steps +to follow to trace a system and view it. The third part explains +briefly how to add a new trace point to the kernel and to user space +applications. The fourth part explains how to create Debian or RPM +packages from the LTTng and LTTV sources. The fifth and last part describes use +of LTTng in the field. +

+These operations are made for installing the LTTng 0.86 tracer on a linux 2.6.X +kernel. You will also find instructions for installation of LTTV 0.12.x : the +Linux Trace Toolkit Viewer. +To see the list of compatibilities between LTTng, ltt-control, LTTV, please +refer to : +LTTng+LTTV versions compatibility + +The ongoing work had the Linux Kernel Markers integrated in the mainline Linux +kernel since Linux 2.6.24 and the Tracepoints since 2.6.28. In its current +state, the lttng patchset is necessary to have the trace clocksource, the +instrumentation and the LTTng high-speed data extraction mechanism added to the +kernel. + +
+
+

Supported architectures

+
+LTTng :
+
+
  • x86 32/64 bits +
  • PowerPC 32 and 64 bits +
  • ARM (with limited timestamping precision, e.g. 1HZ. Need +architecture-specific support for better precision) +
  • MIPS +
  • sh (partial architecture-specific instrumentation) +
  • sparc64 (partial architecture-specific instrumentation) +
  • s390 (partial architecture-specific instrumentation) +
  • Other architectures supported without architecture-specific instrumentation +and with low-resolution timestamps.
    +
    +
    +LTTV :
    +
    +
  • Intel 32/64 bits +
  • PowerPC 32 and 64 bits +
  • Possibly others. Takes care of endianness and type size difference between +the LTTng traces and the LTTV analysis tool. + +
    + + +

    Installation from sources

    +

    + +

    Prerequisites

    + + +
  • Reminder
  • + +

    +See the list of compatibilities between LTTng, ltt-control and LTTV at : +LTTng+LTTV +versions compatibility. + + +

    Getting the LTTng packages

    + +
    +su -
    +mkdir /usr/src/lttng
    +cd /usr/src/lttng
    +(see http://ltt.polymtl.ca/lttng for package listing)
    +wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
    +bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
    +
    + + +

    Getting LTTng kernel sources

    + +
    +su -
    +cd /usr/src
    +wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
    +bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
    +cd linux-2.6.X
    +- For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
    +- For LTTng 0.9.5+ apply the patches in the order specified in the series file,
    +  or use quilt
    +cd ..
    +mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
    +
    + + +

    Installing a LTTng kernel

    + +
    +su -
    +cd /usr/src/linux-2.6.X-lttng-0.x.xx
    +make menuconfig (or make xconfig or make config)
    +    Select the < Help > button if you are not familiar with kernel
    +    configuration.
    +    Items preceded by [*] means they has to be built into the kernel.
    +    Items preceded by [M] means they has to be built as modules.
    +    Items preceded by [ ] means they should be removed.
    +  go to the "General setup" section
    +    Select the following options :
    +    [*] Prompt for development and/or incomplete code/drivers
    +    [*] Activate markers
    +    [*] Activate userspace markers ABI (experimental, optional)
    +    [*] Immediate value optimization (optional)
    +    [*] Linux Trace Toolkit Next Generation (LTTng) --->
    +     or <*> Compile lttng tracing probes
    +     or <*> Linux Trace Toolkit High-speed Lockless Data Relay
    +     or <*> Linux Trace Toolkit Lock-Protected Data Relay
    +     or <*> Linux Trace Toolkit Serializer
    +     or <*>   Linux Trace Toolkit Marker Control
    +     or <*>     Linux Trace Toolkit Tracer
    +    [*] Align Linux Trace Toolkit Traces
    +     or <*> Support logging events from userspace
    +    [*] Support trace extraction from crash dump
    +     or <*> Linux Trace Toolkit Trace Controller
    +     or <*> Linux Trace Toolkit State Dump
    +  Select 
    +  Select 
    +  Select 
    +make
    +make modules_install
    +(if necessary, create a initrd with mkinitrd or your preferate alternative)
    +(mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
    +
    +-- on X86, X86_64
    +make install
    +reboot
    +Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
    +
    +-- on PowerPC
    +cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
    +cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
    +cp .config /boot/config-2.6.X-lttng-0.x.xx
    +depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
    +mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
    +(edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
    +that comes first is the default kernel)
    +ybin
    +select the right entry at the yaboot prompt (see choices : tab, select : type
    +the kernel name followed by enter)
    +Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
    +--
    +
    + +

    Editing the system wide +configuration

    + +

    +You must activate debugfs and specify a mount point. This is typically done in +fstab such that it happens at boot time. If you have never used DebugFS before, +these operation would do this for you : + +

    +mkdir /mnt/debugfs
    +cp /etc/fstab /etc/fstab.lttng.bkp
    +echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
    +
    + +

    +then, rebooting or issuing the following command will activate debugfs : +

    +mount /mnt/debugfs
    +
    + +

    +You need to load the LTT modules to be able to control tracing from user +space. This is done by issuing the following commands. Note however +these commands load all LTT modules. Depending on what options you chose to +compile statically, you may not need to issue all these commands. + +

    +modprobe ltt-trace-control
    +modprobe ltt-marker-control
    +modprobe ltt-tracer
    +modprobe ltt-serialize
    +modprobe ltt-relay
    +modprobe ipc-trace
    +modprobe kernel-trace
    +modprobe mm-trace
    +modprobe net-trace
    +modprobe fs-trace
    +modprobe jbd2-trace
    +modprobe ext4-trace
    +modprobe syscall-trace
    +modprobe trap-trace
    +#if locking tracing is wanted, uncomment the following
    +#modprobe lockdep-trace
    +
    + +

    +If you want to have complete information about the kernel state (including all +the process names), you need to load the ltt-statedump module. This is done by +issuing the command : + +

    +modprobe ltt-statedump
    +
    +

    +You can automate at boot time loading the ltt-control module by : + +

    +cp /etc/modules /etc/modules.bkp
    +echo ltt-trace-control >> /etc/modules
    +echo ltt-marker-control >> /etc/modules
    +echo ltt-tracer >> /etc/modules
    +echo ltt-serialize >> /etc/modules
    +echo ltt-relay >> /etc/modules
    +echo ipc-trace >> /etc/modules
    +echo kernel-trace >> /etc/modules
    +echo mm-trace >> /etc/modules
    +echo net-trace >> /etc/modules
    +echo fs-trace >> /etc/modules
    +echo jbd2-trace >> /etc/modules
    +echo ext4-trace >> /etc/modules
    +echo syscall-trace >> /etc/modules
    +echo trap-trace >> /etc/modules
    +#if locking tracing is wanted, uncomment the following
    +#echo lockdep-trace >> /etc/modules
    +
    + + +

    Getting and installing the +ltt-control package (on the traced machine)

    +

    +(note : the ltt-control package contains lttd and lttctl. Although it has the +same name as the ltt-control kernel module, they are *not* the same thing.) + +

    +su -
    +cd /usr/src
    +wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
    +gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
    +cd ltt-control-0.x-xxxx2006
    +(refer to README to see the development libraries that must be installed on you
    +system)
    +./configure
    +make
    +make install
    +
    + +

    Userspace tracing

    + +
    +Make sure you selected the kernel menuconfig option :
    +     or <*> Support logging events from userspace
    +And that the ltt-userspace-event kernel module is loaded if selected as a
    +module.
    +
    +Simple userspace tracing is available through
    +echo "some text to record" > /mnt/debugfs/ltt/write_event
    +
    +It will appear in the trace under event :
    +channel : userspace
    +event name : event
    +
    + +

    Getting and installing the LTTV package +(on the visualisation machine, same +or different from the visualisation machine)

    + +
    +su -
    +cd /usr/src
    +wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
    +gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
    +cd lttv-0.x.xx-xxxx2008
    +(refer to README to see the development libraries that must be installed on your
    +system)
    +./configure
    +make
    +make install
    +
    + +
    + + +

    Using LTTng and LTTV

    + +
  • IMPORTANT : Arm Linux Kernel Markers after each boot before tracing
  • +
    +ltt-armall
    +
    + +

    Use graphical LTTV to control +tracing and analyse traces

    +
    +lttv-gui (or /usr/local/bin/lttv-gui)
    +  - Spot the "Tracing Control" icon : click on it
    +      (it's a traffic light icon)
    +    - enter the root password
    +    - click "start"
    +    - click "stop"
    +    - Yes
    +      * You should now see a trace
    +
    + +

    Use text mode LTTng to control tracing

    +
    +The tracing can be controlled from a terminal by using the lttctl command (as
    +root).
    +
    +Start tracing :
    +
    +lttctl -C -w /tmp/trace1 trace1
    +
    +Stop tracing and destroy trace channels :
    +
    +lttctl -D trace1
    +
    +see lttctl --help for details.
    +
    +

    +(note : to see if the buffers has been filled, look at the dmesg output after +lttctl -D or after stopping tracing from the GUI, it will show an event lost +count. If it is the case, try using larger buffers. See lttctl --help to learn +how. lttv now also shows event lost messages in the console when loading a trace +with missing events or lost subbuffers.) + +

    Use text mode LTTV

    +

    +Feel free to look in /usr/local/lib/lttv/plugins to see all the text and +graphical plugins available. +

    +For example, a simple trace dump in text format is available with : +

    +lttv -m textDump -t /tmp/trace
    +
    +

    +See lttv -m textDump --help for detailed command line options of textDump. +

    +It is, in the current state of the project, very useful to use "grep" on the +text output to filter by specific event fields. You can later copy the timestamp +of the events to the clipboard and paste them in the GUI by clicking on the +bottom right label "Current time". Support for this type of filtering should +be added to the filter module soon. + +

    Tracing in "Hybrid" mode

    +

    +Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used : +hybrid. It can be especially useful when studying big workloads on a long period +of time. +

    +When using this mode, the most important, low rate control information will be +recorded during all the trace by lttd (i.e. process creation/exit). The high +rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a +flight recorder buffer (now named flight-channelname_X). +

    +The following lttctl commands take an hybrid trace : +

    +Create trace channel, start lttd on normal channels, start tracing: +

    +lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
    +
    +

    +Stop tracing, start lttd on flight recorder channels, destroy trace channels : +

    +lttctl -D -w /tmp/trace2 trace2
    +
    +

    +Each "overwrite" channel is flight recorder channel. + + +

    Tracing in flight recorder mode

    +
  • Flight recorder mode
  • +

    +The flight recorder mode writes data into overwritten buffers for all channels, +including control channels, except for the facilities tracefiles. It consists of +setting all channels to "overwrite". +

    +The following lttctl commands take a flight recorder trace : +

    +lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
    +...
    +lttctl -D -w /tmp/trace3 trace3
    +
    + +
    + + +

    Adding new instrumentations with the +markers

    +

    + +

    Adding kernel +instrumentation

    + +

    +See Documentation/markers.txt +and Documentation/tracepoints.txt in your kernel +tree. +

    +Also see ltt/probes/ +for LTTng probe examples. + +

    Adding userspace instrumentation

    + +Add new events to userspace programs with +userspace markers packages. +Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It +allows inserting markers in executables and libraries, currently only on x86_32 +and x86_64. +See markers-userspace-0.5.tar.bz2 or more recent. + +

    +Note that a new design document for a 3rd generation of tracepoint/marker-based +userspace tracing is available at LTTng User-space Tracing +Design. This new infrastructure is not yet implemented. + +

    +The easy quick-and-dirty way to perform userspace tracing is currently to write +an string to /mnt/debugfs/ltt/write_event. See Userspace tracing in the +installation for sources section of this document. + +


    + +

    Creating Debian or RPM packages

    +

    + +

    Create custom LTTV Debian packages

    + +
    +Use : dpkg-buildpackage -rfakeroot
    +
    +

    +You should then have your LTTV .deb files created for your architecture. + +

    Create custom LTTng packages

    +

    +For building LTTng Debian packages : +get the build tree with patches applies as explained in section 2. + +

    +make menuconfig (or xconfig or config) (customize your configuration)
    +make-kpkg kernel_image
    +
    +

    +You will then see your freshly created .deb in /usr/src. Install it with +

    +dpkg -i /usr/src/(image-name).deb
    +
    +

    +Then, follow the section "Editing the system wide configuration" in section 2. + +


    + +

    Examples of LTTng use in the field

    +

    +A few examples of successful LTTng users : + +

    + + diff --git a/trunk/lttv/Makefile.am b/trunk/lttv/Makefile.am index 01e2baf9..48869279 100644 --- a/trunk/lttv/Makefile.am +++ b/trunk/lttv/Makefile.am @@ -3,4 +3,4 @@ SUBDIRS = ltt lttv doc -EXTRA_DIST = QUICKSTART.html +EXTRA_DIST = LTTngManual.html diff --git a/trunk/lttv/QUICKSTART b/trunk/lttv/QUICKSTART index 639c72d2..130f3ba5 100644 --- a/trunk/lttv/QUICKSTART +++ b/trunk/lttv/QUICKSTART @@ -1,7 +1,7 @@ Linux Trace Toolkit Quickstart ------------------------------ Document officially replaced by -http://ltt.polymtl.ca/svn/trunk/lttv/QUICKSTART.html +http://ltt.polymtl.ca/svn/trunk/lttv/LTTngManual.html as of January 21st, 2009. diff --git a/trunk/lttv/QUICKSTART.html b/trunk/lttv/QUICKSTART.html deleted file mode 100644 index 31c6bbae..00000000 --- a/trunk/lttv/QUICKSTART.html +++ /dev/null @@ -1,620 +0,0 @@ - - - - Linux Trace Toolkit Next Generation Manual - - - -

    Linux Trace Toolkit Next Generation Manual

    - -Author : Mathieu Desnoyers, September 2005
    -Last update : January 21st, 2009
    -(originally known as the LTTng QUICKSTART guide) - -

    Table of Contents

    - - - -
    - -

    Introduction

    -

    -This document is made of five parts : the first one explains how -to install LTTng and LTTV from sources, the second one describes the steps -to follow to trace a system and view it. The third part explains -briefly how to add a new trace point to the kernel and to user space -applications. The fourth part explains how to create Debian or RPM -packages from the LTTng and LTTV sources. The fifth and last part describes use -of LTTng in the field. -

    -These operations are made for installing the LTTng 0.86 tracer on a linux 2.6.X -kernel. You will also find instructions for installation of LTTV 0.12.x : the -Linux Trace Toolkit Viewer. -To see the list of compatibilities between LTTng, ltt-control, LTTV, please -refer to : -LTTng+LTTV versions compatibility - -The ongoing work had the Linux Kernel Markers integrated in the mainline Linux -kernel since Linux 2.6.24 and the Tracepoints since 2.6.28. In its current -state, the lttng patchset is necessary to have the trace clocksource, the -instrumentation and the LTTng high-speed data extraction mechanism added to the -kernel. - -
    -
    -

    Supported architectures

    -
    -LTTng :
    -
    -
  • x86 32/64 bits -
  • PowerPC 32 and 64 bits -
  • ARM (with limited timestamping precision, e.g. 1HZ. Need -architecture-specific support for better precision) -
  • MIPS -
  • sh (partial architecture-specific instrumentation) -
  • sparc64 (partial architecture-specific instrumentation) -
  • s390 (partial architecture-specific instrumentation) -
  • Other architectures supported without architecture-specific instrumentation -and with low-resolution timestamps.
    -
    -
    -LTTV :
    -
    -
  • Intel 32/64 bits -
  • PowerPC 32 and 64 bits -
  • Possibly others. Takes care of endianness and type size difference between -the LTTng traces and the LTTV analysis tool. - -
    - - -

    Installation from sources

    -

    - -

    Prerequisites

    - - -
  • Reminder
  • - -

    -See the list of compatibilities between LTTng, ltt-control and LTTV at : -LTTng+LTTV -versions compatibility. - - -

    Getting the LTTng packages

    - -
    -su -
    -mkdir /usr/src/lttng
    -cd /usr/src/lttng
    -(see http://ltt.polymtl.ca/lttng for package listing)
    -wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
    -bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
    -
    - - -

    Getting LTTng kernel sources

    - -
    -su -
    -cd /usr/src
    -wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
    -bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
    -cd linux-2.6.X
    -- For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
    -- For LTTng 0.9.5+ apply the patches in the order specified in the series file,
    -  or use quilt
    -cd ..
    -mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
    -
    - - -

    Installing a LTTng kernel

    - -
    -su -
    -cd /usr/src/linux-2.6.X-lttng-0.x.xx
    -make menuconfig (or make xconfig or make config)
    -    Select the < Help > button if you are not familiar with kernel
    -    configuration.
    -    Items preceded by [*] means they has to be built into the kernel.
    -    Items preceded by [M] means they has to be built as modules.
    -    Items preceded by [ ] means they should be removed.
    -  go to the "General setup" section
    -    Select the following options :
    -    [*] Prompt for development and/or incomplete code/drivers
    -    [*] Activate markers
    -    [*] Activate userspace markers ABI (experimental, optional)
    -    [*] Immediate value optimization (optional)
    -    [*] Linux Trace Toolkit Next Generation (LTTng) --->
    -     or <*> Compile lttng tracing probes
    -     or <*> Linux Trace Toolkit High-speed Lockless Data Relay
    -     or <*> Linux Trace Toolkit Lock-Protected Data Relay
    -     or <*> Linux Trace Toolkit Serializer
    -     or <*>   Linux Trace Toolkit Marker Control
    -     or <*>     Linux Trace Toolkit Tracer
    -    [*] Align Linux Trace Toolkit Traces
    -     or <*> Support logging events from userspace
    -    [*] Support trace extraction from crash dump
    -     or <*> Linux Trace Toolkit Trace Controller
    -     or <*> Linux Trace Toolkit State Dump
    -  Select 
    -  Select 
    -  Select 
    -make
    -make modules_install
    -(if necessary, create a initrd with mkinitrd or your preferate alternative)
    -(mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
    -
    --- on X86, X86_64
    -make install
    -reboot
    -Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
    -
    --- on PowerPC
    -cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
    -cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
    -cp .config /boot/config-2.6.X-lttng-0.x.xx
    -depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
    -mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
    -(edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
    -that comes first is the default kernel)
    -ybin
    -select the right entry at the yaboot prompt (see choices : tab, select : type
    -the kernel name followed by enter)
    -Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
    ---
    -
    - -

    Editing the system wide -configuration

    - -

    -You must activate debugfs and specify a mount point. This is typically done in -fstab such that it happens at boot time. If you have never used DebugFS before, -these operation would do this for you : - -

    -mkdir /mnt/debugfs
    -cp /etc/fstab /etc/fstab.lttng.bkp
    -echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
    -
    - -

    -then, rebooting or issuing the following command will activate debugfs : -

    -mount /mnt/debugfs
    -
    - -

    -You need to load the LTT modules to be able to control tracing from user -space. This is done by issuing the following commands. Note however -these commands load all LTT modules. Depending on what options you chose to -compile statically, you may not need to issue all these commands. - -

    -modprobe ltt-trace-control
    -modprobe ltt-marker-control
    -modprobe ltt-tracer
    -modprobe ltt-serialize
    -modprobe ltt-relay
    -modprobe ipc-trace
    -modprobe kernel-trace
    -modprobe mm-trace
    -modprobe net-trace
    -modprobe fs-trace
    -modprobe jbd2-trace
    -modprobe ext4-trace
    -modprobe syscall-trace
    -modprobe trap-trace
    -#if locking tracing is wanted, uncomment the following
    -#modprobe lockdep-trace
    -
    - -

    -If you want to have complete information about the kernel state (including all -the process names), you need to load the ltt-statedump module. This is done by -issuing the command : - -

    -modprobe ltt-statedump
    -
    -

    -You can automate at boot time loading the ltt-control module by : - -

    -cp /etc/modules /etc/modules.bkp
    -echo ltt-trace-control >> /etc/modules
    -echo ltt-marker-control >> /etc/modules
    -echo ltt-tracer >> /etc/modules
    -echo ltt-serialize >> /etc/modules
    -echo ltt-relay >> /etc/modules
    -echo ipc-trace >> /etc/modules
    -echo kernel-trace >> /etc/modules
    -echo mm-trace >> /etc/modules
    -echo net-trace >> /etc/modules
    -echo fs-trace >> /etc/modules
    -echo jbd2-trace >> /etc/modules
    -echo ext4-trace >> /etc/modules
    -echo syscall-trace >> /etc/modules
    -echo trap-trace >> /etc/modules
    -#if locking tracing is wanted, uncomment the following
    -#echo lockdep-trace >> /etc/modules
    -
    - - -

    Getting and installing the -ltt-control package (on the traced machine)

    -

    -(note : the ltt-control package contains lttd and lttctl. Although it has the -same name as the ltt-control kernel module, they are *not* the same thing.) - -

    -su -
    -cd /usr/src
    -wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
    -gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
    -cd ltt-control-0.x-xxxx2006
    -(refer to README to see the development libraries that must be installed on you
    -system)
    -./configure
    -make
    -make install
    -
    - -

    Userspace tracing

    - -
    -Make sure you selected the kernel menuconfig option :
    -     or <*> Support logging events from userspace
    -And that the ltt-userspace-event kernel module is loaded if selected as a
    -module.
    -
    -Simple userspace tracing is available through
    -echo "some text to record" > /mnt/debugfs/ltt/write_event
    -
    -It will appear in the trace under event :
    -channel : userspace
    -event name : event
    -
    - -

    Getting and installing the LTTV package -(on the visualisation machine, same -or different from the visualisation machine)

    - -
    -su -
    -cd /usr/src
    -wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
    -gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
    -cd lttv-0.x.xx-xxxx2008
    -(refer to README to see the development libraries that must be installed on your
    -system)
    -./configure
    -make
    -make install
    -
    - -
    - - -

    Using LTTng and LTTV

    - -
  • IMPORTANT : Arm Linux Kernel Markers after each boot before tracing
  • -
    -ltt-armall
    -
    - -

    Use graphical LTTV to control -tracing and analyse traces

    -
    -lttv-gui (or /usr/local/bin/lttv-gui)
    -  - Spot the "Tracing Control" icon : click on it
    -      (it's a traffic light icon)
    -    - enter the root password
    -    - click "start"
    -    - click "stop"
    -    - Yes
    -      * You should now see a trace
    -
    - -

    Use text mode LTTng to control tracing

    -
    -The tracing can be controlled from a terminal by using the lttctl command (as
    -root).
    -
    -Start tracing :
    -
    -lttctl -C -w /tmp/trace1 trace1
    -
    -Stop tracing and destroy trace channels :
    -
    -lttctl -D trace1
    -
    -see lttctl --help for details.
    -
    -

    -(note : to see if the buffers has been filled, look at the dmesg output after -lttctl -D or after stopping tracing from the GUI, it will show an event lost -count. If it is the case, try using larger buffers. See lttctl --help to learn -how. lttv now also shows event lost messages in the console when loading a trace -with missing events or lost subbuffers.) - -

    Use text mode LTTV

    -

    -Feel free to look in /usr/local/lib/lttv/plugins to see all the text and -graphical plugins available. -

    -For example, a simple trace dump in text format is available with : -

    -lttv -m textDump -t /tmp/trace
    -
    -

    -See lttv -m textDump --help for detailed command line options of textDump. -

    -It is, in the current state of the project, very useful to use "grep" on the -text output to filter by specific event fields. You can later copy the timestamp -of the events to the clipboard and paste them in the GUI by clicking on the -bottom right label "Current time". Support for this type of filtering should -be added to the filter module soon. - -

    Tracing in "Hybrid" mode

    -

    -Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used : -hybrid. It can be especially useful when studying big workloads on a long period -of time. -

    -When using this mode, the most important, low rate control information will be -recorded during all the trace by lttd (i.e. process creation/exit). The high -rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a -flight recorder buffer (now named flight-channelname_X). -

    -The following lttctl commands take an hybrid trace : -

    -Create trace channel, start lttd on normal channels, start tracing: -

    -lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
    -
    -

    -Stop tracing, start lttd on flight recorder channels, destroy trace channels : -

    -lttctl -D -w /tmp/trace2 trace2
    -
    -

    -Each "overwrite" channel is flight recorder channel. - - -

    Tracing in flight recorder mode

    -
  • Flight recorder mode
  • -

    -The flight recorder mode writes data into overwritten buffers for all channels, -including control channels, except for the facilities tracefiles. It consists of -setting all channels to "overwrite". -

    -The following lttctl commands take a flight recorder trace : -

    -lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
    -...
    -lttctl -D -w /tmp/trace3 trace3
    -
    - -
    - - -

    Adding new instrumentations with the -markers

    -

    - -

    Adding kernel -instrumentation

    - -

    -See Documentation/markers.txt -and Documentation/tracepoints.txt in your kernel -tree. -

    -Also see ltt/probes/ -for LTTng probe examples. - -

    Adding userspace instrumentation

    - -Add new events to userspace programs with -userspace markers packages. -Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It -allows inserting markers in executables and libraries, currently only on x86_32 -and x86_64. -See markers-userspace-0.5.tar.bz2 or more recent. - -

    -Note that a new design document for a 3rd generation of tracepoint/marker-based -userspace tracing is available at LTTng User-space Tracing -Design. This new infrastructure is not yet implemented. - -

    -The easy quick-and-dirty way to perform userspace tracing is currently to write -an string to /mnt/debugfs/ltt/write_event. See Userspace tracing in the -installation for sources section of this document. - -


    - -

    Creating Debian or RPM packages

    -

    - -

    Create custom LTTV Debian packages

    - -
    -Use : dpkg-buildpackage -rfakeroot
    -
    -

    -You should then have your LTTV .deb files created for your architecture. - -

    Create custom LTTng packages

    -

    -For building LTTng Debian packages : -get the build tree with patches applies as explained in section 2. - -

    -make menuconfig (or xconfig or config) (customize your configuration)
    -make-kpkg kernel_image
    -
    -

    -You will then see your freshly created .deb in /usr/src. Install it with -

    -dpkg -i /usr/src/(image-name).deb
    -
    -

    -Then, follow the section "Editing the system wide configuration" in section 2. - -


    - -

    Examples of LTTng use in the field

    -

    -A few examples of successful LTTng users : - -

    - -