b1053af5 |
1 | <?xml version="1.0" encoding="UTF-8" ?> |
2 | <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" |
3 | "/usr/share/sgml/docbook/dtd/4.3/xdocbook.dtd"> |
4 | <!--<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" >--> |
5 | |
6 | <book> |
7 | |
8 | <bookinfo> |
9 | <title>Linux Trace Toolkit Viewer User Guide</title> |
10 | <authorgroup> |
11 | <author> |
12 | <firstname>Mathieu</firstname> |
13 | <surname>Desnoyers</surname> |
14 | </author> |
15 | </authorgroup> |
16 | |
54bfc655 |
17 | <date>11/01/2006</date> |
18 | <releaseinfo>1.00.02</releaseinfo> |
b1053af5 |
19 | |
20 | <abstract> |
21 | <para> |
22 | This document describes how to install <application>Linux Trace |
23 | Toolkit Viewer</application> and how to use it. |
24 | |
25 | </para> |
26 | </abstract> |
27 | |
28 | <keywordset> |
29 | <keyword>Linux Trace Toolkit Viewer</keyword> |
30 | <keyword>Linux Trace Toolkit</keyword> |
31 | <keyword>tracing</keyword> |
32 | <keyword>Linux</keyword> |
33 | <keyword>visualization</keyword> |
34 | <keyword>operating system</keyword> |
35 | </keywordset> |
36 | |
37 | </bookinfo> |
38 | |
39 | <chapter> |
40 | <title>Introduction</title> |
41 | <para> |
42 | Linux Trace Toolkit (LTT) is a tracing tool that permits to get all the possible |
43 | execution information from the Linux Kernel. It is based on kernel |
44 | instrumentation and a high-speed relay file system to copy the information from |
45 | the kernel space to the user space. |
46 | </para> |
47 | |
48 | <para> |
49 | Linux Trace Toolkit Viewer (LTTV) is the second generation of visualization |
54bfc655 |
50 | tool. It is based on a trace format (the files where the data is recorded on |
51 | disk) written by the LTTng tracer. |
b1053af5 |
52 | </para> |
53 | |
54 | <para> |
55 | This document explains all the steps that are necessary in order to record a |
56 | trace with LTT and view it with LTTV. |
57 | </para> |
58 | </chapter> |
59 | |
60 | <chapter> |
61 | <title>Getting started</title> |
62 | |
63 | <sect1 id="install"> |
54bfc655 |
64 | <title>Installing LTTng and LTTV</title> |
65 | <para> |
66 | Follow the QUICKSTART guide found at |
67 | <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink>. |
68 | </para> |
69 | |
70 | <!-- |
b1053af5 |
71 | <title>Installing LTTV</title> |
72 | <para> |
73 | First, you must download the latests version of LTTV. You should get it from |
74 | this site : <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink>. |
75 | I suggest that you get it from the "Packages" section. |
76 | </para> |
77 | |
78 | <para> |
79 | You need a recent gcc compiler to compile the project. You might want to use gcc |
80 | 3.2 or newer. |
81 | You will also need some libraries in order to compile it. They are described in |
82 | the README of the LTTV package. These are GTK 2.0, GLIB 2.0, "popt" and Pango 1.0. |
83 | Install them if they are not on your system. Remember that if you use a package |
84 | manager from you favourite Linux distribution, you will need to specifically |
85 | install the librairies'development packages. |
86 | </para> |
87 | |
88 | |
89 | <para> |
90 | Then, you are ready to compile LTTV. Extract and untar the file you previously |
91 | downloaded : |
92 | </para> |
93 | |
94 | <screen> |
95 | <prompt>$</prompt> <userinput>tar -xvzof LinuxTraceToolkitViewer-x.x-dddddddd.tar.bz2</userinput> |
96 | </screen> |
97 | |
98 | <para> |
99 | Then, go to the directory newly created, and type : |
100 | </para> |
101 | |
102 | <screen> |
103 | <prompt>$</prompt> <userinput>./configure</userinput> |
104 | <prompt>$</prompt> <userinput>make</userinput> |
105 | <prompt>#</prompt> <userinput>make install</userinput> (as root) |
106 | </screen> |
54bfc655 |
107 | --> |
b1053af5 |
108 | <para> |
109 | At this point, LTTV is installed in the default directory. You may find the |
110 | lttv executable in /usr/local/bin and the librairies in /usr/local/lib. You will |
111 | also notice the presence of the convert executable in /usr/local/bin. This tool |
112 | will be used later to convert from the Linux Trace Toolkit trace format to the |
113 | LTTV format. |
114 | </para> |
54bfc655 |
115 | <!-- |
b1053af5 |
116 | <para> |
117 | You are now ready to go to the next step : installing the LTT kernel tracer. |
118 | </para> |
119 | |
120 | |
121 | </sect1> |
122 | |
123 | |
124 | |
125 | <sect1 id="install-tracer"> |
126 | <title>Installing LTT kernel tracer</title> |
127 | <para> |
128 | The goal of this guide is not to describe the Linux Trace Toolkit project in |
129 | details, as it is a |
130 | seperate project for now. It just gives pointers to the basic steps you must |
131 | take in order to generate a trace suitable for conversion. |
132 | </para> |
133 | |
134 | <para> |
135 | First, go to the <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink> |
136 | website, in the "Patches for the Official LTT" section. Use the latest version |
137 | of patches available. The file name convention used goes like this : |
54bfc655 |
138 | aaaaaa-x.x\-\-bbbbb-y.y.patch. That means a patch made for aaaaa, release x.x, |
139 | that adds bbbbb, release y.y to it. Notice the presence of the \-\- sign that |
b1053af5 |
140 | separates the "from" field from the name of the patch applied. This way, it's |
141 | impossible to be mixed up on the specific sequence of patch application. I |
142 | suggest that you use the "relayfs", "ltt" and then "md" patches. The "md" patch |
143 | adds events useful to LTTV that are not in the official LTT. |
144 | </para> |
145 | |
146 | <para> |
147 | Once you have the patches you need, get the matching Linux kernel version, apply |
148 | the patches on it, configure it, install it, reboot with the new kernel. You then |
149 | have an instrumented kernel ready for tracing. If you have problems during this phase, |
150 | please refer to <ulink |
151 | url="http://www.opersys.com/ltt">www.opersys.com/ltt</ulink>. If you need |
152 | instructions about how to recompile a kernel, see |
153 | <ulink url="http://www.tldp.org/HOWTO/Kernel-HOWTO/">Kernel-HOWTO</ulink>. |
154 | </para> |
155 | |
156 | </sect1> |
157 | |
158 | <sect1 id="install-daemon"> |
159 | <title>Installing LTT trace recording daemon</title> |
160 | <para> |
161 | In order to install the LTT trace recording daemon, you should get the latest |
162 | TraceToolkit (or ltt) package from the LTT ftp site. |
163 | Use the link "Official Linux Trace Toolkit Packages" on the |
164 | <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink> webpage to access it. |
165 | As of November 30, 2004, the most recent version is 0.9.6-pre3. |
166 | </para> |
167 | <para> |
168 | Then, you should apply the TraceToolkit patches from the LTTV website related |
169 | to the package version. Get them from the "Patches for the Official LTT" |
170 | section. |
171 | </para> |
172 | <para> |
173 | You are now ready to install the daemon in your system. Please refer to the |
174 | documentation in the package for details. |
175 | </para> |
176 | <para> |
177 | You may now use the following command to record a sample 30 seconds trace in |
178 | your current directory. Command line switches are described on the official |
179 | LTT website. |
180 | </para> |
181 | <screen> |
182 | <prompt>#</prompt><userinput>tracedaemon -ts30 sample.out sample.proc (as root) userinput></userinput> |
183 | </screen> |
184 | </sect1> |
185 | |
186 | |
187 | |
188 | <sect1 id="convert"> |
189 | <title>Conversion from LTT to LTTV trace format</title> |
190 | <para> |
191 | If you used the default directory for installation, you should find the |
192 | conversion tool in /usr/local/bin/convert. Before using it, some other files are |
193 | necessary. You will find them in |
194 | /usr/local/share/LinuxTraceToolkitViewer/convert/. Those are sysInfo and |
195 | core.xml. |
196 | </para> |
197 | <para> |
198 | sysInfo is a script that get informations about the traced computer. It should |
199 | be invoked like this : |
200 | </para> |
201 | <screen> |
202 | <prompt>$</prompt> <userinput>sh /usr/local/LinuxTraceToolkitViewer/convert/sysInfo</userinput> |
203 | </screen> |
204 | <para> |
205 | It creates a file named sysInfo.out. This file has to be present in the current |
206 | directory where the convert tool will be executed. I suggest that you choose a |
207 | destination directory where will be written converted traces right now, put sysInfo.out in it, at |
208 | use it as current directory for running the convert tool. |
209 | </para> |
210 | <para> |
211 | Once the sysInfo.out file is ready and you have a trace ready for conversion, |
212 | you should invoke convert like the following example. This is for a uniprocessor |
213 | computer. If you whish to get detailed explanation on the parameters, simply |
214 | execute the convert tool without any option. You may also wish to see the |
215 | /usr/local/LinuxTraceToolkitViewer/convert/README file. |
216 | </para> |
217 | <screen> |
218 | <prompt>$</prompt> <userinput>/usr/local/bin/convert sample.proc 1 sample.trace sample.converted</userinput> |
219 | </screen> |
220 | <para> |
221 | You must then copy the core event definition file to the converted trace directory : |
222 | </para> |
223 | <screen> |
224 | <prompt>$</prompt> <userinput>cp /usr/local/share/LinuxTraceToolkitViewer/convert/core.xml sample.converted/</userinput> |
225 | </screen> |
226 | <para> |
227 | You now have a converted trace ready for visualization in LTTV. Congratulations! |
228 | </para> |
54bfc655 |
229 | --> |
b1053af5 |
230 | </sect1> |
231 | |
232 | <sect1 id="running"> |
233 | <title>Running the executable with basic libraries</title> |
234 | <para> |
235 | Starting the graphical mode with the basic viewer activated is as simple as : |
236 | </para> |
237 | <screen> |
54bfc655 |
238 | <prompt>$</prompt> <userinput>lttv-gui</userinput> |
b1053af5 |
239 | </screen> |
240 | <para> |
241 | Using the text mode is very simple too. Look in /usr/local/lib/lttv/plugins for |
242 | the list of modules. You may use the --help switch to get basic help on the |
243 | command line parameters of every loaded modules. To simply output the events of |
244 | a trace in a text file, try the textDump module. The batchAnalysis module |
245 | permits to do batch mode analysis (state and statistics calculation ) on a |
246 | trace. |
247 | </para> |
248 | <screen> |
249 | <prompt>$</prompt> <userinput>lttv -L /usr/local/lib/lttv/plugins -m textDump --help</userinput> |
250 | </screen> |
251 | </sect1> |
252 | </chapter> |
253 | |
254 | <chapter> |
255 | <title>Using LTTV graphical interface</title> |
256 | |
257 | <sect1 id="mainwindow"> |
258 | <title>LTTV main window</title> |
259 | <para> |
260 | This section describes the main functionnalities that are provided by the LTTV |
261 | GUI and how to use them. |
262 | </para> |
263 | <para> |
264 | By default, when the lttv GUI starts with all the graphical modules loaded, |
265 | it loads the statistics viewer, the control flow viewer, and the detailed event |
266 | list inside a tab. Other viewers can be added later to this tab by interacting |
267 | with the main window. Let's describe the operations available on the window : |
268 | </para> |
269 | <screenshot> |
270 | <mediaobject> |
271 | <imageobject> |
272 | <imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-5.png" |
273 | format="PNG" align="center"/> |
274 | </imageobject> |
275 | <imageobject> |
276 | <imagedata srccredit="Mathieu Desnoyers, 2004" |
277 | fileref="lttv-numbered-5.eps" |
278 | format="EPS" align="center"/> |
279 | </imageobject> |
280 | <!--<imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-6.svg" |
281 | format="SVG" align="center" scalefit="1"/> |
282 | </imageobject>--> |
283 | <caption><para>Linux Trace Toolkit Viewer GUI</para></caption> |
284 | </mediaobject> |
285 | </screenshot> |
286 | <orderedlist> |
287 | <listitem> |
288 | <para> |
289 | This toolbar allows you to navigate through the basic functionnalities of LTTV. |
290 | The first button opens a new window and the second one, a new tab. You can leave |
291 | your mouse over the buttons to read the information provided by the tooltips. |
292 | </para> |
293 | </listitem> |
294 | <listitem> |
295 | <para> |
296 | This notebook, containing different tabs, lets you select the "Trace Set" you |
297 | want to interact with. A trace set is an aggregation of traces, synchronised in |
298 | time. You may also want to use one tab per viewer by simply cloning the traceset |
299 | to a new tab. This way, you can have vertically stacked viewers in one tab, as |
300 | well as different viewers, independant from the time interval. Note that once |
301 | the Trace Set cloning is done, each trace set becomes completely independant. |
302 | For Traceset cloning, see the File Menu. |
303 | </para> |
304 | </listitem> |
305 | <listitem> |
306 | <para> |
307 | These buttons let you control the computation in progress on a trace. As |
308 | sometimes the computation may last for a while, you may want to stop it, restart |
309 | it from the beginning or simply to continue from where you stopped. This is |
310 | exactly what those three buttons offer you. |
311 | </para> |
312 | </listitem> |
313 | <listitem> |
314 | <para> |
315 | Buttons on the right side of the last spacer are semantically different from the |
316 | others. While the other buttons at the left side of the bar are built in the |
317 | lttv program and let you operate the basic functionnalities, the buttons at the |
318 | right side let you add a viewer to the active Tab. They belong to the |
319 | viewers themselves. The number of buttons that appears there should directly |
320 | depend on the number of viewer's modules loaded. |
321 | </para> |
322 | </listitem> |
323 | <listitem> |
324 | <para> |
325 | This is a tree representing the multiple statistics available for the current |
326 | traceset. This is shown by the guistatistics viewer. |
327 | </para> |
328 | </listitem> |
329 | <listitem> |
330 | <para> |
331 | This is the Y axis of the guicontrolflow viewer. It shows the process list of |
332 | the traced system. You may notice that it grows : it dynamically adds |
333 | process when they appear in the trace. |
334 | </para> |
335 | </listitem> |
336 | <listitem> |
337 | <para> |
338 | This is a (missing) time bar for the X axis. Maybe will it be used for viewer |
339 | specific buttons eventually. Work in progress. |
340 | </para> |
341 | </listitem> |
342 | <listitem> |
343 | <para> |
344 | The is the current time selected. The concept of current event and current time |
345 | selected is synchronised in a Tab for all the viewers. The control flow viewer |
346 | shows it a vertical white dotted line. You move this marker by clicking on the |
347 | background of the process state graph. This graph shows evolution of each |
348 | process's state through time. The meaning of the colors will be explained later. |
349 | </para> |
350 | </listitem> |
351 | <listitem> |
352 | <para> |
353 | This is the details event list. It shown the detailed information about each |
354 | event of the trace. It is synchronised with the current time and current event, |
355 | so selecting an event changes other viewer's current time and reciprocally. |
356 | </para> |
357 | </listitem> |
358 | <listitem> |
359 | <para> |
360 | You can enter the values of start time and end time you wish to see on the |
361 | screen here. It also supports pasting time as text input, simply by clicking of |
362 | the "Time Frame", "start" or "end:" fields. A valid entry consists of any |
363 | digital input separated by any quantity of non digital characters. For example : |
364 | "I start at 356247.124626 and stop at 724524.453455" would be a valid input |
365 | for the "Time Frame" field. |
366 | </para> |
367 | </listitem> |
368 | <listitem> |
369 | <para> |
370 | This horizontal scrollbar modifies the window of time shown by all the viewers |
371 | in the tab. It is linked with the fields below it (described at number 10 and |
372 | 12). Another way to modify the time shown is to use the zoom buttons of the |
373 | toolbar (yes, the ones that looks like magnifying glasses). |
374 | </para> |
375 | </listitem> |
376 | <listitem> |
377 | <para> |
378 | This field works just like the "Time Frame" field. It modifies the current time |
379 | selected by the viewers. For example, changing its value will change the current |
380 | event selected by the detailed events list and the current time selected by the |
381 | control flow viewer. |
382 | </para> |
383 | </listitem> |
384 | </orderedlist> |
385 | </sect1> |
386 | |
387 | <sect1 id="ControlFlowColors"> |
388 | <title>Control Flow View Colors</title> |
389 | <screenshot> |
390 | <mediaobject> |
391 | <imageobject> |
392 | <imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-color-list.png" |
393 | format="PNG" align="center"/> |
394 | </imageobject> |
395 | <imageobject> |
396 | <imagedata srccredit="Mathieu Desnoyers, 2004" |
397 | fileref="lttv-color-list.eps" |
398 | format="EPS" align="center"/> |
399 | </imageobject> |
400 | <!--<imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-6.svg" |
401 | format="SVG" align="center" scalefit="1"/> |
402 | </imageobject>--> |
403 | <caption><para>Control Flow View Color Legend</para></caption> |
404 | </mediaobject> |
405 | </screenshot> |
406 | |
407 | <para> |
408 | Here is a description of the colors used in the control flow view. Each color |
409 | represents a state of the process at a given time. |
410 | </para> |
411 | |
412 | <itemizedlist> |
413 | <listitem> |
414 | <para> |
415 | White : this color is used for process from which state is not known. It may |
416 | happen when you seek quickly at a far time in the trace just after it has been |
417 | launched. At that moment, the precomputed state information is incomplete. The |
418 | "unknown" state is used to identify this. Note that the viewer gets refreshed |
419 | once the precomputation ends. |
420 | </para> |
421 | </listitem> |
422 | <listitem> |
423 | <para> |
424 | Green : This color is only used for process when they are running in user mode. |
425 | That includes execution of all the source code of an executable as well as the |
426 | libraries it uses. |
427 | </para> |
428 | </listitem> |
429 | <listitem> |
430 | <para> |
431 | Pale blue : A process is doing a system call to the kernel, and the mode is |
432 | switched from process limited rights to super user mode. Only code from the |
433 | kernel (including modules) should be run in that state. |
434 | </para> |
435 | </listitem> |
436 | <listitem> |
437 | <para> |
438 | Yellow : The kernel is running a trap that services a fault. The most frequent |
439 | trap is the memory page fault trap : it is called every time a page is missing |
440 | from physical memory. |
441 | </para> |
442 | </listitem> |
443 | <listitem> |
444 | <para> |
445 | Orange : IRQ servicing routine is running. It interrupts the currently running |
446 | process. As the IRQ does not change the currently running process (on some |
447 | architectures it uses the same stack as the process), the IRQ state is shown in |
448 | the state of the process. IRQ can be nested : a higher priority interrupt can |
449 | interrupt a lower priority interrupt. |
450 | </para> |
451 | </listitem> |
452 | <listitem> |
453 | <para> |
28b797b9 |
454 | Pink : SoftIRQ handler is running. A SoftIRQ is normally triggered by an |
455 | interrupt that whishes to have some work done very soon, but not "now". This is |
456 | especially useful, for example, to have the longest part of the network stack |
457 | traversal done : a too long computation in the interrupt handler would increase |
458 | the latency of the system. Therefore, doing the long part of the computation in |
459 | a softirq that will be run just after the IRQ handler exits will permits to do |
460 | this work while interrupts are enabled, without increasing the system latency. |
461 | </para> |
462 | </listitem> |
463 | <listitem> |
464 | <para> |
b1053af5 |
465 | Dark red : A process in that state is waiting for an input/output operation to |
466 | complete before it can continue its execution. |
467 | </para> |
468 | </listitem> |
469 | <listitem> |
470 | <para> |
471 | Dark yellow : A process is ready to run, but waiting to get the CPU (a schedule |
472 | in event). |
473 | </para> |
474 | </listitem> |
475 | <listitem> |
476 | <para> |
477 | Dark purple : A process in zombie state. This state happens when a process |
478 | exits and then waits for the parent to wait for it (wait() or waitpid()). |
479 | </para> |
480 | </listitem> |
481 | <listitem> |
482 | <para> |
483 | Dark green : A process has just been created by its parent and is waiting for |
484 | first scheduling. |
485 | </para> |
486 | </listitem> |
487 | <listitem> |
488 | <para> |
489 | Magenta : The process has exited, but still has the control of the CPU. It may |
490 | happend if it has some tasks to do in the exit system call. |
491 | </para> |
492 | </listitem> |
493 | </itemizedlist> |
494 | </sect1> |
495 | </chapter> |
496 | |
497 | <chapter> |
498 | <title>Using LTTV text modules</title> |
499 | <sect1 id="batchAnalysis"> |
500 | <title>The batch analysis module</title> |
501 | <para> |
502 | This batch analysis module can be invoked like this : |
503 | </para> |
504 | <screen> |
505 | <prompt>$</prompt> <userinput>lttv -L path/to/lib/plugins -m batchAnalysis\ |
506 | -t trace1 -t trace2 ...</userinput> |
507 | </screen> |
508 | <para> |
509 | It permits to call any registered action to perform in batch mode on all the |
510 | trace set, which consists of the traces loaded on the command line. Actions that |
511 | are built in the batchAnalysis module are statistics computation. They can be |
512 | triggered by using the -s (--stats) switch. |
513 | </para> |
514 | <para> |
515 | However, the batchAnalysis module is mostly a backend for every other text |
516 | module that does batch computation over a complete trace set. |
517 | </para> |
518 | </sect1> |
519 | <sect1 id="textDump"> |
520 | <title>The text dump module</title> |
521 | <para> |
522 | The goal of this module is to convert the binary data of the traces into |
523 | a formatted text file. |
524 | </para> |
525 | <para> |
526 | The text dump module is a good example of a usage of the batch analysis module |
527 | backend. In fact, the text dump module depends on it. You don't need to |
528 | explicitly load the batchAnalysis module though, as lttv offers a rich module |
529 | backend that deals with the dependencies, loading the module automatically if |
530 | needed. |
531 | </para> |
532 | <para> |
533 | The text dump module is invoked just like the batchAnalysis module. It adds more |
534 | options that can be specified in argument. You may specify the -o switch for the |
535 | output file name of the text dump. You can enable the output of the field names |
536 | (the identifier of the fields) with the -l switch. The -s switch, for process |
537 | states, is very useful to indicate the state in which the process is when the |
538 | event happens. |
539 | </para> |
540 | <para> |
541 | If you use the --help option on the textDump module, you will see all the detail |
542 | about the switches that can be used to show per cpu statistics and per process |
543 | statistics. You will notice that you can use both the switches for the |
544 | batchAnalysis module and those for textDump. You will also notice that the |
545 | options --process_state (from textDump) and --stats (from batchAnalysis) has the |
546 | same short name "-s". If you choose to invoke this option using the short name, |
547 | it will use the option of the last module loaded just before the -s switch. |
548 | </para> |
549 | <para> |
550 | For exemple, if you load the textDump module with -m textDump, it will first |
551 | load the batchAnalysis module, and then load itself. As it is the last module |
552 | loaded, the -s switch used after it will signify --process_stats. On the other |
553 | hand, if you choose to specify explicitly the loading of both modules like this |
554 | : |
555 | </para> |
556 | <screen> |
557 | <prompt>$</prompt> <userinput>lttv -L path/to/lib/plugins -m batchAnalysis -s\ |
558 | -m textDump -s -t trace</userinput> |
559 | </screen> |
560 | <para> |
561 | The first "-s" will invoke batchAnalysis --stats and the second "-s" will invoke |
562 | textDump --process_state. The list of options generated by --help follows the |
563 | order of registration of the options by the modules, therefore the invocation |
564 | order of the modules. |
565 | </para> |
566 | </sect1> |
567 | |
568 | </chapter> |
569 | |
570 | |
571 | </book> |