b1053af5 |
1 | <?xml version="1.0" encoding="UTF-8" ?> |
2 | <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN" |
3 | "/usr/share/sgml/docbook/dtd/4.3/xdocbook.dtd"> |
4 | <!--<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" >--> |
5 | |
6 | <book> |
7 | |
8 | <bookinfo> |
9 | <title>Linux Trace Toolkit Viewer User Guide</title> |
10 | <authorgroup> |
11 | <author> |
12 | <firstname>Mathieu</firstname> |
13 | <surname>Desnoyers</surname> |
14 | </author> |
15 | </authorgroup> |
16 | |
17 | <date>29/11/2004</date> |
18 | <releaseinfo>1.00.01</releaseinfo> |
19 | |
20 | <abstract> |
21 | <para> |
22 | This document describes how to install <application>Linux Trace |
23 | Toolkit Viewer</application> and how to use it. |
24 | |
25 | </para> |
26 | </abstract> |
27 | |
28 | <keywordset> |
29 | <keyword>Linux Trace Toolkit Viewer</keyword> |
30 | <keyword>Linux Trace Toolkit</keyword> |
31 | <keyword>tracing</keyword> |
32 | <keyword>Linux</keyword> |
33 | <keyword>visualization</keyword> |
34 | <keyword>operating system</keyword> |
35 | </keywordset> |
36 | |
37 | </bookinfo> |
38 | |
39 | <chapter> |
40 | <title>Introduction</title> |
41 | <para> |
42 | Linux Trace Toolkit (LTT) is a tracing tool that permits to get all the possible |
43 | execution information from the Linux Kernel. It is based on kernel |
44 | instrumentation and a high-speed relay file system to copy the information from |
45 | the kernel space to the user space. |
46 | </para> |
47 | |
48 | <para> |
49 | Linux Trace Toolkit Viewer (LTTV) is the second generation of visualization |
50 | tool. It is |
51 | based on a trace format (the files where the data is recorded on disk) slightly |
52 | different from LTT. As for now, November 29, 2004, the implementation of the new |
53 | trace format in LTT is still not done, we must use a conversion tool to |
54 | transform the original LTT traces to the new format. |
55 | </para> |
56 | |
57 | <para> |
58 | This document explains all the steps that are necessary in order to record a |
59 | trace with LTT and view it with LTTV. |
60 | </para> |
61 | </chapter> |
62 | |
63 | <chapter> |
64 | <title>Getting started</title> |
65 | |
66 | <sect1 id="install"> |
67 | <title>Installing LTTV</title> |
68 | <para> |
69 | First, you must download the latests version of LTTV. You should get it from |
70 | this site : <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink>. |
71 | I suggest that you get it from the "Packages" section. |
72 | </para> |
73 | |
74 | <para> |
75 | You need a recent gcc compiler to compile the project. You might want to use gcc |
76 | 3.2 or newer. |
77 | You will also need some libraries in order to compile it. They are described in |
78 | the README of the LTTV package. These are GTK 2.0, GLIB 2.0, "popt" and Pango 1.0. |
79 | Install them if they are not on your system. Remember that if you use a package |
80 | manager from you favourite Linux distribution, you will need to specifically |
81 | install the librairies'development packages. |
82 | </para> |
83 | |
84 | |
85 | <para> |
86 | Then, you are ready to compile LTTV. Extract and untar the file you previously |
87 | downloaded : |
88 | </para> |
89 | |
90 | <screen> |
91 | <prompt>$</prompt> <userinput>tar -xvzof LinuxTraceToolkitViewer-x.x-dddddddd.tar.bz2</userinput> |
92 | </screen> |
93 | |
94 | <para> |
95 | Then, go to the directory newly created, and type : |
96 | </para> |
97 | |
98 | <screen> |
99 | <prompt>$</prompt> <userinput>./configure</userinput> |
100 | <prompt>$</prompt> <userinput>make</userinput> |
101 | <prompt>#</prompt> <userinput>make install</userinput> (as root) |
102 | </screen> |
103 | |
104 | <para> |
105 | At this point, LTTV is installed in the default directory. You may find the |
106 | lttv executable in /usr/local/bin and the librairies in /usr/local/lib. You will |
107 | also notice the presence of the convert executable in /usr/local/bin. This tool |
108 | will be used later to convert from the Linux Trace Toolkit trace format to the |
109 | LTTV format. |
110 | </para> |
111 | |
112 | <para> |
113 | You are now ready to go to the next step : installing the LTT kernel tracer. |
114 | </para> |
115 | |
116 | |
117 | </sect1> |
118 | |
119 | |
120 | |
121 | <sect1 id="install-tracer"> |
122 | <title>Installing LTT kernel tracer</title> |
123 | <para> |
124 | The goal of this guide is not to describe the Linux Trace Toolkit project in |
125 | details, as it is a |
126 | seperate project for now. It just gives pointers to the basic steps you must |
127 | take in order to generate a trace suitable for conversion. |
128 | </para> |
129 | |
130 | <para> |
131 | First, go to the <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink> |
132 | website, in the "Patches for the Official LTT" section. Use the latest version |
133 | of patches available. The file name convention used goes like this : |
134 | aaaaaa-x.x--bbbbb-y.y.patch. That means a patch made for aaaaa, release x.x, |
135 | that adds bbbbb, release y.y to it. Notice the presence of the -- sign that |
136 | separates the "from" field from the name of the patch applied. This way, it's |
137 | impossible to be mixed up on the specific sequence of patch application. I |
138 | suggest that you use the "relayfs", "ltt" and then "md" patches. The "md" patch |
139 | adds events useful to LTTV that are not in the official LTT. |
140 | </para> |
141 | |
142 | <para> |
143 | Once you have the patches you need, get the matching Linux kernel version, apply |
144 | the patches on it, configure it, install it, reboot with the new kernel. You then |
145 | have an instrumented kernel ready for tracing. If you have problems during this phase, |
146 | please refer to <ulink |
147 | url="http://www.opersys.com/ltt">www.opersys.com/ltt</ulink>. If you need |
148 | instructions about how to recompile a kernel, see |
149 | <ulink url="http://www.tldp.org/HOWTO/Kernel-HOWTO/">Kernel-HOWTO</ulink>. |
150 | </para> |
151 | |
152 | </sect1> |
153 | |
154 | <sect1 id="install-daemon"> |
155 | <title>Installing LTT trace recording daemon</title> |
156 | <para> |
157 | In order to install the LTT trace recording daemon, you should get the latest |
158 | TraceToolkit (or ltt) package from the LTT ftp site. |
159 | Use the link "Official Linux Trace Toolkit Packages" on the |
160 | <ulink url="http://ltt.polymtl.ca">ltt.polymtl.ca</ulink> webpage to access it. |
161 | As of November 30, 2004, the most recent version is 0.9.6-pre3. |
162 | </para> |
163 | <para> |
164 | Then, you should apply the TraceToolkit patches from the LTTV website related |
165 | to the package version. Get them from the "Patches for the Official LTT" |
166 | section. |
167 | </para> |
168 | <para> |
169 | You are now ready to install the daemon in your system. Please refer to the |
170 | documentation in the package for details. |
171 | </para> |
172 | <para> |
173 | You may now use the following command to record a sample 30 seconds trace in |
174 | your current directory. Command line switches are described on the official |
175 | LTT website. |
176 | </para> |
177 | <screen> |
178 | <prompt>#</prompt><userinput>tracedaemon -ts30 sample.out sample.proc (as root) userinput></userinput> |
179 | </screen> |
180 | </sect1> |
181 | |
182 | |
183 | |
184 | <sect1 id="convert"> |
185 | <title>Conversion from LTT to LTTV trace format</title> |
186 | <para> |
187 | If you used the default directory for installation, you should find the |
188 | conversion tool in /usr/local/bin/convert. Before using it, some other files are |
189 | necessary. You will find them in |
190 | /usr/local/share/LinuxTraceToolkitViewer/convert/. Those are sysInfo and |
191 | core.xml. |
192 | </para> |
193 | <para> |
194 | sysInfo is a script that get informations about the traced computer. It should |
195 | be invoked like this : |
196 | </para> |
197 | <screen> |
198 | <prompt>$</prompt> <userinput>sh /usr/local/LinuxTraceToolkitViewer/convert/sysInfo</userinput> |
199 | </screen> |
200 | <para> |
201 | It creates a file named sysInfo.out. This file has to be present in the current |
202 | directory where the convert tool will be executed. I suggest that you choose a |
203 | destination directory where will be written converted traces right now, put sysInfo.out in it, at |
204 | use it as current directory for running the convert tool. |
205 | </para> |
206 | <para> |
207 | Once the sysInfo.out file is ready and you have a trace ready for conversion, |
208 | you should invoke convert like the following example. This is for a uniprocessor |
209 | computer. If you whish to get detailed explanation on the parameters, simply |
210 | execute the convert tool without any option. You may also wish to see the |
211 | /usr/local/LinuxTraceToolkitViewer/convert/README file. |
212 | </para> |
213 | <screen> |
214 | <prompt>$</prompt> <userinput>/usr/local/bin/convert sample.proc 1 sample.trace sample.converted</userinput> |
215 | </screen> |
216 | <para> |
217 | You must then copy the core event definition file to the converted trace directory : |
218 | </para> |
219 | <screen> |
220 | <prompt>$</prompt> <userinput>cp /usr/local/share/LinuxTraceToolkitViewer/convert/core.xml sample.converted/</userinput> |
221 | </screen> |
222 | <para> |
223 | You now have a converted trace ready for visualization in LTTV. Congratulations! |
224 | </para> |
225 | |
226 | </sect1> |
227 | |
228 | <sect1 id="running"> |
229 | <title>Running the executable with basic libraries</title> |
230 | <para> |
231 | Starting the graphical mode with the basic viewer activated is as simple as : |
232 | </para> |
233 | <screen> |
234 | <prompt>$</prompt> <userinput>lttv -L /usr/local/lib/lttv/plugins -m lttvwindow\ |
235 | -m guievents -m guicontrolflow -m guistatistics -t sample.converted/</userinput> |
236 | </screen> |
237 | <para> |
238 | Using the text mode is very simple too. Look in /usr/local/lib/lttv/plugins for |
239 | the list of modules. You may use the --help switch to get basic help on the |
240 | command line parameters of every loaded modules. To simply output the events of |
241 | a trace in a text file, try the textDump module. The batchAnalysis module |
242 | permits to do batch mode analysis (state and statistics calculation ) on a |
243 | trace. |
244 | </para> |
245 | <screen> |
246 | <prompt>$</prompt> <userinput>lttv -L /usr/local/lib/lttv/plugins -m textDump --help</userinput> |
247 | </screen> |
248 | </sect1> |
249 | </chapter> |
250 | |
251 | <chapter> |
252 | <title>Using LTTV graphical interface</title> |
253 | |
254 | <sect1 id="mainwindow"> |
255 | <title>LTTV main window</title> |
256 | <para> |
257 | This section describes the main functionnalities that are provided by the LTTV |
258 | GUI and how to use them. |
259 | </para> |
260 | <para> |
261 | By default, when the lttv GUI starts with all the graphical modules loaded, |
262 | it loads the statistics viewer, the control flow viewer, and the detailed event |
263 | list inside a tab. Other viewers can be added later to this tab by interacting |
264 | with the main window. Let's describe the operations available on the window : |
265 | </para> |
266 | <screenshot> |
267 | <mediaobject> |
268 | <imageobject> |
269 | <imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-5.png" |
270 | format="PNG" align="center"/> |
271 | </imageobject> |
272 | <imageobject> |
273 | <imagedata srccredit="Mathieu Desnoyers, 2004" |
274 | fileref="lttv-numbered-5.eps" |
275 | format="EPS" align="center"/> |
276 | </imageobject> |
277 | <!--<imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-6.svg" |
278 | format="SVG" align="center" scalefit="1"/> |
279 | </imageobject>--> |
280 | <caption><para>Linux Trace Toolkit Viewer GUI</para></caption> |
281 | </mediaobject> |
282 | </screenshot> |
283 | <orderedlist> |
284 | <listitem> |
285 | <para> |
286 | This toolbar allows you to navigate through the basic functionnalities of LTTV. |
287 | The first button opens a new window and the second one, a new tab. You can leave |
288 | your mouse over the buttons to read the information provided by the tooltips. |
289 | </para> |
290 | </listitem> |
291 | <listitem> |
292 | <para> |
293 | This notebook, containing different tabs, lets you select the "Trace Set" you |
294 | want to interact with. A trace set is an aggregation of traces, synchronised in |
295 | time. You may also want to use one tab per viewer by simply cloning the traceset |
296 | to a new tab. This way, you can have vertically stacked viewers in one tab, as |
297 | well as different viewers, independant from the time interval. Note that once |
298 | the Trace Set cloning is done, each trace set becomes completely independant. |
299 | For Traceset cloning, see the File Menu. |
300 | </para> |
301 | </listitem> |
302 | <listitem> |
303 | <para> |
304 | These buttons let you control the computation in progress on a trace. As |
305 | sometimes the computation may last for a while, you may want to stop it, restart |
306 | it from the beginning or simply to continue from where you stopped. This is |
307 | exactly what those three buttons offer you. |
308 | </para> |
309 | </listitem> |
310 | <listitem> |
311 | <para> |
312 | Buttons on the right side of the last spacer are semantically different from the |
313 | others. While the other buttons at the left side of the bar are built in the |
314 | lttv program and let you operate the basic functionnalities, the buttons at the |
315 | right side let you add a viewer to the active Tab. They belong to the |
316 | viewers themselves. The number of buttons that appears there should directly |
317 | depend on the number of viewer's modules loaded. |
318 | </para> |
319 | </listitem> |
320 | <listitem> |
321 | <para> |
322 | This is a tree representing the multiple statistics available for the current |
323 | traceset. This is shown by the guistatistics viewer. |
324 | </para> |
325 | </listitem> |
326 | <listitem> |
327 | <para> |
328 | This is the Y axis of the guicontrolflow viewer. It shows the process list of |
329 | the traced system. You may notice that it grows : it dynamically adds |
330 | process when they appear in the trace. |
331 | </para> |
332 | </listitem> |
333 | <listitem> |
334 | <para> |
335 | This is a (missing) time bar for the X axis. Maybe will it be used for viewer |
336 | specific buttons eventually. Work in progress. |
337 | </para> |
338 | </listitem> |
339 | <listitem> |
340 | <para> |
341 | The is the current time selected. The concept of current event and current time |
342 | selected is synchronised in a Tab for all the viewers. The control flow viewer |
343 | shows it a vertical white dotted line. You move this marker by clicking on the |
344 | background of the process state graph. This graph shows evolution of each |
345 | process's state through time. The meaning of the colors will be explained later. |
346 | </para> |
347 | </listitem> |
348 | <listitem> |
349 | <para> |
350 | This is the details event list. It shown the detailed information about each |
351 | event of the trace. It is synchronised with the current time and current event, |
352 | so selecting an event changes other viewer's current time and reciprocally. |
353 | </para> |
354 | </listitem> |
355 | <listitem> |
356 | <para> |
357 | You can enter the values of start time and end time you wish to see on the |
358 | screen here. It also supports pasting time as text input, simply by clicking of |
359 | the "Time Frame", "start" or "end:" fields. A valid entry consists of any |
360 | digital input separated by any quantity of non digital characters. For example : |
361 | "I start at 356247.124626 and stop at 724524.453455" would be a valid input |
362 | for the "Time Frame" field. |
363 | </para> |
364 | </listitem> |
365 | <listitem> |
366 | <para> |
367 | This horizontal scrollbar modifies the window of time shown by all the viewers |
368 | in the tab. It is linked with the fields below it (described at number 10 and |
369 | 12). Another way to modify the time shown is to use the zoom buttons of the |
370 | toolbar (yes, the ones that looks like magnifying glasses). |
371 | </para> |
372 | </listitem> |
373 | <listitem> |
374 | <para> |
375 | This field works just like the "Time Frame" field. It modifies the current time |
376 | selected by the viewers. For example, changing its value will change the current |
377 | event selected by the detailed events list and the current time selected by the |
378 | control flow viewer. |
379 | </para> |
380 | </listitem> |
381 | </orderedlist> |
382 | </sect1> |
383 | |
384 | <sect1 id="ControlFlowColors"> |
385 | <title>Control Flow View Colors</title> |
386 | <screenshot> |
387 | <mediaobject> |
388 | <imageobject> |
389 | <imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-color-list.png" |
390 | format="PNG" align="center"/> |
391 | </imageobject> |
392 | <imageobject> |
393 | <imagedata srccredit="Mathieu Desnoyers, 2004" |
394 | fileref="lttv-color-list.eps" |
395 | format="EPS" align="center"/> |
396 | </imageobject> |
397 | <!--<imagedata srccredit="Mathieu Desnoyers, 2004" fileref="lttv-numbered-6.svg" |
398 | format="SVG" align="center" scalefit="1"/> |
399 | </imageobject>--> |
400 | <caption><para>Control Flow View Color Legend</para></caption> |
401 | </mediaobject> |
402 | </screenshot> |
403 | |
404 | <para> |
405 | Here is a description of the colors used in the control flow view. Each color |
406 | represents a state of the process at a given time. |
407 | </para> |
408 | |
409 | <itemizedlist> |
410 | <listitem> |
411 | <para> |
412 | White : this color is used for process from which state is not known. It may |
413 | happen when you seek quickly at a far time in the trace just after it has been |
414 | launched. At that moment, the precomputed state information is incomplete. The |
415 | "unknown" state is used to identify this. Note that the viewer gets refreshed |
416 | once the precomputation ends. |
417 | </para> |
418 | </listitem> |
419 | <listitem> |
420 | <para> |
421 | Green : This color is only used for process when they are running in user mode. |
422 | That includes execution of all the source code of an executable as well as the |
423 | libraries it uses. |
424 | </para> |
425 | </listitem> |
426 | <listitem> |
427 | <para> |
428 | Pale blue : A process is doing a system call to the kernel, and the mode is |
429 | switched from process limited rights to super user mode. Only code from the |
430 | kernel (including modules) should be run in that state. |
431 | </para> |
432 | </listitem> |
433 | <listitem> |
434 | <para> |
435 | Yellow : The kernel is running a trap that services a fault. The most frequent |
436 | trap is the memory page fault trap : it is called every time a page is missing |
437 | from physical memory. |
438 | </para> |
439 | </listitem> |
440 | <listitem> |
441 | <para> |
442 | Orange : IRQ servicing routine is running. It interrupts the currently running |
443 | process. As the IRQ does not change the currently running process (on some |
444 | architectures it uses the same stack as the process), the IRQ state is shown in |
445 | the state of the process. IRQ can be nested : a higher priority interrupt can |
446 | interrupt a lower priority interrupt. |
447 | </para> |
448 | </listitem> |
449 | <listitem> |
450 | <para> |
28b797b9 |
451 | Pink : SoftIRQ handler is running. A SoftIRQ is normally triggered by an |
452 | interrupt that whishes to have some work done very soon, but not "now". This is |
453 | especially useful, for example, to have the longest part of the network stack |
454 | traversal done : a too long computation in the interrupt handler would increase |
455 | the latency of the system. Therefore, doing the long part of the computation in |
456 | a softirq that will be run just after the IRQ handler exits will permits to do |
457 | this work while interrupts are enabled, without increasing the system latency. |
458 | </para> |
459 | </listitem> |
460 | <listitem> |
461 | <para> |
b1053af5 |
462 | Dark red : A process in that state is waiting for an input/output operation to |
463 | complete before it can continue its execution. |
464 | </para> |
465 | </listitem> |
466 | <listitem> |
467 | <para> |
468 | Dark yellow : A process is ready to run, but waiting to get the CPU (a schedule |
469 | in event). |
470 | </para> |
471 | </listitem> |
472 | <listitem> |
473 | <para> |
474 | Dark purple : A process in zombie state. This state happens when a process |
475 | exits and then waits for the parent to wait for it (wait() or waitpid()). |
476 | </para> |
477 | </listitem> |
478 | <listitem> |
479 | <para> |
480 | Dark green : A process has just been created by its parent and is waiting for |
481 | first scheduling. |
482 | </para> |
483 | </listitem> |
484 | <listitem> |
485 | <para> |
486 | Magenta : The process has exited, but still has the control of the CPU. It may |
487 | happend if it has some tasks to do in the exit system call. |
488 | </para> |
489 | </listitem> |
490 | </itemizedlist> |
491 | </sect1> |
492 | </chapter> |
493 | |
494 | <chapter> |
495 | <title>Using LTTV text modules</title> |
496 | <sect1 id="batchAnalysis"> |
497 | <title>The batch analysis module</title> |
498 | <para> |
499 | This batch analysis module can be invoked like this : |
500 | </para> |
501 | <screen> |
502 | <prompt>$</prompt> <userinput>lttv -L path/to/lib/plugins -m batchAnalysis\ |
503 | -t trace1 -t trace2 ...</userinput> |
504 | </screen> |
505 | <para> |
506 | It permits to call any registered action to perform in batch mode on all the |
507 | trace set, which consists of the traces loaded on the command line. Actions that |
508 | are built in the batchAnalysis module are statistics computation. They can be |
509 | triggered by using the -s (--stats) switch. |
510 | </para> |
511 | <para> |
512 | However, the batchAnalysis module is mostly a backend for every other text |
513 | module that does batch computation over a complete trace set. |
514 | </para> |
515 | </sect1> |
516 | <sect1 id="textDump"> |
517 | <title>The text dump module</title> |
518 | <para> |
519 | The goal of this module is to convert the binary data of the traces into |
520 | a formatted text file. |
521 | </para> |
522 | <para> |
523 | The text dump module is a good example of a usage of the batch analysis module |
524 | backend. In fact, the text dump module depends on it. You don't need to |
525 | explicitly load the batchAnalysis module though, as lttv offers a rich module |
526 | backend that deals with the dependencies, loading the module automatically if |
527 | needed. |
528 | </para> |
529 | <para> |
530 | The text dump module is invoked just like the batchAnalysis module. It adds more |
531 | options that can be specified in argument. You may specify the -o switch for the |
532 | output file name of the text dump. You can enable the output of the field names |
533 | (the identifier of the fields) with the -l switch. The -s switch, for process |
534 | states, is very useful to indicate the state in which the process is when the |
535 | event happens. |
536 | </para> |
537 | <para> |
538 | If you use the --help option on the textDump module, you will see all the detail |
539 | about the switches that can be used to show per cpu statistics and per process |
540 | statistics. You will notice that you can use both the switches for the |
541 | batchAnalysis module and those for textDump. You will also notice that the |
542 | options --process_state (from textDump) and --stats (from batchAnalysis) has the |
543 | same short name "-s". If you choose to invoke this option using the short name, |
544 | it will use the option of the last module loaded just before the -s switch. |
545 | </para> |
546 | <para> |
547 | For exemple, if you load the textDump module with -m textDump, it will first |
548 | load the batchAnalysis module, and then load itself. As it is the last module |
549 | loaded, the -s switch used after it will signify --process_stats. On the other |
550 | hand, if you choose to specify explicitly the loading of both modules like this |
551 | : |
552 | </para> |
553 | <screen> |
554 | <prompt>$</prompt> <userinput>lttv -L path/to/lib/plugins -m batchAnalysis -s\ |
555 | -m textDump -s -t trace</userinput> |
556 | </screen> |
557 | <para> |
558 | The first "-s" will invoke batchAnalysis --stats and the second "-s" will invoke |
559 | textDump --process_state. The list of options generated by --help follows the |
560 | order of registration of the options by the modules, therefore the invocation |
561 | order of the modules. |
562 | </para> |
563 | </sect1> |
564 | |
565 | </chapter> |
566 | |
567 | |
568 | </book> |