lttng-modules.git
4 years agofix: mm: remove vmalloc_sync_(un)mappings() (v5.8)
Michael Jeanson [Thu, 2 Jul 2020 15:21:42 +0000 (11:21 -0400)] 
fix: mm: remove vmalloc_sync_(un)mappings() (v5.8)

See upstream commit:

  commit 73f693c3a705756032c2863bfb37570276902d7d
  Author: Joerg Roedel <jroedel@suse.de>
  Date:   Mon Jun 1 21:52:36 2020 -0700

    mm: remove vmalloc_sync_(un)mappings()

    These functions are not needed anymore because the vmalloc and ioremap
    mappings are now synchronized when they are created or torn down.

    Remove all callers and function definitions.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ifdefa35b25b4906cde407360e608b77e47cc3808

4 years agoUpdate design document
Mathieu Desnoyers [Tue, 30 Jun 2020 18:29:01 +0000 (14:29 -0400)] 
Update design document

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoAdd lttng-modules design document
Mathieu Desnoyers [Tue, 30 Jun 2020 18:24:29 +0000 (14:24 -0400)] 
Add lttng-modules design document

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: callstack: initialize nested sequence length field name
Mathieu Desnoyers [Tue, 30 Jun 2020 14:41:37 +0000 (10:41 -0400)] 
Fix: callstack: initialize nested sequence length field name

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: callstack: NULL pointer dereference: length field also need fdata
Mathieu Desnoyers [Tue, 30 Jun 2020 14:29:19 +0000 (10:29 -0400)] 
Fix: callstack: NULL pointer dereference: length field also need fdata

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: callstack context memory corruption
Mathieu Desnoyers [Mon, 29 Jun 2020 23:52:08 +0000 (19:52 -0400)] 
Fix: callstack context memory corruption

commit ceabb767180e "tracepoint: Refactor representation of nested types"
introduces two context fields for callstack contexts. Keeping a pointer
to the first field is not valid when adding the second context field to
the array, because the array is reallocated.

Fix this by introducing new context APIs which operate on indexes rather
than pointers:
- lttng_append_context_index,
- lttng_get_context_field_from_index,
- lttng_remove_context_field_index.

Add a NULL check to lttng_find_context so it can be used before adding
the first context.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK (v5.8)
Michael Jeanson [Mon, 15 Jun 2020 15:12:24 +0000 (11:12 -0400)] 
fix: mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK (v5.8)

See upstream commit:

  commit 8d92890bd6b8502d6aee4b37430ae6444ade7a8c
  Author: NeilBrown <neilb@suse.de>
  Date:   Mon Jun 1 21:48:21 2020 -0700

    mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK instead

    After an NFS page has been written it is considered "unstable" until a
    COMMIT request succeeds.  If the COMMIT fails, the page will be
    re-written.

    These "unstable" pages are currently accounted as "reclaimable", either
    in WB_RECLAIMABLE, or in NR_UNSTABLE_NFS which is included in a
    'reclaimable' count.  This might have made sense when sending the COMMIT
    required a separate action by the VFS/MM (e.g.  releasepage() used to
    send a COMMIT).  However now that all writes generated by ->writepages()
    will automatically be followed by a COMMIT (since commit 919e3bd9a875
    ("NFS: Ensure we commit after writeback is complete")) it makes more
    sense to treat them as writeback pages.

    So this patch removes NR_UNSTABLE_NFS and accounts unstable pages in
    NR_WRITEBACK and WB_WRITEBACK.

    A particular effect of this change is that when
    wb_check_background_flush() calls wb_over_bg_threshold(), the latter
    will report 'true' a lot less often as the 'unstable' pages are no
    longer considered 'dirty' (as there is nothing that writeback can do
    about them anyway).

    Currently wb_check_background_flush() will trigger writeback to NFS even
    when there are relatively few dirty pages (if there are lots of unstable
    pages), this can result in small writes going to the server (10s of
    Kilobytes rather than a Megabyte) which hurts throughput.  With this
    patch, there are fewer writes which are each larger on average.

    Where the NR_UNSTABLE_NFS count was included in statistics
    virtual-files, the entry is retained, but the value is hard-coded as
    zero.  static trace points and warning printks which mentioned this
    counter no longer report it.

Change-Id: I18080ca62bc6c1cd7d6da4cb27cc1521fbdca5e1
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: block: remove the error argument to the block_bio_complete (v5.8)
Michael Jeanson [Mon, 15 Jun 2020 15:06:13 +0000 (11:06 -0400)] 
fix: block: remove the error argument to the block_bio_complete (v5.8)

See upstream commit:

  commit d24de76af836260a99ca2ba281a937bd5bc55591
  Author: Christoph Hellwig <hch@lst.de>
  Date:   Wed Jun 3 07:14:43 2020 +0200

    block: remove the error argument to the block_bio_complete tracepoint

    The status can be trivially derived from the bio itself.  That also avoid
    callers like NVMe to incorrectly pass a blk_status_t instead of the errno,
    and the overhead of translating the blk_status_t to the errno in the I/O
    completion fast path when no tracing is enabled.

Fixes: 35fe0d12c8a3 ("nvme: trace bio completion")
Change-Id: I8d1463184d79bfab418a1755bfc6a0200170fff3
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: pipe_buf_operations rework (v5.8)
Michael Jeanson [Mon, 15 Jun 2020 14:51:41 +0000 (10:51 -0400)] 
fix: pipe_buf_operations rework (v5.8)

See upstream commits:

  commit c928f642c29a5ffb02e16f2430b42b876dde69de
  Author: Christoph Hellwig <hch@lst.de>
  Date:   Wed May 20 17:58:16 2020 +0200

    fs: rename pipe_buf ->steal to ->try_steal

    And replace the arcane return value convention with a simple bool
    where true means success and false means failure.

    [AV: braino fix folded in]

  commit b8d9e7f2411b0744df2ec33e80d7698180fef21a
  Author: Christoph Hellwig <hch@lst.de>
  Date:   Wed May 20 17:58:15 2020 +0200

    fs: make the pipe_buf_operations ->confirm operation optional

    Just return 0 for success if it is not present.

  commit 76887c256744740d6121af9bc4aa787712a1f694
  Author: Christoph Hellwig <hch@lst.de>
  Date:   Wed May 20 17:58:14 2020 +0200

    fs: make the pipe_buf_operations ->steal operation optional

    Just return 1 for failure if it is not present.

Change-Id: Ic185632202470db1eb5b012e95e793ff2cb26be7
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: syscalls: Ignore fcntl cmd specific to 32-bit in 64-bit only config
Ruiqiang Hao [Tue, 26 May 2020 03:36:17 +0000 (03:36 +0000)] 
Fix: syscalls: Ignore fcntl cmd specific to 32-bit in 64-bit only config

When CONFIG_64BIT is defined and CONFIG_COMPAT is not defined, the fcntl system call
"F_GETLK64", "F_SETLK64" and "F_SETLKW64" should be ignored.

Signed-off-by: Ruiqiang Hao <Ruiqiang.Hao@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: Implement RING_BUFFER_GET_NEXT_SUBBUF_METADATA_CHECK
Mathieu Desnoyers [Fri, 24 Apr 2020 19:49:42 +0000 (15:49 -0400)] 
Fix: Implement RING_BUFFER_GET_NEXT_SUBBUF_METADATA_CHECK

Get next metadata subbuffer, returning a flag indicating whether the
metadata is guaranteed to be in a consistent state at the end of this
sub-buffer (can be parsed).

This can be used by the consumer to know whether the metadata can be
parsed at the end of this sub-buffer, which is useful to distinguish
between errors and incomplete metadata in live tracing.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: vmalloc_sync_mappings was backported to v5.5.12
Michael Jeanson [Fri, 15 May 2020 19:12:53 +0000 (15:12 -0400)] 
fix: vmalloc_sync_mappings was backported to v5.5.12

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ie554d9c956afc2a8e114fe41e4b3c225d8af40a1

4 years agoUpdate: Additional kernel ranges for vmalloc_sync_mappings
Stefan Bader [Mon, 18 May 2020 14:03:16 +0000 (16:03 +0200)] 
Update: Additional kernel ranges for vmalloc_sync_mappings

Some Ubuntu kernels cannot be directly mapped to an upstream stable
version. Define distro specific ranges for those (4.15, 5.0, 5.3).

Signed-off-by: Stefan Bader <stefan.bader@canonical.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoUpdate: Use vmalloc_sync_mappings for stable kernels
Ovidiu Panait [Thu, 14 May 2020 11:27:17 +0000 (14:27 +0300)] 
Update: Use vmalloc_sync_mappings for stable kernels

Starting from v5.4.28/v5.2.37/v4.19.113/v4.14.175/v4.9.218/v4.4.218, stable
kernel branches backported v5.6 upstream commit [1], causing the following
warnings:
...
[  483.242037] LTTng: vmalloc_sync_all symbol lookup failed.
[  483.257056] Page fault handler and NMI tracing might trigger faults.
...

Extend check for vmalloc_sync_mappings for stable kernels as well.

[1] https://github.com/torvalds/linux/commit/763802b53a427ed3cbd419dbba255c414fdd9e7c

[ Edit: minor coding style fix by Mathieu Desnoyers. ]

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: Use vmalloc_sync_mappings on kernel 5.6 as well
Ovidiu Panait [Thu, 14 May 2020 10:05:24 +0000 (13:05 +0300)] 
Fix: Use vmalloc_sync_mappings on kernel 5.6 as well

Upstream commit [1], that got rid of vmalloc_sync_all and introduced
vmalloc_sync_mappings, is a v5.6 commit:
$ git tag --contains 763802b53a427ed3cbd419dbba255c414fdd9e7c
v5.6
v5.6-rc7
v5.7-rc1
v5.7-rc2
v5.7-rc3

Extend the LINUX_VERSION_CODE check to v5.6 to fix the following warnings:
...
[  483.242037] LTTng: vmalloc_sync_all symbol lookup failed.
[  483.257056] Page fault handler and NMI tracing might trigger faults.
...

[1] https://github.com/torvalds/linux/commit/763802b53a427ed3cbd419dbba255c414fdd9e7c

Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: remove unsupported `ctf_float()` macros
Francis Deslauriers [Tue, 12 May 2020 19:11:05 +0000 (15:11 -0400)] 
Cleanup: remove unsupported `ctf_float()` macros

Tracing floats is not supported for the kernel tracer. Disallow building
kernel probes with those fields, rather than silently ignoring them.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1cf9751df96d2af3b54f725797bd20d7b05f2b38

4 years agoCleanup: have interpreter functions return _DISCARD instead of 0
Francis Deslauriers [Tue, 12 May 2020 15:48:20 +0000 (11:48 -0400)] 
Cleanup: have interpreter functions return _DISCARD instead of 0

It's easier to understand the meaning of the zero return value of these
function using the enum. It makes it obvious.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1df8f704fa9f6768f413c12c3c1de61a94b3aff8

4 years agoCleanup: bytecode: typo: "s16" -> "u16"
Francis Deslauriers [Mon, 11 May 2020 19:04:43 +0000 (15:04 -0400)] 
Cleanup: bytecode: typo: "s16" -> "u16"

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I68901ca2d89d08f2cb69853816e0214c588aa7f8

4 years agoCleanup: Rename patches.i to patches.h
Mathieu Desnoyers [Thu, 7 May 2020 14:51:03 +0000 (10:51 -0400)] 
Cleanup: Rename patches.i to patches.h

This generated header file contains a list of patches applied on the
lttng-modules tree. Based on the C99 specification, ".i" files are not
supposed to be preprocessed, although this header header file is
expected to be preprocessed.

Rename it from ".i" to ".h" to convey that it is a C header meant to be
preprocessed.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move all source files to src/
Mathieu Desnoyers [Wed, 6 May 2020 18:08:22 +0000 (14:08 -0400)] 
Cleanup: Move all source files to src/

This includes *.c, lib/*/*.c, probes/*.c, wrapper/*.c.

Adapt Makefile and Kbuild accordingly. Introduce src/Kbuild.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move patches.i to include/generated/
Michael Jeanson [Wed, 6 May 2020 18:26:01 +0000 (14:26 -0400)] 
Cleanup: Move patches.i to include/generated/

Move patches.i from /extra_version to include/generated/ so we
can include them without using relative path includes.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I87927a372ffeb244f3c097c9bb80eeca7d9f99eb

4 years agoCleanup: Move lttng-modules instrumentation headers
Mathieu Desnoyers [Wed, 6 May 2020 17:44:57 +0000 (13:44 -0400)] 
Cleanup: Move lttng-modules instrumentation headers

The directory hierarchy "instrumentation/events/lttng-module/" only
exists for historical reasons and is not needed anymore. Move all
its contents into "instrumentation/events/".

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Remove toplevel directory from include search path
Mathieu Desnoyers [Wed, 6 May 2020 17:39:19 +0000 (13:39 -0400)] 
Cleanup: Remove toplevel directory from include search path

Now that all include files are moved to include/ (except for those
meant to be included with "#include "...h"), we can remove the toplevel
directory from the include search path.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move blacklist/ headers to include/blacklist/
Mathieu Desnoyers [Wed, 6 May 2020 17:38:49 +0000 (13:38 -0400)] 
Cleanup: Move blacklist/ headers to include/blacklist/

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move wrapper/ headers to include/wrapper/
Mathieu Desnoyers [Wed, 6 May 2020 17:35:50 +0000 (13:35 -0400)] 
Cleanup: Move wrapper/ headers to include/wrapper/

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move instrumentation/ headers to include/instrumentation/
Mathieu Desnoyers [Wed, 6 May 2020 17:34:11 +0000 (13:34 -0400)] 
Cleanup: Move instrumentation/ headers to include/instrumentation/

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Remove deprecated TODO file
Mathieu Desnoyers [Wed, 6 May 2020 17:15:13 +0000 (13:15 -0400)] 
Cleanup: Remove deprecated TODO file

All relevant items that were left were moved to
https://bugs.lttng.org/projects/lttng-modules "Feature".

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: add missing guid_t type to wrapper
Michael Jeanson [Wed, 6 May 2020 15:11:29 +0000 (11:11 -0400)] 
fix: add missing guid_t type to wrapper

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0de39c24a7925b580fabbdaa12dbe05c43cfcd98

4 years agoFix: missing wrapper rename to wrapper_vmalloc_sync_mappings
Michael Jeanson [Wed, 6 May 2020 15:03:32 +0000 (11:03 -0400)] 
Fix: missing wrapper rename to wrapper_vmalloc_sync_mappings

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Idf7082a980c5a604bfef5c69906678b5083a9bbf

4 years agoCleanup: Move headers from toplevel to include/lttng/
Mathieu Desnoyers [Wed, 6 May 2020 14:18:46 +0000 (10:18 -0400)] 
Cleanup: Move headers from toplevel to include/lttng/

- Remove extra "lttng-" from filename (now implied by the path).
- Adapt includes accordingly.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move headers from probes/ to include/lttng/
Mathieu Desnoyers [Wed, 6 May 2020 13:45:16 +0000 (09:45 -0400)] 
Cleanup: Move headers from probes/ to include/lttng/

- Remove extra "lttng-" from filename (now implied by the path).
- Adapt includes accordingly.
- Adapt lttng-syscalls-generate-headers.sh header generation script
  accordingly.
- Remove probes/lttng.h, include its PARAMS() define in the two
  user headers.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move headers from lib/ to include/lttng/
Mathieu Desnoyers [Wed, 6 May 2020 13:36:45 +0000 (09:36 -0400)] 
Cleanup: Move headers from lib/ to include/lttng/

Adapt includes accordingly.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoCleanup: Move lib/ringbuffer/ headers to include/ringbuffer/
Mathieu Desnoyers [Wed, 6 May 2020 13:21:00 +0000 (09:21 -0400)] 
Cleanup: Move lib/ringbuffer/ headers to include/ringbuffer/

Remove the <wrapper/ringbuffer/...> proxy include files, and add the
include/ directory to the preprocessor include search patch.

Adapt all includes accordingly.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: wrapper random documentation
Mathieu Desnoyers [Mon, 13 Apr 2020 18:38:51 +0000 (14:38 -0400)] 
Fix: wrapper random documentation

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoUpdate for kernel 5.7: use vmalloc_sync_mappings on kernels >= 5.7
Mathieu Desnoyers [Tue, 5 May 2020 17:38:31 +0000 (13:38 -0400)] 
Update for kernel 5.7: use vmalloc_sync_mappings on kernels >= 5.7

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoUnbreak LTTng for kernel 5.7
Mathieu Desnoyers [Mon, 4 May 2020 19:00:53 +0000 (15:00 -0400)] 
Unbreak LTTng for kernel 5.7

Linux commit 0bd476e6c67190b5eb7b6e105c8db8ff61103281 ("kallsyms:
unexport kallsyms_lookup_name() and kallsyms_on_each_symbol()") breaks
LTTng-modules by removing symbols used by the LTTng-modules out-of-tree
tracer.

I pointed this out when the change was originally considered before the
5.7 merge window. This generated some discussion but it did not lead to
any concrete proposal to fix the issue. [1]

The commit has been merged in the 5.7 merge window. At that point, as
maintainer of LTTng, I immediately raised a flag about this issue,
proposing an alternative approach to solve this: expose the few symbols
needed by LTTng to GPL modules. This was NACKed on the ground that the
Linux kernel cannot export GPL symbols when there are no in-tree
users. [2]

Steven Rostedt has shown interest in merging LTTng-modules upstream.
LTTng-modules being LGPL, this is very much doable. I have prepared a
tree of LTTng-modules "for upstreaming" and sent it to him privately so
he can review it. Even if in an ideal scenario LTTng-modules is merged
for the following merge window, it leaves LTTng-modules broken on the
5.7 kernel.

In order to ensure that the LTTng-modules kernel tracer continues working
for my end users on kernels 5.7 onwards, as a very last resort, this is
with great reluctance that I created this fix for LTTng modules. It
basically uses kprobes to lookup the kallsyms_lookup_name symbol, and
continues using kallsyms_lookup_name as before.

Link: https://lore.kernel.org/r/20200302192811.n6o5645rsib44vco@localhost
Link: https://lore.kernel.org/r/20200409193543.18115-1-mathieu.desnoyers@efficios.com
Link: https://lwn.net/Articles/817988/
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Will Deacon <will@kernel.org>
CC: akpm@linux-foundation.org
CC: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
CC: Masami Hiramatsu <mhiramat@kernel.org>
CC: rostedt@goodmis.org
CC: Alexei Starovoitov <ast@kernel.org>
4 years agoMove lttng wrappers into own module
Mathieu Desnoyers [Mon, 4 May 2020 18:52:13 +0000 (14:52 -0400)] 
Move lttng wrappers into own module

Currently, we only pull the wrapper symbols into a single sub-module,
either:

lttng-tracer.o:
  - wrapper/random.o
  - wrapper/trace-clock.o
  - wrapper/page_alloc.o

or

lttng-statedump.o:
  - wrapper/irqdesc.o
  - wrapper/fdtable.o

Because lttng-tracer depends on lttng-statedump, we cannot just put all
wrappers into lttng-tracer.o, because it would create a circular
dependency. This will be an issue if we introduce common wrappers which
are used in both lttng-tracer.o and in lttng-statedump.o.

Introduce a new lttng-wrapper.o to contain all wrapper symbols for all
lttng modules.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoIntroduce lttng_guid_gen wrapper for kernels >= 5.7.0
Mathieu Desnoyers [Mon, 13 Apr 2020 16:16:43 +0000 (12:16 -0400)] 
Introduce lttng_guid_gen wrapper for kernels >= 5.7.0

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoinstrumentation: update x86 kvm instrumentation for kernel >= 5.7.0
Mathieu Desnoyers [Mon, 13 Apr 2020 15:44:23 +0000 (11:44 -0400)] 
instrumentation: update x86 kvm instrumentation for kernel >= 5.7.0

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoinstrumentation: update mm_vmscan for kernel >= 5.7.0
Mathieu Desnoyers [Mon, 13 Apr 2020 15:38:48 +0000 (11:38 -0400)] 
instrumentation: update mm_vmscan for kernel >= 5.7.0

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofilter: bytecode already in the list should go before
Francis Deslauriers [Fri, 17 Apr 2020 14:01:40 +0000 (10:01 -0400)] 
filter: bytecode already in the list should go before

Background
==========
This `seqnum` (sequence number) feature is currently unused. It was
designed so that the session daemon could tell the tracer the order in
which the bytecode should be run.

Issue
=====
The current implementation of the session daemon doesn't use this
feature so there is only ever a single bytecode to execute per callsite.

During work on an upcoming feature uses this `seqnum` became useful and
it was realized that the current bytecode linking code would reverse the
order in which the bytecode were executed when all bytecodes have the
same `seqnum` value.

This is due to the fact that the `cds_list_for_each_entry_reverse` loops
until it finds a `seqnum` smaller than the new one.

So if all bytecodes have the same `seqnum`, the new bytecode will be
added at the beginning of the list.

This is not technically a problem since it's the session daemon's job to
set the sequence number if it wants a particular ordering. Even
considering that, we found it counterintuitive that new bytecodes are
added at the beginning of the list in those cases.

Solution
========
This commit makes it so that on equality, the insertion is done after
the existing bytecodes.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I784887e3e6085f9344a2bb429d4f0d30586ebc57

4 years agotracepoint: Refactor representation of nested types before-upstreaming
Mathieu Desnoyers [Tue, 7 Apr 2020 17:07:54 +0000 (13:07 -0400)] 
tracepoint: Refactor representation of nested types

Refactor enumeration, sequence, array, structure, and variant types.

Implement internal data structures to support nested types.

All probe providers using ctf_enum(), ctf_array*() and ctf_sequence*()
are switched to this new internal type representation.

Each of sequence, array, struct and variant gain a "alignment" property,
which is a feature which was needed in lttng-modules to express
alignment for an array or sequence of bits.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agowrapper/compiler.h: Implement __LTTNG_COMPOUND_LITERAL
Mathieu Desnoyers [Mon, 6 Apr 2020 16:00:47 +0000 (12:00 -0400)] 
wrapper/compiler.h: Implement __LTTNG_COMPOUND_LITERAL

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoUpdate to SPDX v3.0 identifiers
Michael Jeanson [Tue, 31 Mar 2020 18:29:29 +0000 (14:29 -0400)] 
Update to SPDX v3.0 identifiers

The short form of GPL-2.0 and LGPL-2.1 were deprecated in favour of the
clearer GPL-2.0-only and GPL-2.0-or-later in the SPDX license list v3.0.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If2337f5c67a2548d7f25043e67006211213cbe3e

4 years agofix: uaccess wrapper for CentOS >= 4.18.0-147
Michael Jeanson [Thu, 2 Apr 2020 18:08:36 +0000 (14:08 -0400)] 
fix: uaccess wrapper for CentOS >= 4.18.0-147

Fixes: #1253
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2a79c1c0e897a6148e60e5599949cd2778d09d50

4 years agofix: ext4 instrumentation for CentOS >= 4.18.0-147
Michael Jeanson [Thu, 2 Apr 2020 18:08:09 +0000 (14:08 -0400)] 
fix: ext4 instrumentation for CentOS >= 4.18.0-147

Fixes: #1253
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1fd54af16fbb02cd4b3ab7fc7d9232708088f1fd

4 years agofix: signal instrumentation for CentOS >= 4.18.0-147
Michael Jeanson [Thu, 2 Apr 2020 18:07:47 +0000 (14:07 -0400)] 
fix: signal instrumentation for CentOS >= 4.18.0-147

Fixes: #1253
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I280013402df6f14222fbb912cdf64d80af3ab265

4 years agofix: kvm instrumentation for CentOS >= 4.18.0-147
Michael Jeanson [Thu, 2 Apr 2020 18:07:21 +0000 (14:07 -0400)] 
fix: kvm instrumentation for CentOS >= 4.18.0-147

Fixes: #1253
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ide20ebf51bec503866ffc96dda3e0b09ebeb14d6

4 years agofix: rcu instrumentation for CentOS >= 4.18.0-80
Michael Jeanson [Thu, 2 Apr 2020 18:06:17 +0000 (14:06 -0400)] 
fix: rcu instrumentation for CentOS >= 4.18.0-80

Fixes: #1253
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1735d2caa7215ce94272aaaa98cbbc8f3a10743d

4 years agoFix: update kvm instrumentation for Ubuntu 5.3.0-45
Michael Jeanson [Mon, 30 Mar 2020 21:43:16 +0000 (17:43 -0400)] 
Fix: update kvm instrumentation for Ubuntu 5.3.0-45

This commit introduced in 5.3.0-43 was dropped in 5.3.0-45 and reintroduced
in 5.3.0-46:

  commit 795f8a34f279e17c279bba46da10f15c5dd00264
  Author: Sean Christopherson <sean.j.christopherson@intel.com>
  Date:   Fri Dec 6 15:57:14 2019 -0800

    KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit KVM

BugLink: https://bugs.launchpad.net/bugs/1867051
    [ Upstream commit 736c291c9f36b07f8889c61764c28edce20e715d ]

Fun times!

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ia5f1a4ba355f592f09e964038b6334ddb3ad5153

4 years agoFix: update kvm instrumentation for Ubuntu 5.3.0-43
Michael Jeanson [Tue, 24 Mar 2020 18:20:48 +0000 (14:20 -0400)] 
Fix: update kvm instrumentation for Ubuntu 5.3.0-43

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1ce5f9ebba997fcc4cfbae6901eed479e2e1a79e

4 years agoFix: fcntl enum: only emit F_GETOWNER_UIDS for kernels >= 3.6
Mathieu Desnoyers [Tue, 24 Mar 2020 13:42:23 +0000 (09:42 -0400)] 
Fix: fcntl enum: only emit F_GETOWNER_UIDS for kernels >= 3.6

It appears in upstream Linux kernel
commit 1d151c337d79 "c/r: fcntl: add F_GETOWNER_UIDS option"
which was first released in v3.6.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agosyscalls: Make the cmd field of fcntl an enum
Geneviève Bastien [Wed, 11 Mar 2020 16:38:51 +0000 (12:38 -0400)] 
syscalls: Make the cmd field of fcntl an enum

The fcntl system call has a cmd parameter which is one of multiple
values defined in the linux/fcntl.h file. This field is now an
enumeration of the various possible values.

Here's an example babeltrace output of the fcntl system call:

[...] syscall_entry_fcntl: { cpu_id = 3 }, { fd = 35,
cmd = ( "F_GETFL" : container = 3 ), arg = 140388746332169 }

Change-Id: I06f511d5dba986aaf01b50057cbe6eac789bbea7
Signed-off-by: Geneviève Bastien <gbastien+lttng@versatic.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agosyscalls: Make the flags and mode fields of open[at] enumerations
Geneviève Bastien [Wed, 11 Mar 2020 16:33:04 +0000 (12:33 -0400)] 
syscalls: Make the flags and mode fields of open[at] enumerations

The open and openat system call have a flags and mode fields, whose values
are defined in the linux/fcntl.h file. These fields are now
enumerations that can be read as a bit field enum, to make the values more
readable / meaningful.

Here's an example babeltrace output of the open system call:

[...] syscall_entry_openat: { cpu_id = 0 }, { dfd = -100,
filename = "/tmp/edg0_383407",
flags = ( "O_RDWR" | "O_CREAT" | "O_TRUNC" : container = 578 ),
mode = ( "S_IWOTH" | "S_IROTH" | "S_IWGRP" | "S_IRGRP" |
   "S_IWUSR" | "S_IRUSR" : container = 438 ) }

Change-Id: Id7a516670b03e52fc75f9ff3c6ba8114c61a3865
Signed-off-by: Geneviève Bastien <gbastien@versatic.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: update kvm instrumentation for Ubuntu 4.15.0-92
Michael Jeanson [Mon, 23 Mar 2020 18:48:24 +0000 (14:48 -0400)] 
Fix: update kvm instrumentation for Ubuntu 4.15.0-92

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: Ib367b9a0ce3846f45313906e710a9a6d644e3955

4 years agoRecord event as soon as one filter evaluates to TRUE
Francis Deslauriers [Fri, 20 Mar 2020 20:23:17 +0000 (16:23 -0400)] 
Record event as soon as one filter evaluates to TRUE

This is a tiny optimization but it can reduce tracing cost in cases
where multiple filters are attached to a probe.

Since we want to trace an event if any of its filter expressions
evaluates to TRUE, there is no need to keep on evaluating expressions
once one is found to be TRUE.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6b5b7697869d3e25dc24d38dc39c34ae1d49dcc9

4 years agoAdd UDP and ICMP packet header information to the tracepoint
Florian Walbroel via lttng-dev [Tue, 17 Mar 2020 09:03:19 +0000 (10:03 +0100)] 
Add UDP and ICMP packet header information to the tracepoint

* UDP transport header
* ICMP transport header

Signed-off-by: Florian Walbroel <walbroel@silexica.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agostatedump: introduce file_table_address
Mathieu Desnoyers [Tue, 10 Mar 2020 19:00:29 +0000 (15:00 -0400)] 
statedump: introduce file_table_address

Currently the LTTng-modules statedump simply iterates over all processes
in the system and assumes all threads share the same file descriptor
table, which is only true if threads were created with clone
CLONE_FILES.

Directly invoking clone without the CLONE_FILES creates threads which
belong to the same process, but have their own file descriptor table.

Therefore, model-wise, we cannot assume that all threads in a process
have the same fd table content.

Add a new "file_table_address" field to the lttng_statedump_process_state
event, which dumps the address of the thread's struct files_struct
pointer. This pointer is guaranteed to never be re-used while we hold
the RCU read-side lock (so for the entire iteration over
processes/threads).

For the lttng_statedump_file_descriptor event, remove the "pid" field
(which is semantically inaccurate) and add a "file_table_address" field,
which contains the struct files_struct address of the file table
containing the file descriptor.

An optimization is performed to eliminate most duplcated file table
content by skipping file table dump if the same file table address is
encountered consecutively while iterating over a process' threads.

This introduces a semantic change to the statedump fields, and will
therefore be introduced in lttng-modules 2.12 onwards, not backported as
a fix.

Fixes: #1245
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoRemove kernel version from syscall headers name
Michael Jeanson [Wed, 4 Mar 2020 20:45:18 +0000 (15:45 -0500)] 
Remove kernel version from syscall headers name

This will reduce the delta when updating the syscall headers and make it
easier to track the changes.

Add a comment in the headers to track the version they were generated
from.

Change-Id: I939e1ebd8ab18d880ff1394e771982b5491d3b19
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoAdd script to automate the syscall extraction process
Michael Jeanson [Wed, 4 Mar 2020 20:37:11 +0000 (15:37 -0500)] 
Add script to automate the syscall extraction process

Change-Id: Ic8e1e62058616f221569e1f5ed2ffb9d54275af5
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoUpdate lttng-syscalls-extractor for v5.6.0
Michael Jeanson [Wed, 4 Mar 2020 20:21:03 +0000 (15:21 -0500)] 
Update lttng-syscalls-extractor for v5.6.0

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I48e4d82484d6edcc7601a40d2f94e5c46ad5184b

4 years agoAdd a Makefile for the lttng-syscalls-extractor module
Michael Jeanson [Wed, 4 Mar 2020 20:06:41 +0000 (15:06 -0500)] 
Add a Makefile for the lttng-syscalls-extractor module

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I0c1cc650d8594ef01b4039bb3d2af94bfddea54d

4 years agoCleanup: Syscall headers scripts shellcheck warnings
Michael Jeanson [Fri, 14 Feb 2020 21:38:27 +0000 (16:38 -0500)] 
Cleanup: Syscall headers scripts shellcheck warnings

Change-Id: Ia616724d942f43f5fde2186bff28446120b17c43
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoRemove lttng-ftrace integration
Mathieu Desnoyers [Mon, 2 Mar 2020 16:26:39 +0000 (11:26 -0500)] 
Remove lttng-ftrace integration

The lttng-ftrace integration (LTTNG_KERNEL_FUNCTION instrumentation
type) was unused for a while now. The "function" probing is actually
done with kprobes and kretprobes (LTTNG_KERNEL_KPROBE and
LTTNG_KERNEL_KRETPROBE).

Remove it so a use of kallsyms_lookup_name() can be removed as well.
Note that in the future we could add back this support by using
register_ftrace_function() which is exported to kernel modules, but
considering that we have not been using this code for a while,
just remove the implementation for now.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoRemove dependency on kallsyms for splice_to_pipe (kernel 4.2+)
Mathieu Desnoyers [Mon, 2 Mar 2020 16:03:19 +0000 (11:03 -0500)] 
Remove dependency on kallsyms for splice_to_pipe (kernel 4.2+)

Upstream commit 2b514574f7e88 "net: af_unix: implement splice for stream
af_unix sockets" exported the "splice_to_pipe" symbol, so use it to
remove a dependency on kallsyms_lookup_name().

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoRemove dependency on kallsyms for irq_to_desc (kernel 3.4+)
Mathieu Desnoyers [Mon, 2 Mar 2020 15:52:54 +0000 (10:52 -0500)] 
Remove dependency on kallsyms for irq_to_desc (kernel 3.4+)

Upstream commit 3911ff30f5d "genirq: export handle_edge_irq() and
irq_to_desc()" exported the irq_to_desc symbol, so use it to remove a
dependency on kallsyms_lookup_name().

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoRemove work-around for signed tracepoint module tainting (kernel 3.15+)
Mathieu Desnoyers [Mon, 2 Mar 2020 15:40:26 +0000 (10:40 -0500)] 
Remove work-around for signed tracepoint module tainting (kernel 3.15+)

Upstream commit 66cc69e34e86a "Fix: module signature vs tracepoints: add
new TAINT_UNSIGNED_MODULE" fixed an issue where the kernel was
considering unsigned modules as tainting the kernel in the same way as a
force-loaded modules, which was causing the tracepoints within those
modules to be hidden.

This fix was merged in kernel 3.15, so there is no use in applying this
work-around starting from that kernel.

This removes a dependency on kallsyms_lookup_name() for the symbol
"tracepoint_module_notify".

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoChange integer base to hex for fields representing addresses
Francis Deslauriers [Fri, 21 Feb 2020 21:59:33 +0000 (16:59 -0500)] 
Change integer base to hex for fields representing addresses

It seems more expected to show addresses in hexadecimal.

I applied this to all fields named "ip" or "addr".

I haven't change the `addr` fields in the i2c.h file as I am not aware
of what is the convention when displaying I2C addresses.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I6ed00b31e319e1d966ab138471248dcf3d2c7315

4 years agoFix: rcu: Fix data-race due to atomic_t copy-by-value (5.5.6, 5.4.22)
Mathieu Desnoyers [Tue, 25 Feb 2020 15:18:11 +0000 (10:18 -0500)] 
Fix: rcu: Fix data-race due to atomic_t copy-by-value (5.5.6, 5.4.22)

The following upstream commit has been backported to stable kernels
5.5.6 and 5.4.22:

  commit 6cf539a87a61a4fbc43f625267dbcbcf283872ed
  Author: Marco Elver <elver@google.com>
  Date:   Wed Oct 9 17:57:43 2019 +0200

    rcu: Fix data-race due to atomic_t copy-by-value

    This fixes a data-race where `atomic_t dynticks` is copied by value. The
    copy is performed non-atomically, resulting in a data-race if `dynticks`
    is updated concurrently.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: y2038: hide timeval/timespec/itimerval/itimerspec types (v5.6)
Michael Jeanson [Mon, 24 Feb 2020 20:11:03 +0000 (15:11 -0500)] 
fix: y2038: hide timeval/timespec/itimerval/itimerspec types (v5.6)

See upstream commit:

  commit c766d1472c70d25ad475cf56042af1652e792b23
  Author: Arnd Bergmann <arnd@arndb.de>
  Date:   Thu Feb 20 20:03:57 2020 -0800

    y2038: hide timeval/timespec/itimerval/itimerspec types

    There are no in-kernel users remaining, but there may still be users that
    include linux/time.h instead of sys/time.h from user space, so leave the
    types available to user space while hiding them from kernel space.

    Only the __kernel_old_* versions of these types remain now.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I986a813ad8b1c753ab1fa07f726b0cc481f049cb

4 years agofix: use timespec64 on kernels that have it
Michael Jeanson [Mon, 24 Feb 2020 19:50:20 +0000 (14:50 -0500)] 
fix: use timespec64 on kernels that have it

This fixes v5.6 which has hidden 'struct timespec' from kernel code and
makes 32bit archs y2038 compliant on v3.17 and newer.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I1daedc4a93cbbebb8f5a1d99c4619cb27f6a9e07

4 years agofix: move lttng_close_on_exec to proper wrapper
Michael Jeanson [Mon, 24 Feb 2020 20:15:36 +0000 (15:15 -0500)] 
fix: move lttng_close_on_exec to proper wrapper

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I330c6d63fc9c1af00c9a9471563d322d036b9a9c

4 years agofix: 'struct timex' removed upstream (v5.6)
Michael Jeanson [Mon, 24 Feb 2020 16:30:22 +0000 (11:30 -0500)] 
fix: 'struct timex' removed upstream (v5.6)

The 'timex' struct was remove in v5.6 and replaced by 2 variants, one
that is y2038 compliant and a compat version for 32bit archs.

Add this temporary fix while we update our syscalls tracepoint headers,
the type of this struct has limited importance since it's only used to
record the adress in the trace.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Change-Id: I085b22f282db57985f1c3d341e7c0866cb20e3c9

4 years agoFix: statedump: consistently check task_cred_xxx() return value for NULL
Mathieu Desnoyers [Thu, 20 Feb 2020 15:46:25 +0000 (10:46 -0500)] 
Fix: statedump: consistently check task_cred_xxx() return value for NULL

trace_lttng_statedump_process_user_ns() internally checks whether
user_ns is NULL. While this does not appear to be a possible return
value for task_cred_xxx(), err on the safe side and check for NULL here
as well to be consistent with the paranoid behavior of
trace_lttng_statedump_process_user_ns().

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: statedump: check task_active_pid_ns return value for NULL
Mathieu Desnoyers [Thu, 20 Feb 2020 14:58:42 +0000 (09:58 -0500)] 
Fix: statedump: check task_active_pid_ns return value for NULL

The lttng-statedump checks the return value of task_active_pid_ns()
before each use within lttng_statedump_process_pid_ns(), but misses
the NULL check before dereferencing pid_ns->parent.

This race happens if a task exists in "dead" state while the statedump
iterates on that task.

Reported-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: lttng-events.c: variable may be used uninitialized
Francis Deslauriers [Tue, 4 Feb 2020 21:05:30 +0000 (16:05 -0500)] 
Fix: lttng-events.c: variable may be used uninitialized

Fixes the following warning:
/home/frdeso/projets/lttng/modules/lttng-events.c: In function ‘print_metadata_escaped_field’:
/home/frdeso/projets/lttng/modules/lttng-events.c:2563:5: error: ‘ret’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
     if (ret)
           ^

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: If2db5e1ad9782fb49d6d07026976d3d22f89f2ab

4 years agoCleanup: remove trailing white spaces across project
Francis Deslauriers [Mon, 17 Feb 2020 23:13:56 +0000 (18:13 -0500)] 
Cleanup: remove trailing white spaces across project

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I2de34e4913f9cefbd72a138f2d3dfb9d67ece311

4 years agosched: Make the sched_switch task state an enum
Geneviève Bastien [Wed, 12 Feb 2020 21:58:25 +0000 (16:58 -0500)] 
sched: Make the sched_switch task state an enum

This gives meaning to the task state value. Only the bit masks are
enumerated, as defined compositions are not exhaustive listing of all
possible values and there would be a lot of unknown. Interpretation of
combination of bit flags is left to the consumer of the event.

Change-Id: I83c5fbee9cba2701c7238c0ac6abd4c8a351b193
Signed-off-by: Geneviève Bastien <gbastien@versatic.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoblock: Make the rwbs field as a bit field enum
Geneviève Bastien [Tue, 11 Feb 2020 16:20:27 +0000 (11:20 -0500)] 
block: Make the rwbs field as a bit field enum

The rwbs value of block request events is in fact a series of bit fields
set to 0 or 1. The enumeration values are all powers of 2, trace readers
could interpret this as a bit field enum and show the result as a
concatenation of the corresponding flags. For example, with a matching
patch for babeltrace2's pretty sink, the output for a rwbs value of 12
would be:

[13:15:49.024354958] (+0.000003868) wilbrod block_rq_complete: { cpu_id = 4 },
    { dev = 8388624, sector = 375490176, nr_sector = 360, error = 0,
      rwbs = ( "RWBS_FLAG_READ" | "RWBS_FLAG_RAHEAD" : container = 12 ) }

Consumers who have no notion of the bit field enum can still use the
integer value of the field as before.

Change-Id: I2e873f16e5a0507e52cfc85c70b471a9f4d5d5d1
Signed-off-by: Geneviève Bastien <gbastien@versatic.net>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: workqueue: add worker function to workqueue_execute_end tracepoint (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 20:35:11 +0000 (15:35 -0500)] 
fix: workqueue: add worker function to workqueue_execute_end tracepoint (v5.6)

See upstream commit :

  commit 1c5da0ec7f20dfb56030fb93f7f52f48e12deb52
  Author: Daniel Jordan <daniel.m.jordan@oracle.com>
  Date:   Mon Jan 13 17:52:39 2020 -0500

    workqueue: add worker function to workqueue_execute_end tracepoint

    It's surprising that workqueue_execute_end includes only the work when
    its counterpart workqueue_execute_start has both the work and the worker
    function.

    You can't set a tracing filter or trigger based on the function, and
    postprocessing scripts interested in specific functions are harder to
    write since they have to remember the work from _start and match it up
    with the same field in _end.

    Add the function name, taking care to use the copy stashed in the
    worker since the work is no longer safe to touch.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: media: v4l2: abstract timeval handling in v4l2_buffer (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 20:13:53 +0000 (15:13 -0500)] 
fix: media: v4l2: abstract timeval handling in v4l2_buffer (v5.6)

See upstream commit :

  commit 77cdffcb0bfb87fe3645894335cb8cb94917e6ac
  Author: Arnd Bergmann <arnd@arndb.de>
  Date:   Mon Dec 16 15:15:00 2019 +0100

    media: v4l2: abstract timeval handling in v4l2_buffer

    As a preparation for adding 64-bit time_t support in the uapi,
    change the drivers to no longer care about the format of the
    timestamp field in struct v4l2_buffer.

    The v4l2_timeval_to_ns() function is no longer needed in the
    kernel after this, but there is userspace code relying on
    it to be part of the uapi header.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: rcu: Remove kfree_rcu() special casing and lazy-callback (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 20:08:04 +0000 (15:08 -0500)] 
fix: rcu: Remove kfree_rcu() special casing and lazy-callback (v5.6)

See upstream commit :

  commit 77a40f97030b27b3fc1640a3ed203870f0817f57
  Author: Joel Fernandes (Google) <joel@joelfernandes.org>
  Date:   Fri Aug 30 12:36:32 2019 -0400

    rcu: Remove kfree_rcu() special casing and lazy-callback handling

    This commit removes kfree_rcu() special-casing and the lazy-callback
    handling from Tree RCU.  It moves some of this special casing to Tiny RCU,
    the removal of which will be the subject of later commits.

    This results in a nice negative delta.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: rcu: Fix data-race due to atomic_t copy-by-value (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 19:51:01 +0000 (14:51 -0500)] 
fix: rcu: Fix data-race due to atomic_t copy-by-value (v5.6)

See upstream commit :

  commit 6cf539a87a61a4fbc43f625267dbcbcf283872ed
  Author: Marco Elver <elver@google.com>
  Date:   Wed Oct 9 17:57:43 2019 +0200

    rcu: Fix data-race due to atomic_t copy-by-value

    This fixes a data-race where `atomic_t dynticks` is copied by value. The
    copy is performed non-atomically, resulting in a data-race if `dynticks`
    is updated concurrently.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: btrfs: make btrfs_ordered_extent naming consistent (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 19:47:23 +0000 (14:47 -0500)] 
fix: btrfs: make btrfs_ordered_extent naming consistent (v5.6)

See upstream commit :

  commit bffe633e00fb6b904817137fc17a44b42efcd985
  Author: Omar Sandoval <osandov@fb.com>
  Date:   Mon Dec 2 17:34:19 2019 -0800

    btrfs: make btrfs_ordered_extent naming consistent with btrfs_file_extent_item

    ordered->start, ordered->len, and ordered->disk_len correspond to
    fi->disk_bytenr, fi->num_bytes, and fi->disk_num_bytes, respectively.
    It's confusing to translate between the two naming schemes. Since a
    btrfs_ordered_extent is basically a pending btrfs_file_extent_item,
    let's make the former use the naming from the latter.

    Note that I didn't touch the names in tracepoints just in case there are
    scripts depending on the current naming.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 19:41:29 +0000 (14:41 -0500)] 
fix: KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit (v5.6)

See upstream commit :

  commit 736c291c9f36b07f8889c61764c28edce20e715d
  Author: Sean Christopherson <sean.j.christopherson@intel.com>
  Date:   Fri Dec 6 15:57:14 2019 -0800

    KVM: x86: Use gpa_t for cr2/gpa to fix TDP support on 32-bit KVM

    Convert a plethora of parameters and variables in the MMU and page fault
    flows from type gva_t to gpa_t to properly handle TDP on 32-bit KVM.

    Thanks to PSE and PAE paging, 32-bit kernels can access 64-bit physical
    addresses.  When TDP is enabled, the fault address is a guest physical
    address and thus can be a 64-bit value, even when both KVM and its guest
    are using 32-bit virtual addressing, e.g. VMX's VMCS.GUEST_PHYSICAL is a
    64-bit field, not a natural width field.

    Using a gva_t for the fault address means KVM will incorrectly drop the
    upper 32-bits of the GPA.  Ditto for gva_to_gpa() when it is used to
    translate L2 GPAs to L1 GPAs.

    Opportunistically rename variables and parameters to better reflect the
    dual address modes, e.g. use "cr2_or_gpa" for fault addresses and plain
    "addr" instead of "vaddr" when the address may be either a GVA or an L2
    GPA.  Similarly, use "gpa" in the nonpaging_page_fault() flows to avoid
    a confusing "gpa_t gva" declaration; this also sets the stage for a
    future patch to combing nonpaging_page_fault() and tdp_page_fault() with
    minimal churn.

    Sprinkle in a few comments to document flows where an address is known
    to be a GVA and thus can be safely truncated to a 32-bit value.  Add
    WARNs in kvm_handle_page_fault() and FNAME(gva_to_gpa_nested)() to help
    document such cases and detect bugs.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: proc: decouple proc from VFS with "struct proc_ops" (v5.6)
Michael Jeanson [Tue, 11 Feb 2020 16:20:41 +0000 (11:20 -0500)] 
fix: proc: decouple proc from VFS with "struct proc_ops" (v5.6)

See upstream commit :

  commit d56c0d45f0e27f814e87a1676b6bdccccbc252e9
  Author: Alexey Dobriyan <adobriyan@gmail.com>
  Date:   Mon Feb 3 17:37:14 2020 -0800

    proc: decouple proc from VFS with "struct proc_ops"

    Currently core /proc code uses "struct file_operations" for custom hooks,
    however, VFS doesn't directly call them.  Every time VFS expands
    file_operations hook set, /proc code bloats for no reason.

    Introduce "struct proc_ops" which contains only those hooks which /proc
    allows to call into (open, release, read, write, ioctl, mmap, poll).  It
    doesn't contain module pointer as well.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoVersion 2.12.0-rc1 v2.12.0-rc1
Mathieu Desnoyers [Wed, 5 Feb 2020 15:36:41 +0000 (10:36 -0500)] 
Version 2.12.0-rc1

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: lttng-syscalls.c: marking wrong syscall probe as unregistered
Francis Deslauriers [Tue, 28 Jan 2020 22:19:20 +0000 (17:19 -0500)] 
Fix: lttng-syscalls.c: marking wrong syscall probe as unregistered

When calling `lttng_syscalls_unregister()` we currently mark as
unregistered the wrong syscall probe type.

Concretely, when we unregister "sys_exit" we wrongfully mark that
sys_enter is unregistered and vice versa.

Given than currently entry and exit probes are always enabled together
(except on internal errors), the effect of this bug is not user-visible.

Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Change-Id: I64acf6a941a3d1fa1bf8be424f834ddb7fb92ace

4 years agoVersion 2.12.0-pre v2.12.0-pre
Mathieu Desnoyers [Tue, 7 Jan 2020 18:09:38 +0000 (13:09 -0500)] 
Version 2.12.0-pre

Integrate the missing 2.11.0-rc1 entries to the changelog. This
was caused by forking to stable-2.11 the commit before the v2.11.0-rc1
tag. In the future, we will try to keep the rc1 tag as the common
ancestor between master and stable branches.

This also updates the lttng-modules version to 2.12.0-pre, which is
a state that will stay until the v2.12.0-rc1 tag.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoBump LTTNG_MODULES_ABI_MINOR_VERSION to 5
Jérémie Galarneau [Fri, 20 Dec 2019 03:11:51 +0000 (22:11 -0500)] 
Bump LTTNG_MODULES_ABI_MINOR_VERSION to 5

New operations were added to the lttng-modules ABI as part of the 2.12
release cycle to support UID tracking and the "session clear"
functionality.

This will allow future LTTng-tools versions to check for those
capabilities.

Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: use user ns wrapper code in new id trackers
Michael Jeanson [Tue, 17 Dec 2019 17:11:11 +0000 (12:11 -0500)] 
fix: use user ns wrapper code in new id trackers

These wrappers are required to translate kuid on kernels prior to v3.5.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: function prototype in wrapper/mm.h
Michael Jeanson [Tue, 17 Dec 2019 17:11:10 +0000 (12:11 -0500)] 
fix: function prototype in wrapper/mm.h

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoID tracker: implement vpid/uid/vuid/gid/vgid trackers
Mathieu Desnoyers [Tue, 17 Dec 2019 15:17:01 +0000 (10:17 -0500)] 
ID tracker: implement vpid/uid/vuid/gid/vgid trackers

Co-developed-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agolttng-abi: Document ioctl numbers reserved by lttng-abi-old.h
Mathieu Desnoyers [Tue, 17 Dec 2019 14:27:54 +0000 (09:27 -0500)] 
lttng-abi: Document ioctl numbers reserved by lttng-abi-old.h

Document the ioctl numbers reserved by lttng-abi-old.h in the
(relatively) new lttng-abi.h, so they are taken into account when
assining numbers to new lttng ioctl commands.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agolttng-clear: stop tracing required
Mathieu Desnoyers [Wed, 23 Oct 2019 16:56:59 +0000 (12:56 -0400)] 
lttng-clear: stop tracing required

Require that tracing is stopped when buffers are cleared. Update
comments and warning checks to that effect.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agosunrpc: introduce lttng_get_clid helper
Mathieu Desnoyers [Thu, 12 Dec 2019 15:39:38 +0000 (10:39 -0500)] 
sunrpc: introduce lttng_get_clid helper

Introduce the lttng_get_clid helper to always check for NULL pointer
when getting the client id. While not always strictly needed depending
on the tracepoint callsite, prefer robustness of instrumentation and
always check for NULL rather than play whack-a-mole.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: sunrpc: use signed integer for client id
Mathieu Desnoyers [Thu, 12 Dec 2019 15:29:37 +0000 (10:29 -0500)] 
Fix: sunrpc: use signed integer for client id

Within include/linux/sunrpc/clnt.h:struct rpc_cltn, the cl_clid field
is an unsigned integer, which is the type expected by the tracepoint
signature.

However, looking into net/sunrpc/clnt.c:rpc_alloc_clid(), its allocation
considers negative signed integer as errors.

Therefore, in order to properly show "-1" in the trace output (rather
than MAX_INT) when called with a NULL task->tk_client, move to a
signed integer as backing type for the client_id field.

Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: sunrpc: null rpc_clnt dereference in rpc_task_queued tracepoint
Mathieu Desnoyers [Thu, 12 Dec 2019 15:29:02 +0000 (10:29 -0500)] 
Fix: sunrpc: null rpc_clnt dereference in rpc_task_queued tracepoint

Based on upstream Linux commit:

commit 0be283f676a1e7b208db0c992283197ef8b52158
Author: Benjamin Coddington <bcodding@redhat.com>
Date:   Tue Jan 23 09:32:35 2018 -0500

    SUNRPC: Fix null rpc_clnt dereference in rpc_task_queued tracepoint

    Backchannel tasks will not have a reference to the rpc_clnt.  Return -1 for
    cl_clid in that case.

Signed-off-by: Benjamin Coddington <bcodding@redhat.com>
Signed-off-by: Trond Myklebust <trondmy@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agoFix: SUNRPC: Fix oops when trace sunrpc_task events in nfs client
Quanyang Wang [Thu, 5 Dec 2019 07:35:32 +0000 (15:35 +0800)] 
Fix: SUNRPC: Fix oops when trace sunrpc_task events in nfs client

See upstream commit :

commit 2ca310fc4160ed0420da65534a21ae77b24326a8
Author: Ditang Chen <chendt.fnst@cn.fujitsu.com>
Date: Fri, 7 Mar 2014 13:27:57 +0800
Subject: SUNRPC: Fix oops when trace sunrpc_task events in nfs client

When tracking sunrpc_task events in nfs client, the clnt pointer may be NULL.

[  139.269266] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004
[  139.269915] IP: [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
[  139.269915] PGD 1d293067 PUD 1d294067 PMD 0
[  139.269915] Oops: 0000 [#1] SMP
[  139.269915] Modules linked in: nfsv4 dns_resolver nfs lockd sunrpc fscache sg ppdev e1000
serio_raw pcspkr parport_pc parport i2c_piix4 i2c_core microcode xfs libcrc32c sd_mod sr_mod
cdrom ata_generic crc_t10dif crct10dif_common pata_acpi ahci libahci ata_piix libata dm_mirror
dm_region_hash dm_log dm_mod
[  139.269915] CPU: 0 PID: 59 Comm: kworker/0:2 Not tainted 3.10.0-84.el7.x86_64 #1
[  139.269915] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[  139.269915] Workqueue: rpciod rpc_async_schedule [sunrpc]
[  139.269915] task: ffff88001b598000 ti: ffff88001b632000 task.ti: ffff88001b632000
[  139.269915] RIP: 0010:[<ffffffffa026f216>]  [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
[  139.269915] RSP: 0018:ffff88001b633d70  EFLAGS: 00010206
[  139.269915] RAX: ffff88001dfc5338 RBX: ffff88001cc37a00 RCX: ffff88001dfc5334
[  139.269915] RDX: ffff88001dfc5338 RSI: 0000000000000000 RDI: ffff88001dfc533c
[  139.269915] RBP: ffff88001b633db0 R08: 000000000000002c R09: 000000000000000a
[  139.269915] R10: 0000000000062180 R11: 00000020759fb9dc R12: ffffffffa0292c20
[  139.269915] R13: ffff88001dfc5334 R14: 0000000000000000 R15: 0000000000000000
[  139.269915] FS:  0000000000000000(0000) GS:ffff88001fc00000(0000) knlGS:0000000000000000
[  139.269915] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  139.269915] CR2: 0000000000000004 CR3: 000000001d290000 CR4: 00000000000006f0
[  139.269915] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  139.269915] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  139.269915] Stack:
[  139.269915]  000000001b633d98 0000000000000246 ffff88001df1dc00 ffff88001cc37a00
[  139.269915]  ffff88001bc35e60 0000000000000000 ffff88001ffa0a48 ffff88001bc35ee0
[  139.269915]  ffff88001b633e08 ffffffffa02704b5 0000000000010000 ffff88001cc37a70
[  139.269915] Call Trace:
[  139.269915]  [<ffffffffa02704b5>] __rpc_execute+0x1d5/0x400 [sunrpc]
[  139.269915]  [<ffffffffa0270706>] rpc_async_schedule+0x26/0x30 [sunrpc]
[  139.269915]  [<ffffffff8107867b>] process_one_work+0x17b/0x460
[  139.269915]  [<ffffffff8107942b>] worker_thread+0x11b/0x400
[  139.269915]  [<ffffffff81079310>] ? rescuer_thread+0x3e0/0x3e0
[  139.269915]  [<ffffffff8107fc80>] kthread+0xc0/0xd0
[  139.269915]  [<ffffffff8107fbc0>] ? kthread_create_on_node+0x110/0x110
[  139.269915]  [<ffffffff815d122c>] ret_from_fork+0x7c/0xb0
[  139.269915]  [<ffffffff8107fbc0>] ? kthread_create_on_node+0x110/0x110
[  139.269915] Code: 4c 8b 45 c8 48 8d 7d d0 89 4d c4 41 89 c9 b9 28 00 00 00 e8 9d b4 e9
e0 48 85 c0 49 89 c5 74 a2 48 89 c7 e8 9d 3f e9 e0 48 89 c2 <41> 8b 46 04 48 8b 7d d0 4c
89 e9 4c 89 e6 89 42 0c 0f b7 83 d4
[  139.269915] RIP  [<ffffffffa026f216>] ftrace_raw_event_rpc_task_running+0x86/0xf0 [sunrpc]
[  139.269915]  RSP <ffff88001b633d70>
[  139.269915] CR2: 0000000000000004
[  140.946406] ---[ end trace ba486328b98d7622 ]---

Signed-off-by: Quanyang Wang <quanyang.wang@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
4 years agofix: ext4: Reserve revoke credits for freed blocks (v5.5)
Michael Jeanson [Tue, 10 Dec 2019 16:41:14 +0000 (11:41 -0500)] 
fix: ext4: Reserve revoke credits for freed blocks (v5.5)

See upstream commit:

  commit 83448bdfb59731c2f54784ed3f4a93ff95be6e7e
  Author: Jan Kara <jack@suse.cz>
  Date:   Tue Nov 5 17:44:29 2019 +0100

    ext4: Reserve revoke credits for freed blocks

    So far we have reserved only relatively high fixed amount of revoke
    credits for each transaction. We over-reserved by large amount for most
    cases but when freeing large directories or files with data journalling,
    the fixed amount is not enough. In fact the worst case estimate is
    inconveniently large (maximum extent size) for freeing of one extent.

    We fix this by doing proper estimate of the amount of blocks that need
    to be revoked when removing blocks from the inode due to truncate or
    hole punching and otherwise reserve just a small amount of revoke
    credits for each transaction to accommodate freeing of xattrs block or
    so.

Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
This page took 0.049173 seconds and 4 git commands to generate.