jjb: Restrict imagebuilder arches to those available in CI cluster

Change-Id: Ie010325d030708b32408904ed44c38ac5be7c0ec
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Increase details output during image building

It can be unclear what is going wrong without the raw output.

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: Ia55bfb46ac7f69fa50ecf0b38fdec75f6a3fd6c2

ansible: cross-compilers find source gcc version via dpkg

Change-Id: I828a012e7fef1fdaf0bf4e7a0afe6a3eacb53787
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: imagebuilder: run apt-get update before dependency installation

It can happen that the snapshotted root node could have old IPs that
are no longer valid cached for the apt mirrors. Running `apt-get
update` will ensure those are cleared.

Change-Id: Id0aa9ff0bfd40c5140d1397d4e6173a97225c26f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: images: do not force compilers_legacy_install to false

Change-Id: I5caaf0dae89576e359584ddd3f668799d9f1265e

ansible: Do not install legacy compilers for development nodes

Change-Id: I967b206d541101194036b16aa456f8775902afcc

jjb: lttng-tools: Only run clang root builds with clang-16

The root builds & tests take a long time, and there are fewer nodes
available. Running with all clang versions tends to bottle neck the
throughput.

Change-Id: I4a4df392c48c888e32fd336c3825ba6dcaafd932
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Reduce repetition in lttng-tools job definitions

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: If6465a947162e75274dcefe65bb4825d6697dc33

jjb: Run lttng-tools root builds when reviewing lttng-modules changes

Change-Id: I9946576f10b5d4017cdb69713c2535fea55f1e58
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: yocto23: replaced powerpcspe with powerpc

The Yocto 2.3 builder was moved from a p2020rdb to a p3041ds, this means
the CPU cores changed from e500v2 to e500mc. Both are 32bit big-endian
PowerPC but the e500mc is a more classic powerpc architecture while the
e500v2 is a rarer powerpcspe variant which has been deprecated.

Change-Id: If420a16cf97823c95908a6f0ebe9b34dc154d833
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: add jdk-17 for jenkins 2.479

The new Jenkins LTS agent.jar is compiled for JAVA >= 17, add it to the
SLES and RHEL nodes.

Change-Id: I3fbd76d09f0c13a6dcd2b3e4368b4f17c4a91e5f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

jjb: Use `**/*.log` to glob TAP log files

`**/*.*` catches other files left when running with more verbose test
configurations. `*.log` is the suffix for automake test log files[1].

See this proposed patch to lttng-tools master[2] which updates the log
suffix for the additional verbose log files.

[1]: https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html
[2]: https://review.lttng.org/c/lttng-tools/+/13430

Change-Id: If50a775af193a2973fd35b2b56337be21791fcc0
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Correct default kernel.core_pattern sysctl configuration

Change-Id: Iec26481bf65b6d66e810fec54360a226f133d160
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Reduce verbosity of rasdaemon-exporter

Change-Id: I883648f846952f37205d19c73c19c648d4bf5bcb
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Reduce verbosity of prometheus-node-exporter logs

Change-Id: Ife6e81686e0235ac7fa291f4a6d9c843d0d13823
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update user tasks to use fully qualified task names

Change-Id: I2bd4369642bcbbd844dec0ca041261745763d85f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add access tag for managing users

Change-Id: I6668b982a1bf470b68f868ed680d9216fc2b2741
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update odion's public keys

Change-Id: Id049955cbacc4670b6673548009f1b7e8c68383c
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Correct condition for `ci_role` promtail tag

Change-Id: I1e4805cd11528246ea14f510637c26e1b539d053
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install and configure promtail

Change-Id: Ia0625da7547412fdb41abdfd31036d0a378c939e
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Remove unused amd64 nodes

These nodes were added when on-ramping the new CI cluster, but have
since been superceded by `ci-node-deb12-amd64-[1-2][a-d]-\d+`

Change-Id: Ia86f8368a0311b65c835b5bdfb04067788d1712f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Remove el7 node

Change-Id: Id3b56ca1e8ba4d26c844d707e3cb33744e1614a6
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Add linux-kernel ci jobs

Change-Id: I56d2814d3ced0be7716d25c194ab52d5dffd2177
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Add `to_groovy` filter to convert variables to groovy syntax

To update jobs that use this filter, use recent jjb versions and
execute it as follows:

```
PYTHONPATH=. jenkins-job update ./jobs/linux.yml
```

Change-Id: I09b4b5e3e57f4bfeb76eac00399f3bb885bb1b2b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install device-tree-compiler for lttng-modules builds

When building a native build of the Linux kernel on non-x86 arches,
device-tree-compiler is required.

Change-Id: Id5627d3c164677e49310ccdec95a7e2aa08dc3ca
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install lttng-modules build dependencies on other node archs

In preparation for doing Linux kernel builds on those nodes.

Change-Id: I66f66748d9099a53a98027c6824b22dd7748b76b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Clone lttng-modules Linux mirror only for `node_amd64` nodes

Change-Id: Ia39a49582f574ff73452fd9a4ea642362371da58
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add info command to `manage_jenkins_node.py`

Change-Id: I0ce0177ae637d56e865594ce253b9ab52619f546
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add flag to snapshot-rootnode to run package upgrades

This defaults to false because the snapshot-rootnode needs to be run
anytime any configuration changes are made, and doing package upgrades
is not always desired.

Change-Id: Id1ab8dcd44c5e7172949bf251252cd6348e1f1c3
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add flag to disable upgrades when updating CI nodes

This play defaults to "true", since the normal ansible
playbook (`site.yml`) can be used to install new packages and
configuration without running the package upgrades.

Change-Id: Ifcdce209f7cac1cc72ca27e5ad3d3f150793fdfc
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Skip legacy compiler sources when not installing them

Change-Id: I289c56f7c6ce7ff5fdfb23c0514d049d7645a1b0
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Do not install legacy compilers on standalone nodes

These compilers are used only by the lttng-modules jobs to build old
kernels, which aren't run on standalone nodes.

Change-Id: I984d2b00048eaf9b5d599c3e59bfe48d713327f9
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Include compiler role on standalone nodes

Change-Id: Ib2d976ebd2efa0ec08df6a69856416fa27e3420e
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

WIP: jjb: Kernel dev jobs

Change-Id: I8889b7ccab5d1b4ec2f26153ff9d73a9ec175f72

ansible: Correct logging statement in manage_jenkins script

Change-Id: I0fc69fca0f6004738d4044b84215fa456916aef2
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add play to update non-standalone CI nodes

Change-Id: I7ca97377786a9f348c5273590d857c81b51dd635
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install gawk for building lttng-modules

While gawk isn't a dependency of lttng-modules, it is now required to
build the Linux kernel when `CONFIG_BUILTIN_MODULE_RANGES` is enabled.

See upstream commit:

    commit 5f5e7344322f0b0676579af054c787ed57d1c1df
    Author: Kris Van Hees <kris.van.hees@oracle.com>
    Date:   Fri Sep 6 10:45:03 2024 -0400

        kbuild: generate offset range data for builtin modules

        Create file module.builtin.ranges that can be used to find where
        built-in modules are located by their addresses. This will be useful for
        tracing tools to find what functions are for various built-in modules.

Change-Id: I6d9b1e93857131de30dc1c2527be93cd3327a296
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

Fix indentation in shell scripts

There is currently a mix of tabs and spaces in shell scripts.  The most
common style appears to be 4 spaces, so convert everything to that.
Modify `.editorconfig` to try to enforce that style.

Change-Id: Ic2c9a051a2713cc97d071bf3581952ee16985aa3
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>

jjb: babeltrace: lint: print shellcheck version

 - Make `print_tooling()` print the shellcheck version, if shellcheck is
   present
 - Source `scripts/common/print.sh` in the babeltrace lint jobs
 - Call `print_tooling` in the babeltrace lint jobs

Change-Id: Ie40aaca4a060a077d1be49b18f6d9f5edcf80749
Signed-off-by: Simon Marchi <simon.marchi@efficios.com>

Merge "jjb: babeltrace: check for untracked or modified files in the git repo"

jjb: Print shellcheck version if the tool is available

Change-Id: I356fbc23c3ad0023987c2b45f6c969624929e957
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: babeltrace: check for untracked or modified files in the git repo

To help spot missing entries in .gitignore, or build steps that somehow
modify source files, do a `git status` in the git repo after builds.
Fail the build if there are untracked or modified files.

Change-Id: Id93da324d234e30979418f8f77d29e98f24677c8
Signed-off-by: Simon Marchi <simon.marchi@polymtl.ca>

jjb: Add armhf and arm64 as build options for Debian images

Change-Id: I2f2bcae64d5003495fcceaececaaa000621923a6
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Allow unattended upgrades from Debian backports

Change-Id: Ia201cb63c660ab2b11757647af363de83d66043c
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add apt update handler

Change-Id: I832079b5d13b92f46316322f9e3879043b67486e
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Manage multiple netplan configurations

Change-Id: I8e3b59f1fafcaf1c94440b1e334026baa4fa4d00
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install prometheus metrics plugin on Jenkins servers

Change-Id: Ic79f3e341408a6ede06ac08e5194a222023295fe
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install web tools on developer nodes

Change-Id: I0707e8db6abdd29f0f9a5558f156799ac1544f85
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Add clang builds for lttng-tools

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: I852ab648b0f5a405367b631e4a9a83fa9e0f452b

jjb: Add clang builds for lttng-ust

Change-Id: I9d85c23775ffaee8c52aa0c62e7126c8d77a4bcf
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Add clang builds for liburcu

Change-Id: I79927757b51080790fd46d6bdf27ead2deab63d4
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Remove EL7 hosts

Change-Id: I45f1b717e051736e9f0eeb0edba87565f7941d3f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Remove special cases for EL7 from build scripts

Change-Id: Ifaba321a0a52beacbc3b560245d7fc0c47542c02
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Remove el7 platform from EL linux builds

Change-Id: I4dd069efded84ee075604ffca138806eaad99924
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Remove babeltrace CTF2 review jobs

The ctf2 feature branch no longer exists.

Change-Id: I77cbe254ca82f18ee7dff074fae04e33011d6369
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Run log4j2 lttng-ust-java-tests with LTTng-tools 2.14+

Change-Id: I96f540afea43ccfaa2df97a7edfa86ae816ca1f2
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Use jgit for MacOSX Babeltrace jobs

Following a number of checkout errors where git fetch hung and timed
out from the OSX builder to review.lttng.org, `jgit` was tested as an
alternative. The same hangs don't appear to happen when using `jgit`.

Change-Id: I20231150e066b71acdec3f9b309d7f93ad76ac2f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: add riscv64 to portbuild jobs

Add deb13-riscv64 to the main portbuild jobs, the boards are quite slow
so don't add them to the dev_review jobs yet.

Change-Id: I5c063cae03789ac1986641b8f517e81ef9e91a0b
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: babeltrace: add python3-setuptools on Debian

Python 3.12 removed distutils, so we added support for setuptools in
Babeltrace when building with Python >= 3.12.

Install python3-setuptools as Python 3.12 as landed in Debian sid.

Change-Id: Ia9db6dbd855f254d0ee1858c6e4de37585fd4da4
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: common-node: explicitly install openjdk-17

Explicitly install version 17 of openjdk as this is currently required
by our Jenkins setup and the default version varies between Debian
releases.

Change-Id: I28b1823b763fca989b4b7469582fed0166ad4e07
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: remove binutils-gdb role from riscv64 nodes

Not all dependencies are currently installable on riscv64 and we don't
plan to run the gdb jobs on riscv64 at the moment.

Change-Id: I64a25dbc6639ca122baec540ba70786d8965123f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: Add riscv64 nodes

Change-Id: If900b4309ef54ae396559058fafdef96bfb3166c
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

jjb: lttng-modules: update ubuntu supported kernel versions

Change-Id: Ia2a3b361397e1690ad12272391730f8d06a053c8
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: add kernel headers for arm64 and armhf

Change-Id: If27a1528514d7c7493c48c5e49ab4710560bdcad
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

Remove obsolete AUTHORS

Change-Id: Idef55ee5c8579f4ba92564748f39efc820f8af5a
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: Add python3-jenkins dep to README

Change-Id: I5e750b58532f55a0ffd6b936868e9050aaa714c6
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: Remove ci-node-deb12-armhf-0[5-8]

Change-Id: If97b7b8efab5de2708a0221cb995a93ecbb4e489
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jbb: Run librseq clang builds with clang-16

Change-Id: I22006cd42ca2406d5652088032e5fa7de17fb199
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Run Babeltrace jobs with clang-16

Change-Id: Id04e980bf2d8c532050383eb155fb3ccde4e7eeb
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install gcc 14 cross compilers on Debian trixie

Change-Id: I4aa810ccdf36b4371f21c7c490fb4c66d7983d1d
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Use sandbox for image builder pipeline

This avoids having to approve the script after each change.

Change-Id: I37488fe1b7c79b128875b3e56f709fd8d8b9519b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

jjb: Update image building jobs to use incus-client

Change-Id: I2e701af95b16227459bbcec403996275d893325b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update lz4 tool package name for Debian

liblz4-tool has been a transitional package since at least Debian
buster. In Debian trixie, this transitional package has been
removed. The new package name `lz4` replaces it.

Change-Id: Idd36b8cb1dff80e031b704b0335601c35c1f05f3
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Install gcc 14 on Debian trixie

Change-Id: I277db7562a0d29dde81361359eddd3f7f91b899a
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update available clang compilers for Debian Trixie

Change-Id: I603f526655e095dccd8467c1c4182f7f9f9dffa5
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: fix ci-instances playbook for Incus

Change-Id: Id05b03a6689749ef9c3acadd1a16c12cf42e3244
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

Add more arm64/armhf CI instances

Change-Id: I477ec48911326adc85c22b590ea49d8cba902e46
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: modernize dist-upgrade playbook

Use ansible.builtin.apt in a single operation.

Change-Id: Iaccd7a5aaa90fdfd5bb68ee16ef092ea1132b830
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: Add ci-master-04

Change-Id: I632de7842bc5b6fe6edf365ca14181786e5c1ab8
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add jenkins role to playbook for jenkins hosts

Change-Id: I3282369beffd760609bebeb722e5783b6d4d3f87
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add note on how to skip slow tasks

Change-Id: Iab1b508a52e4d9288015510be0f63fcc15ab6a0b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add role for installing Jenkins

Change-Id: Ia24ad484f83763bcff451fda82b64ffcdb008f79
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: netplan: fixed host netif by mac address

Since we track the bookworm-backports kernel on the CI amd64 hosts, the
name of the network interfaces may change after a kernel update. Change
the netplan config to bind the interface based on its mac address which
doesn't change.

Change-Id: I7c2a040964f22187fb6a76e1ec3cd1b6e05a2679
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

jjb: liburcu: workaround a TLS issue on Cygwin

The TLS emulation layer provided by GCC on Cygwin has a deadlock issue
with the 'test_perthreadlock'. Use our internal TLS implementation on
Cygwin to work around this.

Change-Id: Id9e8c6623715e529ba01fcb128d75e922766098f
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>

ansible: Remove group read access to netplan configuration files

Netplan warns that the configuration files shouldn't be group readable
when running `netplan apply`. This change will silence that warning.

Change-Id: I3d390f5626f92760e2a4ab7690f9c63b64ef639b
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update interfaces names for amd64 CI hosts

A recent update has cause the predictable names to include the PCI
slot number.

Change-Id: I1e411a911151f62b57048504d3da5dbc4bcb3d9f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Reorganize cluster groups in ansible inventory

Change-Id: I99bfb4b1a95be3002d60fb664d7a16c05a165f0e
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Apply lxd role only to hosts in the lxd group

This will continue to allow bootstrapping of LXD clusters if needed
for testing.

Change-Id: Ib0cffa8ad8f50e30242d638ad12c65032cbb5789
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add arm64 ci hosts to lxd_cluster_ci

Change-Id: I5e4c5d2bf9ce0d2cb8848cfa69ce5e52aea9a3af
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add incus migration steps

Change-Id: I344b9398b2a396d47ee9b934f85d21e5cbf293d5
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Update readme for bootstrapping LXD hosts

Change-Id: I0bd605d61a82ebe12c6ac3bfdac8bfa7ba02b6f8
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add test cluster

Signed-off-by: Kienan Stewart <kstewart@efficios.com>
Change-Id: I9afd73e9cbadbc5698e0562835a1f6406edf45bb

ansible: Add minimal role to install incus and incus-tools

Change-Id: Ic7026bc40e48b3f46061f8039c8c6ed77fcaac81
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Set images and backups storage volumes on hosts

By placing images and backups in the default (zfs) storage, the root
partition won't get filled with that.

Change-Id: I617cc6d1b82d28e65f8a848567e634f4a5a2516f
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Fix core pattern match

The output of /proc/sys/kernel/core_pattern is single-quoted with the
default core pattern.

Change-Id: I7da0b590afdf247cbac3b0944dfa42d9258bdbcf
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Correct typo in common Debian setup

Change-Id: I79a8f74b41c260bc8bd6bad8497ad3c7a070ed56
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add missing default variables for common-node

Required for commit c426b44b233d1a84ede09b1fcd062cfe8315ac58.

Change-Id: I0cee581202bb4f2f452748ec0bcaf7c7afb29456
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Disable, wait, and enable root nodes during snapshot

This change decreases the number of manual steps required to perform
the updates to the rootnode snapshots.

Change-Id: I0ad25a4e282498ce5f146f75dcabaa84a6eb2400
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Add helper script to perform frequent tasks against CI nodes

Change-Id: I4715027db1920dedf85512548cd0a575d7a64d1c
Signed-off-by: Kienan Stewart <kstewart@efficios.com>

ansible: Run black on ansible scripts

Change-Id: Iff19c3648aa68d3b42e2ea79ff1d4538476cfeed
Signed-off-by: Kienan Stewart <kstewart@efficios.com>