Introduce Depends-on for lttng-tools gerrit job

The development of a lttng-tools feature can happen in parallel with a
lttng-ust feature. The need to build and test a lttng-tools gerrit
change against a gerrit lttng-ust change arises.

This patch introduce gerrit-based inter project dependency
testing for the lttng-tools project against lttng-ust.

The dependency can be expressed at the change's commit message level:

    Depends-on: lttng-ust: <lttng-ust-change-id>

e.g
    Fix: client_flush_outgoing_queue expect an active client

    Move validated and active assignation before the call to
    client_flush_outgoing_queue.

    Depends-on: lttng-ust: If8557b77f2a03b05b924cdc26ecf7618fff46289

    Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
    Change-Id: I873b2fad567eb6d59a53a12cba02451afd1d9827

For now, only lttng-ust is supported but babeltrace and lttng-modules
could someday be prime candidate for "Depends-on".

Note that the "Depends-on" line is optional. If none is present, we
import lttng-ust from the lttng-ust base job as it is done today.

For building ust, we reuse the script "scripts/lttng-ust/build.sh".

Note that we override the "$build" variable to "std" for the dependency
since the build type for the lttng-ust dependency is irrelevant for the
current job.

Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>

diff --git a/jobs/lttng-tools.yaml b/jobs/lttng-tools.yaml
index 548ba6a7b786b3e344fefafa4adefde92c10e893..e22cb92d666b7203facd9ad06a8c128c6c2be172 100644 (file)
--- a/jobs/lttng-tools.yaml
+++ b/jobs/lttng-tools.yaml
@@ -213,6 +213,132 @@
       - shell:
          !include-raw-escape: scripts/lttng-tools/build.sh
 
+- lttng-tools_build_builders_gerrit: &lttng-tools_build_builders_gerrit
+    name: 'lttng-tools_build_builders_gerrit'
+    builders:
+      - shell:
+         !include-raw-escape: scripts/lttng-tools/gerrit-depends-on.sh
+      - inject:
+          properties-file: gerrit_custom_dependencies.properties
+      - conditional-step:
+          condition-kind: and
+          on-evaluation-failure: run
+          condition-operands:
+              - condition-kind: shell
+                condition-command: 'test -z "$GERRIT_DEP_LTTNG_UST"'
+              - condition-kind: regex-match
+                label: '$conf'
+                regex: (std|static|agents|debug-rcu)
+          steps:
+            - copyartifact:
+                project: lttng-ust_{ustversion}_{buildtype}/liburcu_version=${{liburcu_version}},arch=${{arch}},conf=${{conf}},build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+      - conditional-step:
+          condition-kind: regex-match
+          label: '$conf'
+          regex: (std|static)
+          on-evaluation-failure: run
+          steps:
+            - copyartifact:
+                project: liburcu_${{liburcu_version}}_{buildtype}/arch=${{arch}},conf=${{conf}},build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+            - copyartifact:
+                project: babeltrace_${{babeltrace_version}}_{buildtype}/arch=${{arch}},conf=${{conf}},build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+      - conditional-step:
+          condition-kind: regex-match
+          label: '$conf'
+          regex: no-ust
+          on-evaluation-failure: run
+          steps:
+            - copyartifact:
+                project: liburcu_${{liburcu_version}}_{buildtype}/arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+            - copyartifact:
+                project: babeltrace_${{babeltrace_version}}_{buildtype}/arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+            - copyartifact:
+                project: lttng-ust_{ustversion}_{buildtype}/liburcu_version=${{liburcu_version}},arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+      - conditional-step:
+          condition-kind: regex-match
+          label: '$conf'
+          regex: (agents|relayd-only)
+          on-evaluation-failure: run
+          steps:
+            - copyartifact:
+                project: liburcu_${{liburcu_version}}_{buildtype}/arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+            - copyartifact:
+                project: babeltrace_${{babeltrace_version}}_{buildtype}/arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+      - conditional-step:
+          condition-kind: regex-match
+          label: '$conf'
+          regex: debug-rcu
+          on-evaluation-failure: run
+          steps:
+            - copyartifact:
+                project: liburcu_${{liburcu_version}}_{buildtype}/arch=${{arch}},conf=${{conf}},build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+            - copyartifact:
+                project: babeltrace_${{babeltrace_version}}_{buildtype}/arch=${{arch}},conf=std,build=std
+                which-build: last-successful
+                stable: false
+                filter: 'build/**'
+                target: 'deps'
+                do-not-fingerprint: true
+      - shell:
+         !include-raw-escape: scripts/lttng-tools/clean_processes_coredump.sh
+      - conditional-step:
+          condition-kind: shell
+          condition-command: 'test ! -z "$GERRIT_DEP_LTTNG_UST"'
+          steps:
+            - shell:
+               !include-raw-escape:
+                   - scripts/lttng-tools/gerrit-build-override.sh
+                   - scripts/lttng-ust/build.sh
+      - shell:
+         !include-raw-escape: scripts/lttng-tools/gerrit-install-deps.sh
+      - shell:
+         !include-raw-escape: scripts/lttng-tools/build.sh
+
 - lttng-tools_build_builders_win: &lttng-tools_build_builders_win
     name: 'lttng-tools_build_builders_win'
     builders:
@@ -528,7 +654,7 @@
           trigger-for-unreviewed-patches: true
 
     <<: *lttng-tools_build_axes_defaults
-    <<: *lttng-tools_build_builders_defaults
+    <<: *lttng-tools_build_builders_gerrit
 
     publishers:
       - postbuildscript:

diff --git a/scripts/lttng-tools/gerrit-build-override.sh b/scripts/lttng-tools/gerrit-build-override.sh
new file mode 100755 (executable)
index 0000000..003ee7a
--- /dev/null
+++ b/scripts/lttng-tools/gerrit-build-override.sh
@@ -0,0 +1,22 @@
+#!/bin/bash -exu
+#
+# Copyright (C) 2020 Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# This file should be used as a jenkins job builder RAW import allowing the
+# override of the "build" variable on shell builder execution.
+
+# shellcheck disable=SC2034
+build=std

diff --git a/scripts/lttng-tools/gerrit-depends-on.sh b/scripts/lttng-tools/gerrit-depends-on.sh
new file mode 100755 (executable)
index 0000000..062323b
--- /dev/null
+++ b/scripts/lttng-tools/gerrit-depends-on.sh
@@ -0,0 +1,65 @@
+#!/bin/bash -exu
+# shellcheck disable=SC2103
+#
+# Copyright (C) 2020 Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+#Required variables
+GERRIT_NAME=${GERRIT_NAME:-}
+WORKSPACE=${WORKSPACE:-}
+conf=${conf:-}
+
+gerrit_url="https://${GERRIT_NAME}"
+gerrit_query="?o=CURRENT_REVISION&o=DOWNLOAD_COMMANDS"
+gerrit_json_query=".revisions[.current_revision].ref"
+
+possible_depends_on="lttng-ust"
+re="Depends-on: (${possible_depends_on}): ([^'$'\n'']*)"
+property_file="${WORKSPACE}/gerrit_custom_dependencies.properties"
+
+# Create the property file even if it ends up being empty
+touch "$property_file"
+
+# Move to lttng-tools source directory
+pushd "${WORKSPACE}/src/lttng-tools"
+
+git rev-list --format=%B --max-count=1 HEAD | while read -r line; do
+    if ! [[ ${line} =~ ${re} ]]; then
+        continue
+    fi
+
+    project=${BASH_REMATCH[1]}
+    gerrit_id=${BASH_REMATCH[2]}
+
+    project_sanitize=${BASH_REMATCH[1]//-/_}
+
+    if [ "$conf" = "no-ust" ] && [ "$project" = "lttng-ust" ]; then
+        # No need to checkout lttng-ust for this configuration axis
+        continue
+    fi
+
+    # Export the GERRIT_DEP_... into the property file for further jenkins usage
+    echo "GERRIT_DEP_${project_sanitize^^}=${gerrit_id}" >> "$property_file"
+
+    # Get the change latest ref
+    ref=$(curl "${gerrit_url}/changes/${gerrit_id}${gerrit_query}" | tail -n+2 | jq -r "$gerrit_json_query")
+    git clone "${gerrit_url}/${project}" "$WORKSPACE/src/$project"
+    pushd "$WORKSPACE/src/$project"
+    git fetch "${gerrit_url}/${project}" "$ref"
+    git checkout FETCH_HEAD
+    popd
+done
+
+popd

diff --git a/scripts/lttng-tools/gerrit-install-deps.sh b/scripts/lttng-tools/gerrit-install-deps.sh
new file mode 100755 (executable)
index 0000000..a04f82c
--- /dev/null
+++ b/scripts/lttng-tools/gerrit-install-deps.sh
@@ -0,0 +1,24 @@
+#!/bin/bash -exu
+#
+# Copyright (C) 2020 Jonathan Rajotte-Julien <jonathan.rajotte-julien@efficios.com>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Required variables
+WORKSPACE=${WORKSPACE:-}
+
+# Move any custom dependencies to the deps folder
+if [ -d "$WORKSPACE/build/" ]; then
+    cp -r "$WORKSPACE/build/" "$WORKSPACE/deps/"
+fi