test: Make it possible to run the integration tests standalone

Currently, to run the integration tests, it's still necessary to
install various other build tools besides meson: A compiler, gperf,
libcap, ... which we want to avoid in CI systems where we receive
prebuilt systemd packages and only want to test them. Examples are
Debian's autopkgtest CI and Fedora CI. Let's make it possible for
these systems to run the integration tests without having to install
any other build dependency besides meson by extracting the logic
required to run the integration tests with meson into a separate
subdirectory and adding a standalone top-level meson.build file which
can be used to configure a meson tree with as its only purpose running
the integration tests.

Practically, we do the following:
- all the integration test directories and integration-test-wrapper.py
  are moved from test/ to test/integration-test/.
- All the installation logic is kept out of test/integration-test/ or
  any of its subdirectories and moved into test/meson.build instead.
- We add test/integration-test/standalone/meson.build to run the
  integration tests standalone. This meson file includes
  test/integration-test via a cute symlink hack to trick meson into
  including a parent directory with subdir().
- Documentation is included on how to use the new standalone mode.
- TEST-64-UDEV-STORAGE and TEST-85-NETWORK are changed to generate separate
  units for each testcase to make them behave more like the other integration
  tests.

.github/workflows/coverage.yml

@@ -13,7 +13,7 @@ on:
       - v[0-9]+-stable
     paths:
       - .github/workflows/coverage.yml
-      - test/integration-test-wrapper.py
+      - test/integration-tests/integration-test-wrapper.py
 
 permissions:
   contents: read

.github/workflows/linter.yml

@@ -37,7 +37,7 @@ jobs:
           VALIDATE_GITHUB_ACTIONS: true
 
       - name: Check that tabs are not used in Python code
-        run: sh -c '! git grep -P "\\t" -- src/boot/generate-hwids-section.py src/ukify/ukify.py test/integration-test-wrapper.py'
+        run: sh -c '! git grep -P "\\t" -- src/boot/generate-hwids-section.py src/ukify/ukify.py test/integration-tests/integration-test-wrapper.py'
 
       - name: Install ruff and mypy
         run: |
@@ -47,18 +47,18 @@ jobs:
       - name: Run mypy
         run: |
           python3 -m mypy --version
-          python3 -m mypy src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-test-wrapper.py
+          python3 -m mypy src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-tests/integration-test-wrapper.py
 
       - name: Run ruff check
         run: |
           ruff --version
-          ruff check src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-test-wrapper.py
+          ruff check src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-tests/integration-test-wrapper.py
 
       - name: Run ruff format
         run: |
           ruff --version
-          if ! ruff format --check src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-test-wrapper.py
+          if ! ruff format --check src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-tests/integration-test-wrapper.py
           then
               echo "Please run 'ruff format' on the above files or apply the diffs below manually"
-              ruff format --check --quiet --diff  src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-test-wrapper.py
+              ruff format --check --quiet --diff  src/boot/generate-hwids-section.py src/test/generate-sym-test.py src/ukify/ukify.py test/integration-tests/integration-test-wrapper.py
           fi

docs/ARCHITECTURE.md

@@ -122,7 +122,7 @@ Sources in `test/TEST-*` implement system-level testing for executables, librari
 Most of those tests should be able to run via `systemd-nspawn`, which is orders-of-magnitude faster than `qemu`, but some tests require privileged operations like using `dm-crypt` or `loopdev`.
 They are clearly marked if that is the case.
 
-See [`test/README.md`](https://github.com/systemd/systemd/blob/main/test/README.md) for more specific details.
+See [`test/integration-tests/README.md`](https://github.com/systemd/systemd/blob/main/test/integration-tests/README.md) for more specific details.
 
 ## hwdb
 

docs/CODE_QUALITY.md

@@ -22,9 +22,9 @@ available functionality:
    configure -C build -D…".)
 
 2. Run the full integration test suite as described in
-   [test/README.md](/../test/README.md). This will build OS images with a number
-   of integration tests and run them using `systemd-nspawn` and `qemu`. Requires
-   root.
+   [test/integration-tests/README.md](/../test/integration-tests/README.md).
+   This will build OS images with a number of integration tests and run them
+   using `systemd-nspawn` and `qemu`. Requires root.
 
 3. Use `./coccinelle/run-coccinelle.sh` to run all
    [Coccinelle](http://coccinelle.lip6.fr/) semantic patch scripts we ship. The

mkosi.images/build/mkosi.conf.d/opensuse/mkosi.build.chroot

@@ -64,6 +64,12 @@ if ((COVERAGE)); then
     MKOSI_CFLAGS="$MKOSI_CFLAGS -fprofile-dir=/coverage"
 fi
 
+# The opensuse spec tars up stuff in test/ and unpacks it in test/integration-tests, which we now use for our
+# own purposes, so let's get rid of that specific bit of logic in the opensuse spec until they've had a chance
+# to adapt.
+sed "pkg/$PKG_SUBDIR${GIT_SUBDIR:+/$GIT_SUBDIR}/systemd.spec" -e '1062,1075d' -e '/integration-tests\/README/d' >/tmp/systemd.spec
+mount --bind /tmp/systemd.spec "pkg/$PKG_SUBDIR${GIT_SUBDIR:+/$GIT_SUBDIR}/systemd.spec"
+
 build() {
     IFS=
     # shellcheck disable=SC2046

mkosi.postinst.chroot

@@ -58,8 +58,8 @@ SYSTEMD_REPART_MKFS_OPTIONS_EXT4="" \
     --dry-run=no \
     --size=auto \
     --offline=true \
-    --root test/TEST-24-CRYPTSETUP \
-    --definitions test/TEST-24-CRYPTSETUP/keydev.repart \
+    --root test/integration-tests/TEST-24-CRYPTSETUP \
+    --definitions test/integration-tests/TEST-24-CRYPTSETUP/keydev.repart \
     "$OUTPUTDIR/keydev.raw"
 
 can_test_pkcs11() {
@@ -132,7 +132,7 @@ EOF
     certtool --generate-self-signed \
       --load-privkey="pkcs11:token=TestToken;object=RSATestKey;type=private" \
       --load-pubkey="pkcs11:token=TestToken;object=RSATestKey;type=public" \
-      --template "test/TEST-24-CRYPTSETUP/template.cfg" \
+      --template "test/integration-tests/TEST-24-CRYPTSETUP/template.cfg" \
       --outder --outfile "/tmp/rsa_test.crt"
 
     pkcs11-tool --module "$SOFTHSM_MODULE" --token-label "TestToken" --pin "env:GNUTLS_PIN" --so-pin "env:GNUTLS_SO_PIN" --write-object "/tmp/rsa_test.crt" --type cert --label "RSATestKey"
@@ -144,7 +144,7 @@ EOF
     certtool --generate-self-signed \
       --load-privkey="pkcs11:token=TestToken;object=ECTestKey;type=private" \
       --load-pubkey="pkcs11:token=TestToken;object=ECTestKey;type=public" \
-      --template "test/TEST-24-CRYPTSETUP/template.cfg" \
+      --template "test/integration-tests/TEST-24-CRYPTSETUP/template.cfg" \
       --outder --outfile "/tmp/ec_test.crt"
 
     pkcs11-tool --module "$SOFTHSM_MODULE" --token-label "TestToken" --pin "env:GNUTLS_PIN" --so-pin "env:GNUTLS_SO_PIN" --write-object "/tmp/ec_test.crt" --type cert --label "ECTestKey"

test/TEST-52-HONORFIRSTSHUTDOWN/meson.build

@@ -1,9 +0,0 @@
-# SPDX-License-Identifier: LGPL-2.1-or-later
-
-integration_tests += [
-        integration_test_template + {
-                'name' : fs.name(meson.current_source_dir()),
-        },
-]
-
-testdata_subdirs += [meson.current_source_dir() / 'TEST-52-HONORFIRSTSHUTDOWN.units']

test/integration-tests/README.md

@@ -151,6 +151,31 @@ options to provide mkosi with a directory containing the systemd packages or a
 repository file that points to a repository with the systemd packages that
 should be installed.
 
+If the rpms are installed on the host system that the tests are running on,
+you'll probably want to disable usage of the tools tree so that the tools from
+the host system are used to build the image by adding the following to
+`mkosi.local.conf`:
+
+```conf
+[Build]
+ToolsTree=
+```
+
+On the other hand, if the rpms are available but not installed on the host
+system, you'll want to make sure they're installed into the mkosi tools tree so
+that they're used to build the image by using either
+`ToolsTreePackageDirectories=` or `ToolsTreeSandboxTrees=` similarly to
+`PackageDirectories=` or `SandboxTrees=` mentioned above.
+
+Finally, we'll make use of the standalone mode of running the integration tests
+to avoid having to install any build dependencies.
+
+```sh
+$ mkosi -f sandbox -- meson setup testsuite test/integration-tests/standalone
+$ mkosi -f
+$ mkosi sandbox -- meson test -C testsuite --num-processes "$(($(nproc) / 4))"
+```
+
 ### SELinux AVCs
 
 To have `TEST-06-SELINUX` check for SELinux denials, write the following to

test/integration-tests/TEST-04-JOURNAL/meson.build

@@ -8,5 +8,3 @@ integration_tests += [
                 'storage' : 'persistent',
         },
 ]
-
-testdata_subdirs += [meson.current_source_dir() / 'TEST-04-JOURNAL.units']

test/integration-tests/TEST-06-SELINUX/meson.build

@@ -13,5 +13,3 @@ integration_tests += [
                 'mkosi-args' : integration_test_template['mkosi-args'] + ['--runtime-build-sources=no'],
         },
 ]
-
-testdata_subdirs += [meson.current_source_dir() / 'TEST-06-SELINUX.units']

test/integration-tests/TEST-07-PID1/meson.build

@@ -6,5 +6,3 @@ integration_tests += [
                 'unit' : files('TEST-07-PID1.service'),
         },
 ]
-
-testdata_subdirs += [meson.current_source_dir() / 'TEST-07-PID1.units']

test/integration-tests/TEST-16-EXTEND-TIMEOUT/meson.build

@@ -7,5 +7,3 @@ integration_tests += [
                 'coredump-exclude-regex' : '/(bash|sleep)$',
         },
 ]
-
-testdata_subdirs += [meson.current_source_dir() / 'TEST-16-EXTEND-TIMEOUT.units']