Initial integration tests by WilliamRoebuck · Pull Request #194 · eclipse-score/lifecycle

WilliamRoebuck · 2026-05-12T09:55:49Z

Implemented the following tests:

Improvements/fixes to test helper & smoke test

github-actions · 2026-05-12T09:56:45Z

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //:license-check

Status: ⚠️ Needs Review

Click to expand output

[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.4.2) and connecting to it...
INFO: Invocation ID: bedf546c-c409-445f-95fa-affcdd6a1748
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Analyzing: target //:license-check (1 packages loaded)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (29 packages loaded, 10 targets configured)

Analyzing: target //:license-check (85 packages loaded, 10 targets configured)

Analyzing: target //:license-check (137 packages loaded, 1196 targets configured)

Analyzing: target //:license-check (145 packages loaded, 4887 targets configured)

Analyzing: target //:license-check (157 packages loaded, 8082 targets configured)

Analyzing: target //:license-check (157 packages loaded, 8094 targets configured)

Analyzing: target //:license-check (160 packages loaded, 8223 targets configured)

INFO: Analyzed target //:license-check (162 packages loaded, 10232 targets configured).
[12 / 16] JavaToolchainCompileClasses external/rules_java+/toolchains/platformclasspath_classes; 0s disk-cache, processwrapper-sandbox
[14 / 16] JavaToolchainCompileBootClasspath external/rules_java+/toolchains/platformclasspath.jar; 0s disk-cache, processwrapper-sandbox
[15 / 16] Building license.check.license_check.jar (); 0s disk-cache, multiplex-worker
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 20.608s, Critical Path: 2.35s
INFO: 16 processes: 12 internal, 3 processwrapper-sandbox, 1 worker.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

github-actions · 2026-05-13T10:59:19Z

The created documentation from the pull request is available at: docu-html

* Add process args test eclipse-score#172 * Add run target switch test eclipse-score#174 * Add runtime crash test eclipse-score#177 * Add monitoring failure test eclipse-score#178 * Add crash on startup test eclipse-score#179

NicolasFussberger · 2026-05-18T09:32:08Z

+int main()
+{
+    deploy_and_crash_if_not_present("crashed_once");
+    deploy_and_crash_if_not_present("crashed_twice");


Where are these files being cleaned up again?

NicolasFussberger · 2026-05-18T09:44:16Z

+    derivation_technique="requirements-analysis",
+)
+def test_crash_on_startup(target, setup_test, test_output_dir, remote_test_dir):
+    """Integration test verifying that the launch manager correctly handles processes


I think we need some more description of what is being tested here and what is the expected behaviour.
The process is asking for this information:

The objective of the test.

Inputs

Expected outcome (e.g. “A success message is displayed.”)

Test environment (e.g. network configuration, clean system state)

I know there is more info in control_client_mock.cpp available, but I assume we need this info in a test spec as part of a test report at least at a high level.

Maybe something like:

""" Objective: Verifies that the launch manager correctly handles processes that crash before reporting kRunning. Case 1: Process crashes before Running state but eventually starts up successfully before the configured number of restart attempts is exceeded. Expected Behaviour: Process startup successful, RunTarget activation successful Case 2: Process keeps crashing, exceeding the number of restart attempts. Expected Behaviour: Process startup fails, LaunchManager executes recovery action. """

NicolasFussberger · 2026-05-18T09:51:13Z

+#include <score/lcm/control_client.h>
+#include <score/lcm/lifecycle_client.h>
+
+score::lcm::ControlClient client;


After merging #191 we cannot create the ControlClient in global scope as mw::log might not be initialized yet and does not support logging at this point in time.
Same for all other tests

Signed-off-by: Timo Steuerwald <142218678+TimoSteuerwaldETAS@users.noreply.github.com>

NicolasFussberger · 2026-05-22T06:57:49Z

+    TEST_STEP("Check args")
+    {
+        ASSERT_GT(g_argc, 1) << "Not enough arguments";
+        EXPECT_STREQ(g_argv[1], "S-CORE rules!") << "Second argument was not as expected";


Should this also check that the first argument is the binary name?

Done in a048389

NicolasFussberger · 2026-05-26T05:36:31Z

-    TEST_STEP("Activate RunTarget Running") {
+
+    // We have to wait for the initial state transition to fully complete, otherwise unexpected failures can occur
+    sleep(1);


For these workarounds, please always link the open problem #198

Fixed in a048389

NicolasFussberger · 2026-05-26T05:39:27Z

+    download_xml_results(target, remote_test_dir, test_output_dir)
+    all_files, failing_files = check_for_failures(test_output_dir)
+    assert len(all_files) == 2, f"Didn't find the expected number of files {all_files}"
+    assert len(failing_files) == 0, f"Found failures in files {failing_files}"


I wonder if we can abstract from this somehow.
This comparison of created files seems to be one of the main asserts deciding whether the test fails or succeeds.

Currently, it is based on comparing a raw number where it is easy to introduce one-off mistake. Maybe something like listing the components that were expected to start or something like that which is more explicit?

Done in 55c9b98 , also updated them to use the fixture

WilliamRoebuck requested a deployment to workflow-approval May 12, 2026 09:55 — with GitHub Actions Waiting

WilliamRoebuck force-pushed the feature/integration-tests-1 branch from bfff457 to 784bdd7 Compare May 12, 2026 10:20

WilliamRoebuck requested a deployment to workflow-approval May 12, 2026 10:20 — with GitHub Actions Waiting

WilliamRoebuck requested a deployment to workflow-approval May 12, 2026 10:21 — with GitHub Actions Waiting

WilliamRoebuck marked this pull request as draft May 12, 2026 11:19

WilliamRoebuck requested a deployment to workflow-approval May 13, 2026 10:54 — with GitHub Actions Waiting

WilliamRoebuck marked this pull request as ready for review May 13, 2026 11:36

WilliamRoebuck added 3 commits May 13, 2026 12:37

Initial integration tests

f19157b

* Add process args test eclipse-score#172 * Add run target switch test eclipse-score#174 * Add runtime crash test eclipse-score#177 * Add monitoring failure test eclipse-score#178 * Add crash on startup test eclipse-score#179

Fix pipeline failure

63430ad

Fixed some flaky issues

f82bf7e

WilliamRoebuck force-pushed the feature/integration-tests-1 branch from 937434e to f82bf7e Compare May 13, 2026 11:37

WilliamRoebuck temporarily deployed to workflow-approval May 13, 2026 11:37 — with GitHub Actions Inactive

WilliamRoebuck mentioned this pull request May 13, 2026

Spurious recovery action trigger on first run target activation #198

Open

2 tasks

NicolasFussberger requested a review from TimoSteuerwaldETAS May 18, 2026 08:39

NicolasFussberger reviewed May 18, 2026

View reviewed changes

NicolasFussberger requested changes May 20, 2026

View reviewed changes

TimoSteuerwaldETAS added 2 commits May 20, 2026 14:11

code review rework mainly regarding test specification

fd1ba96

code review rework: Move ControlClient variable to test

044edf0

TimoSteuerwaldETAS requested a deployment to workflow-approval May 20, 2026 14:07 — with GitHub Actions Waiting

Merge branch 'main' into feature/integration-tests-1

9a061a2

Signed-off-by: Timo Steuerwald <142218678+TimoSteuerwaldETAS@users.noreply.github.com>

TimoSteuerwaldETAS requested a deployment to workflow-approval May 20, 2026 14:45 — with GitHub Actions Waiting

NicolasFussberger reviewed May 22, 2026

View reviewed changes

TimoSteuerwaldETAS requested a deployment to workflow-approval May 22, 2026 14:50 — with GitHub Actions Waiting

NicolasFussberger reviewed May 26, 2026

View reviewed changes

Adapt cfg & add sleep for initial rt

84d0c5e

TimoSteuerwaldETAS force-pushed the feature/integration-tests-1 branch from 96c73a1 to 84d0c5e Compare May 26, 2026 11:04

WilliamRoebuck added 2 commits May 26, 2026 12:18

Tidy test helper

12e9e33

Add check for first argument, add links to issue

a048389

WilliamRoebuck deployed to workflow-approval May 26, 2026 14:29 — with GitHub Actions Active

WilliamRoebuck temporarily deployed to workflow-approval May 26, 2026 14:29 — with GitHub Actions Inactive

WilliamRoebuck added 2 commits May 26, 2026 15:48

Check files instead of count of files

55c9b98

Increase heartbeat delay to avoid double heartbeat

0b140fb

WilliamRoebuck requested a deployment to workflow-approval May 26, 2026 14:49 — with GitHub Actions Waiting

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial integration tests#194

Initial integration tests#194
WilliamRoebuck wants to merge 11 commits into
eclipse-score:mainfrom
etas-contrib:feature/integration-tests-1

WilliamRoebuck commented May 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 12, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026

Uh oh!

NicolasFussberger May 18, 2026

Uh oh!

NicolasFussberger May 18, 2026

Uh oh!

NicolasFussberger May 18, 2026

Uh oh!

NicolasFussberger May 22, 2026

Uh oh!

WilliamRoebuck May 26, 2026 •

edited

Loading

Uh oh!

NicolasFussberger May 26, 2026

Uh oh!

WilliamRoebuck May 26, 2026 •

edited

Loading

Uh oh!

NicolasFussberger May 26, 2026

Uh oh!

WilliamRoebuck May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

WilliamRoebuck commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

License Check Results

Uh oh!

github-actions Bot commented May 13, 2026

Uh oh!

NicolasFussberger May 18, 2026

Choose a reason for hiding this comment

Uh oh!

NicolasFussberger May 18, 2026

Choose a reason for hiding this comment

Uh oh!

NicolasFussberger May 18, 2026

Choose a reason for hiding this comment

Uh oh!

NicolasFussberger May 22, 2026

Choose a reason for hiding this comment

Uh oh!

WilliamRoebuck May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicolasFussberger May 26, 2026

Choose a reason for hiding this comment

Uh oh!

WilliamRoebuck May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicolasFussberger May 26, 2026

Choose a reason for hiding this comment

Uh oh!

WilliamRoebuck May 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

WilliamRoebuck commented May 12, 2026 •

edited

Loading

github-actions Bot commented May 12, 2026 •

edited

Loading

WilliamRoebuck May 26, 2026 •

edited

Loading

WilliamRoebuck May 26, 2026 •

edited

Loading