Skip to content

chore(ci): Fix parent POM dependency change detection#22022

Merged
gnodet merged 2 commits into
mainfrom
detect-dependencies-fix
Mar 16, 2026
Merged

chore(ci): Fix parent POM dependency change detection#22022
gnodet merged 2 commits into
mainfrom
detect-dependencies-fix

Conversation

@gnodet
Copy link
Copy Markdown
Contributor

@gnodet gnodet commented Mar 16, 2026

Summary

The detect-dependencies CI action was never finding affected modules for dependabot PRs that bump versions in parent/pom.xml.

Root cause: The script searched for ${property-name} (e.g., ${jaxb-impl-version}) in module pom.xml files. This never matches because modules inherit managed versions from the parent's <dependencyManagement> without referencing the property directly — they just declare <artifactId>jaxb-impl</artifactId> without a <version>.

Fix: New approach that maps properties to artifacts to modules:

  1. Diff parent/pom.xml to find changed property names
  2. Parse parent/pom.xml to find which <artifactId> uses each property as its <version> in <dependencyManagement>
  3. Search module pom.xml files for those artifact IDs
  4. Run tests for affected modules

Example: For PR #22019 (jaxb-impl-version bump), the old script found 0 modules. The new script correctly finds camel-drill, camel-ibm-cos, and camel-swift.

Additional improvements

  • Posts a PR comment summarizing which modules were tested and whether tests passed/failed
  • Writes step summary output
  • Removes the overly restrictive version|dependency|artifact keyword filter on property names
  • Adds proper skip-mvnd-install input to avoid reinstalling mvnd
  • Only runs on non-experimental matrix builds (skips JDK 25)
  • Caps at 50 modules to avoid excessive test runs

Test plan

  • Verified locally that jaxb-impl-versionjaxb-implcamel-drill, camel-ibm-cos, camel-swift detection works
  • CI will validate the script runs without errors
  • Next dependabot PR touching parent/pom.xml will exercise the full flow

The detect-dependencies action was not finding affected modules because
it searched for ${property-name} references in module pom.xml files.
This never matches because modules inherit managed versions from the
parent without referencing the property directly.

New approach:
1. Detect changed properties in parent/pom.xml diff
2. Map each property to the artifact(s) it versions in dependencyManagement
3. Find modules that depend on those artifacts by searching for artifactId
4. Run tests for affected modules

Also adds:
- PR comment posting with test results summary
- Step summary output
- Proper skip-mvnd-install input in action.yaml
- Run only on non-experimental matrix (skip JDK 25)
- Remove overly restrictive version|dependency|artifact filter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using build-all, build-dependents, skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

Instead of grepping pom.xml files for artifactId references (which
misses transitive dependencies), use toolbox:tree-find to scan the
full dependency tree of every module in the reactor.

This accurately detects modules affected by BOM version bumps (e.g.,
netty-bom -> all modules using any io.netty artifact transitively).

Key changes:
- Extract groupId:artifactId from parent pom (not just artifactId)
- Detect BOM imports and search by groupId wildcard
- Use awk to parse toolbox output (mvnd strips module prefixes
  when captured to variables)
- Add coverage to exclusion list

Tested locally: jaxb-impl-version correctly finds camel-parquet-avro
(transitive via hadoop-common -> jersey-json -> jaxb-impl).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gnodet gnodet merged commit 5ccc089 into main Mar 16, 2026
2 checks passed
vignesh-manel pushed a commit to vignesh-manel/camel that referenced this pull request Mar 18, 2026
* chore(ci): Fix parent POM dependency change detection

The detect-dependencies action was not finding affected modules because
it searched for ${property-name} references in module pom.xml files.
This never matches because modules inherit managed versions from the
parent without referencing the property directly.

New approach:
1. Detect changed properties in parent/pom.xml diff
2. Map each property to the artifact(s) it versions in dependencyManagement
3. Find modules that depend on those artifacts by searching for artifactId
4. Run tests for affected modules

Also adds:
- PR comment posting with test results summary
- Step summary output
- Proper skip-mvnd-install input in action.yaml
- Run only on non-experimental matrix (skip JDK 25)
- Remove overly restrictive version|dependency|artifact filter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(ci): Use Maveniverse Toolbox for accurate dependency detection

Instead of grepping pom.xml files for artifactId references (which
misses transitive dependencies), use toolbox:tree-find to scan the
full dependency tree of every module in the reactor.

This accurately detects modules affected by BOM version bumps (e.g.,
netty-bom -> all modules using any io.netty artifact transitively).

Key changes:
- Extract groupId:artifactId from parent pom (not just artifactId)
- Detect BOM imports and search by groupId wildcard
- Use awk to parse toolbox output (mvnd strips module prefixes
  when captured to variables)
- Add coverage to exclusion list

Tested locally: jaxb-impl-version correctly finds camel-parquet-avro
(transitive via hadoop-common -> jersey-json -> jaxb-impl).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@squakez
Copy link
Copy Markdown
Contributor

squakez commented Mar 26, 2026

Example: For PR #22019 (jaxb-impl-version bump), the old script found 0 modules. The new script correctly finds camel-drill, camel-ibm-cos, and camel-swift.

For the record, this assumption is wrong. Those components are not using the dependency, but they are excluding it, see:

                <exclusion>
                    <groupId>com.sun.xml.bind</groupId>
                    <artifactId>jaxb-impl</artifactId>
                </exclusion>

The script was working correctly and this change was merged based on a wrong assumption.

gnodet added a commit that referenced this pull request Mar 26, 2026
This reverts the detect-dependencies CI action back to its original
implementation. The reworked version is causing issues in CI.
A proper fix will be done in #22247.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 26, 2026
Reverts the detect-dependencies CI action back to its original
implementation. The reworked version was causing issues in CI.
A proper fix will be done in #22247.
gnodet added a commit to gnodet/camel that referenced this pull request Mar 26, 2026
- Restore maxNumberOfTestableProjects to 50 (was accidentally changed to 1000)
- Restore root project change handling: exit early with informative comment
- Restrict POM dependency analysis to parent/pom.xml only (matches original
  detect-test.sh behavior; detection improvements deferred to follow-up PR)
- Add explicit comment about intentional Toolbox removal (pre-apache#22022 approach)
- Add skip_full_build input for /component-test dispatches to avoid full
  regen.sh build (uses quick targeted build with -Dquickly instead)
- Document multi-JDK artifact upload behavior (intentional overwrite for
  resilience when one JDK fails)
- Update CI-ARCHITECTURE.md to reflect all changes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 26, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit to gnodet/camel that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert apache#22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
gnodet added a commit that referenced this pull request Mar 27, 2026
Merge the three separate CI actions (detect-dependencies, component-test,
incremental-build) into a single incremental-build action and streamline
the workflow files:

- Merge POM dependency detection (from detect-dependencies) and
  /component-test logic into incremental-build.sh
- Rationalize pr-build-main.yml to support both PR builds and
  workflow_dispatch with extra_modules and skip_full_build inputs
- Replace pr-manual-component-test.yml with a lightweight dispatcher
  that resolves component names and dispatches pr-build-main.yml
- Add pr-test-commenter.yml (workflow_run) to post unified test
  summary comments on PRs (including fork PRs)
- Generate a single unified PR comment combining file-path analysis,
  POM dependency detection, and /component-test results
- Add CI-ARCHITECTURE.md documenting the workflow ecosystem
- Revert #22022 (detect-dependencies action) since its functionality
  is now integrated into incremental-build
- Delete obsolete actions: detect-dependencies, component-test

Bug fixes found during CI testing:
- Fix pipefail issue in dependent module count (set -o pipefail
  caused mvn|wc||echo to produce multi-line output)
- Use ${{ github.repository }} instead of hardcoded 'apache/camel'
  in action.yaml default for github-repo input
- Only append module exclusion list when using -amd flag (exclusion
  modules must be in the reactor to be excluded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants