Skip to content

CAMEL-23594: Fix property resolution issues in kamelet endpoint URIs#23391

Open
cunningt wants to merge 1 commit into
apache:mainfrom
cunningt:urldecodefixkamelets
Open

CAMEL-23594: Fix property resolution issues in kamelet endpoint URIs#23391
cunningt wants to merge 1 commit into
apache:mainfrom
cunningt:urldecodefixkamelets

Conversation

@cunningt
Copy link
Copy Markdown
Contributor

Problem

Kamelet integration tests were failing due to two property resolution issues:

  1. Trailing whitespace in property values — YAML literal block scalars (|) preserve trailing newlines, which cause URISyntaxException when the property value is resolved into an endpoint URI string.

  2. URL-encoded property values — The YAML DSL uses URISupport.createQueryString() to build kamelet URIs, which URL-encodes property values (e.g., application/jsonapplication%2Fjson). Since KameletComponent uses useRawUri()=true to preserve sensitive values, automatic URL-decoding during URI parsing is skipped, leaving encoded values like application%2Fjson to be passed to kamelet templates.

Solution

Fix 1: Trim whitespace in DefaultPropertiesParser

File: core/camel-base/src/main/java/org/apache/camel/component/properties/DefaultPropertiesParser.java

Added .trim() when returning resolved property values in doGetPropertyValue(). This strips leading/trailing whitespace, which is safe since no legitimate property value depends on surrounding whitespace.

Fix 2: URL-decode kamelet parameters

File: components/camel-kamelet/src/main/java/org/apache/camel/component/kamelet/KameletComponent.java

Added URL-decoding for non-RAW string parameter values in createEndpoint(), between extraction and RAW resolution. This reverses the encoding done by URISupport.createQueryString() while preserving RAW-wrapped sensitive values (which use RAW() syntax and skip URL encoding).

Testing

  • ✅ All 91 property component tests pass (org.apache.camel.component.properties.*Test)
  • ✅ All 81 kamelet component tests pass
  • ✅ All 337 YAML DSL tests pass

These fixes resolve kamelet integration test failures where:

  • Property values with trailing newlines caused URISyntaxException
  • URL-encoded values like application%2Fjson were incorrectly passed as Content-Type headers, causing HTTP 415 errors

Related Issues

CAMEL-23594

Two related fixes for property values used in kamelet endpoint URIs:

1. Trim whitespace from resolved property values in DefaultPropertiesParser.
   Property values containing trailing whitespace (e.g., from YAML literal
   block scalars using "|") cause URISyntaxException when resolved into
   endpoint URIs. The trim is safe since legitimate property values never
   depend on surrounding whitespace.

2. URL-decode kamelet endpoint parameters in KameletComponent.
   When YAML DSL creates kamelet URIs, it URL-encodes property values
   (e.g., "application/json" → "application%2Fjson"). Since KameletComponent
   uses useRawUri=true to preserve sensitive values, automatic URL-decoding
   is skipped. This fix manually decodes non-RAW parameter values so they
   are passed correctly to kamelet templates.

Together these fixes resolve kamelet integration test failures where
property values were either malformed (trailing newline causing
URISyntaxException) or incorrectly encoded (Content-Type header sent
as "application%2Fjson" instead of "application/json").

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@github-actions
Copy link
Copy Markdown
Contributor

🧪 CI tested the following changed modules:

  • components/camel-kamelet
  • core/camel-base

ℹ️ Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the test-dependents label to force testing all dependents.

Build reactor — dependencies compiled but only changed modules were tested (2 modules)
  • Camel :: Base
  • Camel :: Kamelet

⚙️ View full build and test results

Copy link
Copy Markdown
Contributor

@davsclaus davsclaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code on behalf of Claus Ibsen

Thanks for tackling this issue — the URL-encoding problem in kamelet parameters is a real pain point. I have a few concerns with the current approach that I think are worth addressing before merging.

Summary

  1. DefaultPropertiesParser.trim() is too broad — This affects every property resolution across the entire framework, not just YAML/kamelet scenarios. Property values that legitimately contain leading/trailing whitespace (passwords, PEM content, regex patterns, template strings) would be silently corrupted. The root cause — YAML literal block scalars (|) preserving trailing newlines — should be fixed at the YAML parsing layer, not in the global properties parser.

  2. No tests included — Per project guidelines, every PR must include tests for bug fixes. At minimum: a test showing URL-encoded kamelet parameter values are properly decoded, a test for the properties parser whitespace handling, and a regression test ensuring RAW-wrapped values are not double-decoded.

  3. URL-decoding may affect non-YAML code pathscreateEndpoint() is called from all DSLs (Java, XML, YAML). If a value arrives without encoding (e.g., from Java DSL), URLDecoder.decode() could corrupt values containing % characters (e.g., 100% would trigger decode errors). The fix should either be guarded to YAML-originated calls, or the encoding should be fixed at the source (YAML DSL query string construction) rather than decoded at the destination.

  4. Missing upgrade guide entry — The trim() change in DefaultPropertiesParser is a behavior change to a core component that should be documented.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

answer = value;
}
return answer;
return answer != null ? answer.trim() : null;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .trim() applies to every property resolution across the entire framework, not just YAML/kamelet scenarios. Property values that legitimately contain leading/trailing whitespace (passwords with trailing spaces, PEM data, regex patterns) would be silently corrupted.

The root cause is YAML literal block scalars (|) preserving trailing newlines. This should be fixed at the YAML parsing layer where the newline is introduced, not in the global properties parser.

Consider removing this change and instead trimming at the YAML DSL level where property values are extracted from block scalars.

if (entry.getValue() instanceof String s
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(")
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) {
entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createEndpoint() is called from all DSLs (Java, XML, YAML), not just YAML. If a value arrives without URL-encoding (e.g., from Java DSL to("kamelet:foo?contentType=application/json")), URLDecoder.decode() will still process it. While application/json survives decoding, a value containing % (like 100%done) would be corrupted or throw an IllegalArgumentException.

Consider either:

  • Fixing the encoding at the source (the YAML DSL's URISupport.createQueryString() call) so values aren't double-encoded
  • Or guarding this decode with a check that the value actually contains URL-encoded sequences (e.g., %xx patterns)

@davsclaus
Copy link
Copy Markdown
Contributor

I think the double decoding is a bit weird. Also can you point to which kamelts yaml files are having problems. It can be that they do wrong yaml - when you have multi line its |= vs >= etc

@oscerd oscerd self-requested a review May 21, 2026 08:22
Copy link
Copy Markdown
Contributor

@gnodet gnodet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this, Tom. The URL-encoding asymmetry between useRawUri()=true and the YAML DSL's createQueryString(params) is a real problem. I have two concerns with the current approach -- one blocking (the trim() change) and one that I think needs discussion (the URL-decode fix).

1. DefaultPropertiesParser.trim() -- too broad, risk of data corruption (blocking)

This change applies to every property resolution across the entire Camel framework, not just kamelet/YAML scenarios. Properties resolved via {{...}} placeholders are used for passwords, PEM certificates, regex patterns, file paths, and many other values where leading/trailing whitespace could be significant.

The root cause is that YAML literal block scalars (|) preserve trailing newlines. The right fix should be at the YAML parsing layer where the values are extracted, not in the global properties parser. The DefaultPropertiesParser has been stable for years and changing it silently could break existing users who depend on whitespace-preserving property values (e.g., System.getProperty() or environment variables that intentionally contain whitespace).

I would recommend removing this part of the change entirely, and if trailing newlines from YAML block scalars are a problem, fix it in the YAML DSL property extraction logic.

2. URL-decoding in KameletComponent.createEndpoint() -- fragile

The URL-decode fix works around the encoding done by URISupport.createQueryString(params) (which uses encode=true by default) in YamlRoutesBuilderLoader line 605. However:

  • createEndpoint() is called from ALL DSLs (Java, XML, YAML), not just YAML. When a Java DSL route uses to("kamelet:foo?contentType=application/json"), the value application/json arrives unencoded. While URLDecoder.decode() is a no-op for that specific value, any value containing a bare % (like 100%complete or a regex [a-z]%[0-9]) will throw IllegalArgumentException.

  • The fix in CAMEL-23284 took the opposite approach for {{placeholder}} encoding: it un-encoded at the source (query.replace("%7B%7B", "{{").replace("%7D%7D", "}}") in YamlRoutesBuilderLoader). Fixing the encoding at the source would be more consistent and safer -- e.g., using createQueryString(params, false) for kamelet URIs in YamlRoutesBuilderLoader, similar to what KameletDeserializer and YamlSupport already do.

  • If decoding at the destination is preferred for some reason, at minimum wrap the URLDecoder.decode() call in a try/catch for IllegalArgumentException so it does not crash on values with bare % characters.

3. Missing tests

Per project guidelines, bug-fix PRs must include tests. This PR needs at minimum:

  • A test showing URL-encoded kamelet parameter values (e.g., application%2Fjson) are properly decoded
  • A regression test ensuring RAW-wrapped values survive the decode step
  • A test for a value containing bare % (to verify it does not crash)

Claude Code on behalf of Guillaume Nodet

answer = value;
}
return answer;
return answer != null ? answer.trim() : null;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This .trim() applies to every property resolution across the entire Camel framework -- system properties, environment variables, .properties files, Spring Boot config, etc. Property values that legitimately contain trailing whitespace (passwords, PEM data, regex, template strings) would be silently corrupted.

The root cause is YAML literal block scalars (|) preserving trailing newlines. Please fix this at the YAML parsing layer where property values are extracted, not in the global properties parser.

Suggest removing this change entirely.

Claude Code on behalf of Guillaume Nodet

if (entry.getValue() instanceof String s
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(")
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) {
entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URLDecoder.decode() will throw IllegalArgumentException on values containing bare % characters (e.g., 100%complete or [a-z]%[0-9]). This createEndpoint() method is called from all DSLs, not just YAML, so non-encoded values will reach here.

Consider either:

  1. Fix at the source: use createQueryString(params, false) in YamlRoutesBuilderLoader for kamelet URIs (like KameletDeserializer and YamlSupport already do with encode=false). This would be consistent with the CAMEL-23284 approach.
  2. Or guard the decode: at minimum wrap in try/catch IllegalArgumentException and fall back to the original value.
Suggested change
entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8));
for (Map.Entry<String, Object> entry : parameters.entrySet()) {
if (entry.getValue() instanceof String s
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "(")
&& !s.startsWith(URISupport.RAW_TOKEN_PREFIX + "{")) {
try {
entry.setValue(URLDecoder.decode(s, StandardCharsets.UTF_8));
} catch (IllegalArgumentException e) {
// value contains bare % not part of URL encoding, leave as-is
}
}
}

Though option 1 (fixing the encoding at the YAML DSL source) would be the cleanest solution.

Claude Code on behalf of Guillaume Nodet

@cunningt
Copy link
Copy Markdown
Contributor Author

cunningt commented May 22, 2026

Thank you for the feedback! I'm working on changes to minimize the scope of where trim / decode run and target kamelets.

The issue here I see is that there are 7 transformation kamelet integration tests failing : https://github.com/apache/camel-kamelets/actions/runs/26155039895/job/76932071431

Error: CommonIT.transformation()[1] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[2] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[3] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[4] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[5] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[6] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'
Error: CommonIT.transformation()[7] » TestCaseFailed Action timeout after 60000 milliseconds. Failed to receive message on endpoint: 'httpServer.inbound'

The root cause of all of the failures there is that the content type is being URL-encoded and application/json → application%2Fjson, causing unexpected behavior and lots of test failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants