Skip to content

Add HTTP response processing mode with lazy decompression#6708

Open
vlsi wants to merge 2 commits into
apache:masterfrom
vlsi:http-response-processing-mode
Open

Add HTTP response processing mode with lazy decompression#6708
vlsi wants to merge 2 commits into
apache:masterfrom
vlsi:http-response-processing-mode

Conversation

@vlsi
Copy link
Copy Markdown
Collaborator

@vlsi vlsi commented Jun 1, 2026

What

Replaces the boolean "Save response as MD5 hash?" option on HTTP Request and HTTP Request Defaults with a Processing mode combo that controls how the response body is handled:

  • Default — leave the property unset: inherit from HTTP Request Defaults if set, otherwise behave like Store response.
  • Store response (decompress on access) — store the raw, possibly compressed, body and decompress it lazily, only when it is read (assertion, post-processor, listener). The new default; responses that are never read are never decompressed, saving CPU and memory.
  • Fetch and discard (headers only) — read the response but do not store the body.
  • Checksum (MD5 of compressed) / Checksum (MD5 of decompressed) — store a 32-character MD5 hash instead of the body. The decompressed variant matches the former "Save as MD5".

It also adds a ResponseDecoder SPI with gzip, deflate, and brotli decoders (so decompression runs lazily and is pluggable) and a tooltip explaining the field.

Builds on #6389 by @jgaalen (discussion in #6388); the first commit keeps him as author.

Why the migration is resolve-on-read

The legacy behaviour is driven by the boolean HTTPSampler.md5, which can live in two places: on the sampler (HTTPSamplerBase) and on HTTP Request Defaults (a ConfigTestElement). Old plans must keep working, and an explicit md5=false on a sampler must still override md5=true inherited from the defaults.

A migration that rewrites md5 → responseProcessingMode in setProperty cannot close both cases:

  • ConfigTestElement lives in core, which cannot reference the HTTP enum/schema (the dependency is http → core), so the defaults element can't be migrated there.
  • A setProperty migration only sees an element's own properties; the md5 inherited from the defaults reaches the sampler through config merge (mergeIn / addProperty), which bypasses setProperty.

So correctness is resolved on read: getResponseProcessingMode() returns the explicit responseProcessingMode when set, otherwise falls back to the legacy md5 — the sampler's own, or the one inherited from the defaults via merge — otherwise the schema default. This reuses the existing boolean merge semantics (an explicit md5=false overrides an inherited md5=true), so both the sampler and the HTTP Request Defaults cases behave exactly as before, for any origin: loaded JMX, plans built programmatically, or direct property access. The GUI additionally rewrites the property to responseProcessingMode and drops md5 when the element is saved.

TestElementUpgrader SPI — will be split into a separate PR

The second commit adds a small core SPI — TestElementUpgrader, discovered via ServiceLoader and run by SaveService right after a plan is loaded — that upgrades legacy property values on load (NameUpdater only renames keys/classes). The HTTP-side ResponseProcessingModeUpgrader rewrites md5 → responseProcessingMode; because it keys off the property name, it upgrades both samplers and the ConfigTestElement defaults, without core depending on http.

It is not required for correctness (resolve-on-read already covers runtime); it just normalises old plans into the new property on load and re-save. I will extract it into its own PR — it is included here only to show the full picture and a concrete usage example. Happy to drop that commit from this PR if you prefer.

How to verify

  • ./gradlew :src:core:testClasses :src:protocol:http:testClasses
  • New tests: ResponseProcessingModeInheritanceTest (resolve-on-read, inheritance and override), HttpTestSampleGuiResponseProcessingModeTest / HttpDefaultsGuiResponseProcessingModeTest (GUI round-trip), ResponseProcessingModeUpgraderTest / ResponseProcessingModeUpgradeOnLoadTest (the SPI), plus the ported ResponseDecoderRegistryTest and gzip/deflate/brotli decoder tests.
  • In the GUI: HTTP Request → Advanced → Response Processing. Load a plan that used "Save response as MD5 hash?" and confirm the combo shows Checksum (MD5 of decompressed).

🤖 Generated with Claude Code

@vlsi vlsi force-pushed the http-response-processing-mode branch from 3117c93 to b8789af Compare June 1, 2026 08:41
jvangaalen and others added 2 commits June 1, 2026 12:04
Store the raw, possibly compressed, response body in SampleResult and
decompress it on demand in getResponseData() rather than eagerly while the
response is read. The original bytes stay available for checksums, and
responses that are never read are never decompressed.

Add a ServiceLoader-based ResponseDecoder SPI with a registry and built-in
gzip, deflate, and brotli decoders, so protocols can register further
content encodings.

Replace the boolean "Store as MD5" option with a responseProcessingMode
that controls how the body is handled: store it compressed (the new
default), fetch and discard it, or store an MD5 checksum of the compressed
or the decompressed stream. HTTP implementations pass the raw body and its
Content-Encoding to SampleResult, so decompression runs lazily through the
decoder registry.

The GUI shows the mode as a plain combo box with a leading "Default" entry;
selecting Default leaves the property unset so it can be inherited from HTTP
Request Defaults. Old test plans keep working: when responseProcessingMode
is unset, getResponseProcessingMode() falls back to the legacy
HTTPSampler.md5 property, whether set on the sampler or inherited from HTTP
Request Defaults. This reuses the boolean merge semantics, so an explicit
md5=false on a sampler still overrides md5=true on the defaults. The GUI
reflects the resolved mode and rewrites it to responseProcessingMode on save.

Based on the work in apache#6389.

Co-authored-by: Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SaveService can rename keys and classes on load via NameUpdater, but it cannot
transform property *values*. Add a TestElementUpgrader service interface,
discovered with ServiceLoader, that upgrades legacy properties on a loaded test
plan. SaveService runs all registered upgraders to a fix point after reading a
plan, so chained upgrades converge without numbering the migrations.

Add the first upgrader: an HTTP-side ResponseProcessingModeUpgrader maps the
legacy HTTPSampler.md5 flag to responseProcessingMode. It keys off the property
name, so it upgrades both HTTP samplers and HTTP Request Defaults (a
ConfigTestElement), which core cannot reference directly.

This normalises old plans into the new property on load and on re-save. Runtime
correctness for plans built another way still relies on
getResponseProcessingMode() falling back to the legacy flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vlsi vlsi force-pushed the http-response-processing-mode branch from b8789af to 53e8bb5 Compare June 1, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant