Skip to content

✨ upsteam parity#53

Open
techouse wants to merge 14 commits into
mainfrom
feat/upstream-fixes-v2
Open

✨ upsteam parity#53
techouse wants to merge 14 commits into
mainfrom
feat/upstream-fixes-v2

Conversation

@techouse
Copy link
Copy Markdown
Owner

@techouse techouse commented May 23, 2026

This pull request introduces several improvements and bug fixes to the query string decoder, aligning its behavior more closely with Node.js qs v6.15, especially around list and object merging, duplicate handling, and array limits. The most significant changes are the addition of a new strict_merge option, improved handling of list limits, and enhanced duplicate key semantics. Documentation and tests have also been updated to reflect these changes.

Decoder behavior and options:

  • Added a new DecodeOptions.strict_merge option (default True) to control how object/scalar conflicts are merged: when enabled, conflicting values are wrapped in a list (Node.js qs 6.15+ parity); when disabled, legacy behavior is restored, adding non-empty string scalars as object keys. [1] [2] [3] [4]
  • Improved the handling of duplicate bracket-array assignments: bracket-array keys always combine values into arrays, regardless of the global duplicate strategy. [1] [2]
  • Updated the semantics of the list_limit option to act as a maximum element count (matching Node.js qs arrayLimit), not a maximum index. Now, only indices below the limit are included in lists; higher indices are converted to dicts. [1] F167923eL310R342, [2] [3] [4]
  • Improved error handling for exceeding list limits, including clearer error messages and new CommaOverflowDict to mark over-limit comma-split values. [1] [2] [3] [4]

Documentation and tests:

  • Updated documentation in README.rst to describe the new strict_merge option, bracket-array duplicate behavior, and revised list limit semantics. [1] [2]
  • Added and revised test cases to cover new and legacy behaviors, including strict/legacy merge, bracket-array handling, and list limit edge cases. [1] [2] [3] [4] [5]

Changelog:

  • Updated CHANGELOG.md to document the new features and fixes.

Summary by CodeRabbit

  • New Features

    • Added strict_merge option to control object-vs-scalar merge behavior during decoding.
  • Bug Fixes

    • list_limit now treated as a maximum element count with adjusted boundary behavior.
    • Bracket-array entries always combine; comma-split lists that exceed limits convert to an overflow object.
    • Align encode/decode delimiter handling and normalize dotted-key behavior.
  • Documentation

    • Clarified arrays, list limits, comma-splitting, and merge-conflict semantics.
  • Tests

    • Expanded tests for strict_merge, list limits, duplicates, and overflow cases.

Review Change Stack

@techouse techouse requested a review from Copilot May 23, 2026 08:18
@techouse techouse self-assigned this May 23, 2026
@techouse techouse added the enhancement New feature or request label May 23, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 23, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3c272fa1-c4e5-426a-aeb4-65d3482d00d3

📥 Commits

Reviewing files that changed from the base of the PR and between c6c6e44 and 47ca334.

📒 Files selected for processing (2)
  • src/qs_codec/utils/utils.py
  • tests/unit/utils_test.py

📝 Walkthrough

Walkthrough

PR adds DecodeOptions.strict_merge, redefines list_limit as a maximum element count, forces bracket-array keys to combine, converts over-limit comma-splits to CommaOverflowDict or raises, preserves OverflowDict subclasses on copy, and updates tests/docs accordingly.

Changes

Query String Decoding: Strict Merge, List Limits, and Bracket-Array Semantics

Layer / File(s) Summary
Configuration: DecodeOptions, OverflowDict, and Docs
src/qs_codec/models/decode_options.py, src/qs_codec/models/overflow_dict.py, CHANGELOG.md, docs/README.rst
Adds DecodeOptions.strict_merge (default True), rewrites list_limit docs to define it as max element count (boundary at index 19/20), and modifies OverflowDict copy/deepcopy to preserve subclasses. Docs and changelog updated to reflect these behaviors.
Core Decode: Comma Limits, Bracket Arrays, and Index Boundaries
src/qs_codec/decode.py
Refactors _parse_array_value to separate comma-limit enforcement (enforce_comma_limit), decodes comma-splittable values without immediate enforcement then converts or raises on overflow, detects []= bracket-array assignments to wrap values and force Duplicates.COMBINE, and changes numeric-index list allocation to index < list_limit (treat index >= list_limit as dict key or raise).
Utils: Merge and Combine with Strict Merge Support
src/qs_codec/utils/utils.py
Utils.merge now respects strict_merge when merging a scalar into a mapping (wrap into [mapping, scalar] by default; legacy boolean-key/object behavior when disabled). Utils.combine preserves OverflowDict subclass, computes next numeric index, appends b as a single value, and treats CommaOverflowDict as a single-element source.
Tests: DecodeOptions Defaults and Construction
tests/unit/decode_options_test.py
Asserts DecodeOptions().strict_merge is True and verifies strict_merge persists across constructor variants (including different Duplicates settings).
Tests: List Limits, Comma Overflow, and Strict Merge
tests/unit/decode_test.py
Extensive parametrized tests for list_limit boundary behavior, list_limit=0 conversions, comma-splitting overflow conversion to OverflowDict (including negative limits), bracket-notation always-combine regardless of Duplicates, strict_merge object-vs-scalar scenarios, and mixed structured-flat parity updates.
Tests: Utils.merge/combine and Examples
tests/unit/utils_test.py, tests/unit/example_test.py
Updates expectations for Utils.merge strict-merge wrapping and legacy mode; Utils.combine tests now assert single nested append semantics and subclass preservation; example tests reflect duplicates/strict_merge behaviors and element-count wording updates.
Test Fixtures: Comparison Cases
tests/comparison/test_cases.json
Adds two new encoding test cases: mixed array/object with string element, and object keyed by numeric-string "20" producing a[20]=f.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested labels

test

Poem

🐇 A query string hops through the fields,
Strict merges cuddle conflicting yields,
When lists reach twenty they tumble aside,
Brackets unite — overflow nests hide,
Docs and tests cheer the codec's new stride.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title ':sparkles: upsteam parity' is vague and contains a typo ('upsteam' instead of 'upstream'). It provides no specific information about the changes. Replace with a specific, clear title that describes the main changes, such as 'Add strict_merge option and align list limit semantics with Node.js qs 6.15'.
✅ Passed checks (3 passed)
Check name Status Explanation
Description check ✅ Passed The PR description comprehensively covers objectives, changes, and testing considerations, but does not follow the required template format with sections like 'Type of change' and 'Checklist'.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/upstream-fixes-v2

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the decoder to better match Node.js qs v6.15+ behavior, mainly around merge semantics, duplicate handling for bracket-arrays, and list_limit enforcement/overflow behavior.

Changes:

  • Add DecodeOptions.strict_merge (default True) to wrap object/scalar merge conflicts in a list, with a legacy mode when disabled.
  • Adjust decoding semantics for bracket-array duplicates (always combine) and refine list_limit behavior + overflow handling (including comma-split overflow marking).
  • Update docs, comparison fixtures, and unit tests to cover the new parity behaviors.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/qs_codec/decode.py Implements new list-limit message helper, bracket-array duplicate behavior, comma-overflow handling, and updated list/index limit logic.
src/qs_codec/utils/utils.py Updates merge semantics for strict merge conflicts and refines overflow combination behavior (including comma overflow nesting).
src/qs_codec/models/decode_options.py Adds strict_merge option (default True).
src/qs_codec/models/overflow_dict.py Preserves subclass type across copy/deepcopy and introduces CommaOverflowDict.
tests/unit/decode_test.py Expands coverage for strict merge behavior, bracket-array duplicate combining, revised list-limit semantics, and comma overflow cases.
tests/unit/utils_test.py Updates merge tests to reflect strict merge default behavior and legacy mode behavior.
tests/unit/example_test.py Updates examples for bracket-array duplicates, strict merge behavior, and revised list-limit semantics wording.
tests/unit/decode_options_test.py Asserts DecodeOptions.strict_merge default value.
tests/comparison/test_cases.json Adds/adjusts parity fixtures for strict merge and list-limit semantics.
docs/README.rst Documents strict merge, bracket-array duplicate semantics, and revised list-limit semantics.
CHANGELOG.md Notes the new option and behavior fixes.

Comment thread src/qs_codec/decode.py
Comment thread src/qs_codec/decode.py
@codecov
Copy link
Copy Markdown

codecov Bot commented May 23, 2026

Codecov Report

❌ Patch coverage is 93.54839% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.68%. Comparing base (96140c2) to head (47ca334).

Files with missing lines Patch % Lines
src/qs_codec/decode.py 89.65% 3 Missing ⚠️
src/qs_codec/utils/utils.py 96.15% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##              main      #53      +/-   ##
===========================================
- Coverage   100.00%   99.68%   -0.32%     
===========================================
  Files           22       22              
  Lines         1904     1932      +28     
===========================================
+ Hits          1904     1926      +22     
- Misses           0        6       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/unit/utils_test.py (1)

604-618: ⚡ Quick win

Assert copy-on-write in the new scalar-conflict merge tests.

Line 608 and Line 616 validate result shape, but they don’t verify that target stays unmodified (and unaliased) across strict/legacy paths. Add non-mutation assertions so these tests enforce the repo contract directly.

Suggested test hardening
     def test_merge_mapping_target_with_scalar_source_wraps_with_strict_merge(self) -> None:
         target = {"a": "b"}
         source = "scalar"

         result = Utils.merge(target, source)  # type: ignore[arg-type]

         assert result == [{"a": "b"}, "scalar"]
+        assert target == {"a": "b"}
+        assert result[0] is not target

     def test_merge_mapping_target_with_scalar_source_uses_legacy_strict_merge_false(self) -> None:
         target = {"a": "b"}
         source = "scalar"

         result = Utils.merge(target, source, DecodeOptions(strict_merge=False))  # type: ignore[arg-type]

         assert result == {"a": "b", "scalar": True}
+        assert target == {"a": "b"}
+        assert result is not target

As per coding guidelines: **/*.py: Do not mutate caller inputs—copy/normalize before traversal.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/utils_test.py` around lines 604 - 618, Update the two tests
test_merge_mapping_target_with_scalar_source_wraps_with_strict_merge and
test_merge_mapping_target_with_scalar_source_uses_legacy_strict_merge_false to
assert copy-on-write behavior: after calling Utils.merge(target, source, ...)
verify the original target dict remains unchanged (e.g., equals {"a": "b"}) and
is not aliased into the result (ensure result does not reference target object).
Use the same symbols (target, result, Utils.merge, DecodeOptions) to locate
where to add non-mutation assertions.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unit/utils_test.py`:
- Around line 604-618: Update the two tests
test_merge_mapping_target_with_scalar_source_wraps_with_strict_merge and
test_merge_mapping_target_with_scalar_source_uses_legacy_strict_merge_false to
assert copy-on-write behavior: after calling Utils.merge(target, source, ...)
verify the original target dict remains unchanged (e.g., equals {"a": "b"}) and
is not aliased into the result (ensure result does not reference target object).
Use the same symbols (target, result, Utils.merge, DecodeOptions) to locate
where to add non-mutation assertions.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3f8603af-a717-4309-85eb-9ed498d78546

📥 Commits

Reviewing files that changed from the base of the PR and between 96140c2 and f8659ff.

📒 Files selected for processing (11)
  • CHANGELOG.md
  • docs/README.rst
  • src/qs_codec/decode.py
  • src/qs_codec/models/decode_options.py
  • src/qs_codec/models/overflow_dict.py
  • src/qs_codec/utils/utils.py
  • tests/comparison/test_cases.json
  • tests/unit/decode_options_test.py
  • tests/unit/decode_test.py
  • tests/unit/example_test.py
  • tests/unit/utils_test.py

@codacy-production
Copy link
Copy Markdown

codacy-production Bot commented May 23, 2026

Not up to standards ⛔

🔴 Issues 1 medium

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
Complexity 1 medium

View in Codacy

🟢 Metrics 8 complexity · 0 duplication

Metric Results
Complexity 8
Duplication 0

View in Codacy

🟢 Coverage 93.55% diff coverage · -0.31% coverage variation

Metric Results
Coverage variation -0.31% coverage variation (-1.00%)
Diff coverage 93.55% diff coverage

View coverage diff in Codacy

Coverage variation details
Coverable lines Covered lines Coverage
Common ancestor commit (96140c2) 1904 1904 100.00%
Head commit (47ca334) 1932 (+28) 1926 (+22) 99.69% (-0.31%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details
Coverable lines Covered lines Diff coverage
Pull request (#53) 62 58 93.55%

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Comment thread src/qs_codec/decode.py
Comment thread src/qs_codec/models/decode_options.py Outdated
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/qs_codec/utils/utils.py (1)

539-549: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Copy b before inserting it into the overflow result.

This branch preserves a, but it still stores the original b object inside the returned OverflowDict. Any later in-place cleanup of the merged structure can mutate the caller's sequence/mapping through that alias.

Suggested normalization
             if not isinstance(b, Undefined):
-                a_copy[str(idx)] = b
+                if isinstance(b, list):
+                    a_copy[str(idx)] = list(b)
+                elif isinstance(b, tuple):
+                    a_copy[str(idx)] = list(b)
+                elif isinstance(b, ABCMapping):
+                    a_copy[str(idx)] = dict(b)
+                else:
+                    a_copy[str(idx)] = b
             return a_copy

As per coding guidelines "Do not mutate caller inputs—copy/normalize before traversal (shallow copy for mappings; deep-copy only when a callable filter may mutate; index-projection for sequences)".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/qs_codec/utils/utils.py` around lines 539 - 549, The branch that appends
b into the new OverflowDict (in the Utils.is_overflow branch) must insert a copy
of b instead of the original to avoid aliasing caller data; before the line
a_copy[str(idx)] = b, create a copied version (e.g., shallow-copy
mappings/sequences or deepcopy when callers may mutate via callables) and assign
a_copy[str(idx)] = b_copy; reference: Utils.is_overflow, OverflowDict,
Undefined, _numeric_key_pairs, variables a, b, a_copy.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/qs_codec/decode.py`:
- Around line 242-245: The check enforcing options.list_limit inside
_parse_array_value is being applied to plain key=value assignments and causes
DecodeOptions(list_limit=-1, raise_on_limit_exceeded=True) to raise for non-list
tokens; change the guard so the current_list_length >= options.list_limit check
only runs when we are actually building a list (i.e., when the call path
indicates comma/bracket array parsing or when a list container is present), not
for simple scalar assignments in _parse_array_value; update the condition that
references current_list_length and options.list_limit (and any early-return
paths) to first verify we are in list-building mode (for example by checking a
list context flag or the presence of an existing list value) before enforcing
the limit so decode("a=b", ...) no longer triggers negative-limit logic.

---

Outside diff comments:
In `@src/qs_codec/utils/utils.py`:
- Around line 539-549: The branch that appends b into the new OverflowDict (in
the Utils.is_overflow branch) must insert a copy of b instead of the original to
avoid aliasing caller data; before the line a_copy[str(idx)] = b, create a
copied version (e.g., shallow-copy mappings/sequences or deepcopy when callers
may mutate via callables) and assign a_copy[str(idx)] = b_copy; reference:
Utils.is_overflow, OverflowDict, Undefined, _numeric_key_pairs, variables a, b,
a_copy.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7a9241a2-6c3b-41dd-8ccd-0e644fe5712b

📥 Commits

Reviewing files that changed from the base of the PR and between f8659ff and c6c6e44.

📒 Files selected for processing (5)
  • src/qs_codec/decode.py
  • src/qs_codec/models/decode_options.py
  • src/qs_codec/utils/utils.py
  • tests/unit/decode_test.py
  • tests/unit/utils_test.py

Comment thread src/qs_codec/decode.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants