Skip to content

Validate out-of-order tables incrementally while parsing#509

Merged
frostming merged 2 commits into
python-poetry:masterfrom
gaoflow:fix-479-quadratic-validation
Jun 15, 2026
Merged

Validate out-of-order tables incrementally while parsing#509
frostming merged 2 commits into
python-poetry:masterfrom
gaoflow:fix-479-quadratic-validation

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

Fixes #479.

Parsing n dotted keys (or table headers) that share a prefix is super-cubic: every append to an out-of-order key triggers OutOfOrderTableProxy.validate, which re-merges — and deep-copies — every earlier fragment into a fresh temp container, and the membership probe in append() rebuilds the full proxy each time as well. Profiling the issue's payload shows ~13M deepcopy calls for 60 keys.

Three changes, all confined to parse time (_parsed containers):

  • _validate_out_of_order_table keeps a per-key cache of (fragments validated so far, temp container) and resumes from there, so each fragment is merged and deep-copied exactly once. Fragments are only appended during parsing; any other mutation (remove, _remove_at, _replace_at) clears the cache, and a validation failure drops the entry so a caught-and-retried error can't resume from a partially merged container.
  • __contains__ skips rebuilding the proxy when the cache shows exactly the current fragments already validated clean.
  • The no-key validation sweep iterates a maintained set of out-of-order keys instead of every key in the map (the per-key path still re-checks isinstance(idx, tuple), so stale entries are harmless).

Benchmark (the issue's repro):

keys before after
40 1.3 s 5.5 ms
80 15.6 s 10 ms
160 219 s 19 ms
1000 0.15 s

Error behavior is unchanged: collisions across fragments still raise at parse time (added tests), and the existing suite passes unmodified.

@gaoflow
gaoflow force-pushed the fix-479-quadratic-validation branch from 191e8d2 to eb798da Compare June 12, 2026 09:33
@gaoflow
gaoflow force-pushed the fix-479-quadratic-validation branch from eb798da to a0dc05d Compare June 15, 2026 03:08
@frostming
frostming merged commit ae1b679 into python-poetry:master Jun 15, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Security] Quadratic complexity DoS when parsing dotted keys with shared prefix (CWE-400)

2 participants