Support nested sourced_from fields end-to-end - #1252
Merged
ellisandrews-toast merged 11 commits intoJun 29, 2026
Conversation
myronmarston
changed the base branch from
main
to
myron/nested-sourced-from-prep
June 12, 2026 22:35
myronmarston
force-pushed
the
nested-sourced-from-script-and-acceptance
branch
from
June 12, 2026 22:35
f4549d9 to
4e853ea
Compare
myronmarston
force-pushed
the
nested-sourced-from-script-and-acceptance
branch
from
June 12, 2026 23:15
4e853ea to
b4b5815
Compare
myronmarston
left a comment
myronmarston
left a comment
Collaborator
There was a problem hiding this comment.
This is looking good @ellisandrews-toast! Left some feedback.
…cript-and-acceptance
ellisandrews-toast
marked this pull request as ready for review
June 18, 2026 19:57
ellisandrews-toast
requested review from
BrianSigafoos-SQ,
bsorbo,
jwils,
jwondrusch,
marcdaniels-toast and
rossroberts-toast
as code owners
June 18, 2026 19:57
This was referenced Jun 23, 2026
myronmarston
left a comment
myronmarston
left a comment
Collaborator
There was a problem hiding this comment.
Looking great! Left a few more suggestions.
myronmarston
approved these changes
Jun 29, 2026
myronmarston
left a comment
myronmarston
left a comment
Collaborator
There was a problem hiding this comment.
Forgot to approve with my review above. I view all remaining feedback as fine to defer to a followup.
Collaborator
Author
|
Will open a fresh PR to address remaining feedback after merging this one |
jwils
added a commit
that referenced
this pull request
Jul 1, 2026
Distilled from the last month of review comments (PRs #1224, #1231, #1247, #1251, #1252): Code: - Polymorphism over `is_a?`/`instance_of?` type checks. - Core gems stay ignorant of extensions. - Fix the root cause across a category, not just the reported instance. - Comment non-obvious "why"; link a tracking issue for known limitations. Tests: - Tests must not be satisfiable by a degenerate constant return. - Make tests self-contained about details that matter. - Drive behavior through the public API, not internal collaborators. - Preserve full coverage when relocating tests.
jwils
added a commit
that referenced
this pull request
Jul 1, 2026
Distilled from the last month of review comments (PRs #1224, #1231, #1247, #1251, #1252): Code: - Polymorphism over `is_a?`/`instance_of?` type checks. - Core gems stay ignorant of extensions. - Fix the root cause across a category, not just the reported instance. - Comment non-obvious "why"; link a tracking issue for known limitations. Tests: - Tests must not be satisfiable by a degenerate constant return. - Make tests self-contained about details that matter. - Drive behavior through the public API, not internal collaborators. - Preserve full coverage when relocating tests.
jwils
added a commit
that referenced
this pull request
Jul 2, 2026
Distilled from the last month of review comments (PRs #1224, #1231, #1247, #1251, #1252): Code: - Polymorphism over `is_a?`/`instance_of?` type checks. - Core gems stay ignorant of extensions. - Fix the root cause across a category, not just the reported instance. - Comment non-obvious "why"; link a tracking issue for known limitations. Tests: - Tests must not be satisfiable by a degenerate constant return. - Make tests self-contained about details that matter. - Drive behavior through the public API, not internal collaborators. - Preserve full coverage when relocating tests.
anthonycastiglia-toast
pushed a commit
to anthonycastiglia-toast/elasticgraph
that referenced
this pull request
Jul 10, 2026
…om fields. This is the parts of block#1252 which are ready to merge.
ellisandrews-toast
added a commit
that referenced
this pull request
Jul 15, 2026
Addresses the unresolved (non-blocking) review comments left on `index_data.painless` in #1252. Mechanical cleanups only — no behavior change; verified by the existing nested + top-level multi-source acceptance specs. - Use `segment.field` map-access shorthand in `buildNestedElementKeyParts` and `extractNestedElement` - More optimal null checking / returning in `extractNestedElement` - Enhance the `findInList` comment (first-match-wins) and reword the unbounded-growth note to reference #1270 Schema artifacts regenerated (`INDEX_DATA_UPDATE_SCRIPT_ID` updated accordingly).
This was referenced Jul 15, 2026
ellisandrews-toast
added a commit
that referenced
this pull request
Jul 20, 2026
…ling (#1310) This PR adds a unit test to cover a gap identified in this prior PR, but deferred: #1252 (comment)
ellisandrews-toast
added a commit
that referenced
this pull request
Jul 23, 2026
Addresses the review threads on #1252 that remained open after it merged, specifically pertaining to the acceptance tests. This fills the coverage gaps identified in review — mostly around the `__nested_sourced_data` buffer's edge cases — and reworks the spec for readability, so that every detail a test's assertions depend on is visible in the test body rather than hidden in helper defaults.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the nested
sourced_fromfeature by wiring up the indexing-time piece: theupdate_index_data.painlessscript now resolvessourced_fromfields onto nested elements of an indexed document, not just top-level fields. This is the final PR in the nestedsourced_fromseries (building on the runtime-metadata, relationship-chain, and update-target resolution PRs already merged to main).What's included
index_data.painless) — nested-apply logic: buffers each related event's sourced fields in__nested_sourced_datakeyed by a per-element key, then re-applies them onto the target nested element on every event (so a later self-event that overwrites the nested array doesn't drop sourced fields). Handles out-of-order ingestion, list and object path segments, and multi-byte element ids via a length-prefixed key encoding.__versionsand__nested_sourced_data; top-level events keep a bare relationship key so existing documents stay byte-identical and version-recovery is unchanged.update.rbpasses the registered nested-path config (sourcedFromNestedPaths,sourcedFromNestedPathIdentifiers,sourcedFromNestedFields) through to the script.__nested_sourced_datamapping — added to every index (internal_fields), mirroring__versions.config/schema/teams.rbgains aTeam → staff.coaches[]/general_manager → *Profileshape exercising nestedsourced_fromover both list and singleton-object path segments, with distinct source types.nested_multi_source_indexing_spec.rbacceptance coverage (out-of-order, root re-index, stale event, relationship-mutation rejection, multi-byte ids, all-object + list paths); factories for the new types; unit coverage for the threaded params and the relationship-name guarantee the codec relies on.Deferred
sourced_fromon list/__counts fields (not yet supported) — separate PR.__nested_sourced_dataentries and detection of nested-element re-targeting — tracked as known limitations.