add LeafSegmentsBundle to give callers more flexibility for processing segments#18710
Merged
Merged
Conversation
…g segments and streamline missing segment handling
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| */ | ||
| boolean canHandleSegments(); | ||
|
|
||
| default boolean canLoadSegmentOnDemand(DataSegment segment) |
Check notice
Code scanning / CodeQL
Useless parameter Note
gianm
approved these changes
Nov 3, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After #18683, in virtual storage mode there are no longer 'missing' segments after the initial timeline operation to obtain the
DataSegmentobjects for the given set ofSegmentDescriptor, so we can simplify things in such as way as to both give callers additional flexibility on how they handle loading on demand (not really utilized in this PR).To model this, a new
LeafSegementsBundlehas been added that holds the result of partitioning a list ofDataSegmentAndDescriptorsinto:SegmentReferenceto place a hold on them)DataSegmentin the timeline in virtual storage mode, or the segment has been dropped beforegetCachedSegmentcould be called in regular mode).Two new methods has been added to
SegmentManager,getSegmentsBundleto build aLeafSegmentsBundle, andcanLoadSegmentsOnDemandto indicate that aSegmentCacheManagercan support loading a segment on demand if it was not present in the cache already.ServerManagerhas been transitioned to use these methods, even though it still downloads all 'load on demand' segments up front before building query runners.This change also lets us streamline the handling of 'missing' segments in
ServerManager. (e.g. those which are dropped from a historical after the broker has already picked them and issued a query and are added to the header as a 'missing' segment for the broker to retry). Instead of handling missing segments 'inline' and using aReportTimelineMissingSegmentQueryRunner, because we are usingLeafSegmentsBundlewe can just split the handling of these segments out as soon as we make the bundle and directly add them to the missing segments of the response context.ReportTimelineMissingSegmentQueryRunnerhas been removed, since the only remaining uses