Skip to content

Add append method to VaryingDimension for explicit chunk extension #3870

@maxrjones

Description

@maxrjones

Context:

From review discussion in #3802 (comment), @dcherian suggested adding an append(self, sizes: Iterable[int]) method to VaryingDimension to allow explicitly extending chunk edges.

Current behavior:

VaryingDimension.resize(new_extent) handles growth by appending a single chunk covering the gap:

# [10, 10, 10] resized from 30 to 45 produces [10, 10, 10, 15]

This works but gives the caller no control over how the new region is partitioned.

Proposed behavior:

# Explicit chunk extension
dim = VaryingDimension([10, 10, 10], extent=30)
dim = dim.append([10, 5])  # -> edges=(10, 10, 10, 10, 5), extent=45

# Would raise on FixedDimension (regular grids don't have explicit edges)

Open questions:

  • Should this live on VaryingDimension, ChunkGrid, or Array?
  • Should FixedDimension.append raise, or convert to VaryingDimension?
  • How does this interact with metadata serialization? Appending chunks to a regular grid would change the serialization format from "regular" to "rectilinear".
  • Should Array.resize() accept an optional chunks parameter that delegates to this? (Also noted in the design doc's resize section)

Related:

shall we also add append(self, sizes: Iterable[int]) to allow a different mode of extension (would fail for the FixedDimension).

Originally posted by @dcherian in #3802 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions