Add multi storage driver

[SuperSandro2000]

No description provided.

[Copilot]

Pull request overview

This PR introduces a new “multi” storage driver intended to support migration between two storage backends, and extends the storage driver interface to provide validation-time reads that can trigger on-demand migration.

Changes:

• Added ReadBlobForValidation / ReadManifestForValidation to keppel.StorageDriver and updated validation paths to use them.
• Introduced a new internal/drivers/multi storage driver that can read from the “new” backend first and fall back to “old”, copying on first validation.
• Centralized the blob chunk size constant as keppel.ChunkSizeBytes.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
internal/processor/manifests.go	Uses ReadManifestForValidation() during manifest validation to enable migration-on-validation.
internal/processor/blobs.go	Uses ReadBlobForValidation() during blob validation; switches chunking to keppel.ChunkSizeBytes.
internal/keppel/utils.go	Defines ChunkSizeBytes as a shared chunking constant.
internal/keppel/storage_driver.go	Extends the StorageDriver interface with validation-specific read methods.
internal/drivers/trivial/storage.go	Implements the new validation read methods by delegating to existing reads.
internal/drivers/openstack/swift.go	Implements the new validation read methods by delegating to existing reads.
internal/drivers/filesystem/storage.go	Implements the new validation read methods by delegating to existing reads.
internal/drivers/multi/storage.go	Adds the new multi storage driver with migration-on-validation logic.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[majewsky]

Please split the introduction of keppel.NotFoundInStorageError into a separate PR. Here are comments specifically on that part:

[SuperSandro2000]

The new error type and the feedback all went into #761

[majewsky]

These are my notes for the file internal/drivers/multi/storage.go. Overall, I'm very happy with the overall implementation quality; most of my notes are just papercuts and nitpicks. Once these are addressed and the PR is rebased on main, I will have a look at the remaining bits and pieces, but we should be good to go here soon.

[majewsky]

There should be documentation for this driver. You can leave this to me if you're not into technical writing; I'm just putting this here so we don't forget.

[github-actions]

Merging this branch will not change overall coverage

Impacted Packages	Coverage Δ	🤖
github.com/sapcc/keppel/internal/drivers/filesystem	0.00% (ø)
github.com/sapcc/keppel/internal/drivers/multi	0.00% (ø)
github.com/sapcc/keppel/internal/drivers/openstack	0.00% (ø)
github.com/sapcc/keppel/internal/drivers/trivial	0.00% (ø)
github.com/sapcc/keppel/internal/keppel	83.36% (ø)
github.com/sapcc/keppel/internal/processor	83.72% (ø)

---

Coverage by file

Changed files (no unit tests)

Changed File	Coverage Δ	Total	Covered	Missed	🤖
github.com/sapcc/keppel/internal/drivers/filesystem/storage.go	0.00% (ø)	0	0	0
github.com/sapcc/keppel/internal/drivers/multi/storage.go	0.00% (ø)	0	0	0
github.com/sapcc/keppel/internal/drivers/openstack/swift.go	0.00% (ø)	0	0	0
github.com/sapcc/keppel/internal/drivers/trivial/storage.go	0.00% (ø)	0	0	0
github.com/sapcc/keppel/internal/keppel/storage_driver.go	66.67% (ø)	132	88	44
github.com/sapcc/keppel/internal/keppel/utils.go	89.19% (ø)	407	363	44
github.com/sapcc/keppel/internal/processor/blobs.go	70.45% (ø)	792	558	234
github.com/sapcc/keppel/internal/processor/manifests.go	84.03% (ø)	2442	2052	390

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.