re-add XXH64 to read borg 1.x integrity data, #9935 - #9970
Merged
ThomasWaldmann merged 1 commit intoJul 30, 2026
Conversation
XXH64 (and the XXH64FileHashingWrapper) were removed in borgbackup#9672 / borgbackup#9750, which switched borg 2.x file integrity to SHA256. But borg 1.x wrote XXH64 checksums into the repo index/hints integrity data, so we need XXH64 to verify those files when reading a borg 1.x (legacy) repository during `borg transfer`. Rather than re-introducing the external "xxhash" PyPI package (and the libxxhash system dependency on msys2 that borgbackup#9750 dropped), add a small, dependency-free, streaming XXH64 implementation in cython to crypto.low_level. It is only used on the read path; borg 2.x native repos keep using SHA256. XXH64 is non-cryptographic and must not be used as a security mechanism. Tests use the official xxHash sanity-check vectors (the test buffer generator is transcribed verbatim from xxHash tests/sanity_test.c) and cover all code paths, streaming vs one-shot, and a legacy XXH64 integrity round-trip through IntegrityCheckedFile. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ThomasWaldmann
force-pushed
the
re-add-xxh64-9935
branch
from
July 29, 2026 21:36
368d9c7 to
15eb490
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9970 +/- ##
==========================================
+ Coverage 85.70% 85.73% +0.02%
==========================================
Files 95 95
Lines 16815 16856 +41
Branches 2577 2582 +5
==========================================
+ Hits 14411 14451 +40
- Misses 1668 1669 +1
Partials 736 736 ☔ View full report in Codecov by Harness. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Re-adds XXH64 support (removed in #9672 / #9750) so
borg transfercan verify the integrity data of a borg 1.x source repository.borg 1.x wrote XXH64 checksums into the repo
index.N/hints.Nintegrity data. When #9672 switched borg 2.x file integrity to SHA256 and #9750 dropped thexxhashdependency,XXH64disappeared fromSUPPORTED_ALGORITHMS, so reading a legacy repo's integrity data now hits the "Unknown algorithm" path and skips verification. Fixes #9935.How
Instead of re-introducing the external
xxhashPyPI package (and thelibxxhashmsys2 system dependency that #9750 deliberately removed), this adds a small, dependency-free, streaming XXH64 implementation in Cython toborg.crypto.low_level:cdef class XXH64(+ a one-shotxxh64()helper) with an interface compatible with the oldxxhash.xxh64FACTORY(XXH64()→.update()→.digest()/.hexdigest()), canonical big-endian digest.uint64_tC constants; byte-wise little-endian reads (correct on big-endian hosts too).XXH64FileHashingWrapperre-added and re-registered inSUPPORTED_ALGORITHMS.XXH64 is only used on the read path — borg 2.x native repos keep writing SHA256. It is non-cryptographic and must not be used as a security mechanism (noted in the code).
Since we implement locally, there is intentionally no change to
pyproject.tomlorscripts/msys2-install-deps— nothing to re-add.Tests
tests/sanity_test.c; expected digests cross-checked against the reference implementation (empty =ef46db3751d8e999, empty+prime-seed =ac75fda2929b17ef, …). Covers empty / 1-byte / sub-32 / single-stripe / multi-stripe / 4 KiB, seeds 0 and PRIME32, streaming vs one-shot across every chunk alignment, and buffer-protocol inputs.IntegrityCheckedFile(verify OK + corruption detected).🤖 Generated with Claude Code