Skip to content

Remove zstandard dependency in favor of numcodecs#1838

Merged
normanrz merged 8 commits into
v3from
v3-zstd
Aug 11, 2024
Merged

Remove zstandard dependency in favor of numcodecs#1838
normanrz merged 8 commits into
v3from
v3-zstd

Conversation

@normanrz
Copy link
Copy Markdown
Member

@normanrz normanrz commented May 5, 2024

I added zstandard as a dependency because the zstd in numcodecs doesn't expose the checksum config. This will be introduced in zarr-developers/numcodecs#519. Once that PR is merged, we can use the numcodecs version.

@normanrz normanrz added the V3 label May 5, 2024
@normanrz normanrz self-assigned this May 5, 2024
@normanrz normanrz changed the base branch from main to v3 May 5, 2024 20:14
@normanrz normanrz added this to the 3.0.0 milestone May 5, 2024
@normanrz normanrz changed the base branch from v3 to v3-fix-deps May 6, 2024 20:01
@mkitti
Copy link
Copy Markdown
Contributor

mkitti commented May 8, 2024

The correct default level should be 0 which will then allow the Zstd C library to use the default setting according to how it was built:
https://github.com/facebook/zstd/blob/97291fc5020a8994019ab76cf0cda83a9824374c/lib/zstd.h#L338

Copy link
Copy Markdown
Contributor

@mkitti mkitti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this level the default should stay 0 which the Zstandard C library will interpret as use the default that was built with as defined by zstd.h:
https://github.com/facebook/zstd/blob/97291fc5020a8994019ab76cf0cda83a9824374c/lib/zstd.h#L338

Comment thread src/zarr/codecs/zstd.py Outdated
@normanrz normanrz changed the base branch from v3-fix-deps to v3 July 22, 2024 14:55
@normanrz normanrz requested a review from d-v-b July 22, 2024 16:11
@normanrz
Copy link
Copy Markdown
Member Author

normanrz commented Jul 22, 2024

Now that numcodecs 0.13 is out, we can move forward with this.

@normanrz normanrz requested a review from jhamman July 26, 2024 15:13
@normanrz normanrz merged commit d9a142a into v3 Aug 11, 2024
@normanrz normanrz deleted the v3-zstd branch August 11, 2024 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants