Skip to content

[SS-69] use updated iceberg-rust fork. new auth hooks for S3 Tables REST and S3 FileIO#36557

Closed
ublubu wants to merge 1 commit into
MaterializeInc:mainfrom
ublubu:new-iceberg-rust
Closed

[SS-69] use updated iceberg-rust fork. new auth hooks for S3 Tables REST and S3 FileIO#36557
ublubu wants to merge 1 commit into
MaterializeInc:mainfrom
ublubu:new-iceberg-rust

Conversation

@ublubu
Copy link
Copy Markdown
Contributor

@ublubu ublubu commented May 14, 2026

In preparation for sinking to GCP's Lakehouse, I've pulled in the stable release v0.9.0 for iceberg-rust and added a RequestAuthenticator trait for customizing auth support.

Auth options:

  • AWS SigV4 (now in this PR, previously in our iceberg-rust fork)
  • GCP (via gcp_auth in an upcoming PR)

This PR is still in Draft state because it depends on iceberg-rust changes that have not yet been reviewed. (e.g. MaterializeInc/iceberg-rust#1)

@ublubu ublubu changed the title [SS-69] DRAFT: use updated iceberg-rust fork. different hooks for S3 Tables REST and S3 FileIO [SS-69] DRAFT: use updated iceberg-rust fork. new hooks for S3 Tables REST and S3 FileIO May 14, 2026
@ublubu ublubu changed the title [SS-69] DRAFT: use updated iceberg-rust fork. new hooks for S3 Tables REST and S3 FileIO [SS-69] DRAFT: use updated iceberg-rust fork. new auth hooks for S3 Tables REST and S3 FileIO May 14, 2026
@ublubu ublubu force-pushed the new-iceberg-rust branch 4 times, most recently from 66f9270 to 72e0d8f Compare May 14, 2026 20:10
Comment on lines +207 to +213
if !new_query.is_empty() {
let url = req.url_mut();
let mut pairs = url.query_pairs_mut();
for (name, value) in new_query {
pairs.append_pair(name, &value);
}
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude wrote this section a little different from what @DAlperin had. I'm curious if there were problems with just appending the new SigV4 query params (instead of making sure to clobber any conflicting preexisting query params).

@ublubu ublubu changed the title [SS-69] DRAFT: use updated iceberg-rust fork. new auth hooks for S3 Tables REST and S3 FileIO [SS-69] use updated iceberg-rust fork. new auth hooks for S3 Tables REST and S3 FileIO May 22, 2026
@ublubu
Copy link
Copy Markdown
Contributor Author

ublubu commented May 22, 2026

closing in favor of #36692

@ublubu ublubu closed this May 22, 2026
@ublubu ublubu deleted the new-iceberg-rust branch May 22, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant