Skip to content

[SS-69] Make it obvious: Iceberg sink's AWS connection isn't used.#36693

Draft
ublubu wants to merge 1 commit into
kynan/new-iceberg-rustfrom
kynan/iceberg-unused-storage
Draft

[SS-69] Make it obvious: Iceberg sink's AWS connection isn't used.#36693
ublubu wants to merge 1 commit into
kynan/new-iceberg-rustfrom
kynan/iceberg-unused-storage

Conversation

@ublubu
Copy link
Copy Markdown
Contributor

@ublubu ublubu commented May 22, 2026

For Iceberg sinks to S3 Tables, we've been reusing the Iceberg Catalog connection's AWS creds.
Users separately specify AWS creds for the Iceberg sink itself, but we (validate and then) ignore them.

We could remove the AWS creds option from Iceberg sinks, but:

  • Users have already defined Iceberg sinks with superfluous AWS creds (because it's required).
  • We'll likely need creds on Iceberg sinks when we support Polaris + <insert storage service here>.

In this PR:

  • Add comments about the sink creds being ignored.
  • Make sink creds optional. Iceberg sinks to Google BigLake will just reuse the catalog connection's GCP creds.

stacked on #36692

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant