Skip to content

Core: Wire REST catalog table encryption#16672

Closed
hkwi wants to merge 1 commit into
apache:mainfrom
hkwi:rest-encryption-wiring
Closed

Core: Wire REST catalog table encryption#16672
hkwi wants to merge 1 commit into
apache:mainfrom
hkwi:rest-encryption-wiring

Conversation

@hkwi

@hkwi hkwi commented Jun 3, 2026

Copy link
Copy Markdown

Summary

This PR wires Iceberg table encryption into REST catalog table operations.

The REST catalog now creates a KeyManagementClient from encryption.kms-type or
encryption.kms-impl, passes it to RESTTableOperations, and returns an encryption-aware
EncryptionManager / EncryptingFileIO for encrypted tables. It also adds generated encryption
keys to REST commit requests as AddEncryptionKey metadata updates.

Background

TableOperations.encryption() defaults to a plaintext encryption manager. REST table operations
currently do not override that default, so tables configured with encryption.key-id can still use
the plaintext manager on the REST client write path.

Hive catalog already wires a KMS client into table operations and adds generated encryption keys to
table metadata before commit. This PR applies the same client-side encryption wiring to REST catalog
table operations.

Changes

  • Load a KeyManagementClient in RESTSessionCatalog when KMS catalog properties are configured
  • Pass the KMS client to REST table operations
  • Return an encryption-aware EncryptionManager and EncryptingFileIO for encrypted REST tables
  • Add generated AddEncryptionKey metadata updates to create, replace, and simple REST commits
  • Reject removal or modification of encryption.key-id on existing encrypted tables
  • Use a REST-local raw FileIO tracking wrapper so table loading does not initialize KMS state
  • Add REST catalog tests for encrypted create transactions, missing KMS configuration, and encrypted table load behavior

Testing

  • ./gradlew :iceberg-core:compileJava :iceberg-core:compileTestJava :iceberg-core:test --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedCreateTransactionAddsEncryptionKeys --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedCreateTransactionRequiresKmsClient --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedTableLoadDoesNotRequireKmsClient --no-daemon
  • ./gradlew :iceberg-core:test --tests org.apache.iceberg.io.TestFileIOTracker --tests org.apache.iceberg.rest.TestRESTCatalog.testCustomTableOperationsInjection --tests org.apache.iceberg.rest.TestFreshnessAwareLoading.customTableOperationsWithFreshnessAwareLoading --no-daemon
  • ./gradlew :iceberg-core:spotlessCheck --no-daemon
  • git diff --check

@github-actions github-actions Bot added the core label Jun 3, 2026
@hkwi hkwi changed the title Core: REST catalog の table encryption を配線する Core: Wire REST catalog table encryption Jun 3, 2026
@hkwi hkwi force-pushed the rest-encryption-wiring branch from 6cb5647 to b14bc50 Compare June 3, 2026 11:05
@hkwi

hkwi commented Jun 3, 2026

Copy link
Copy Markdown
Author

I found #13225, which is already covering this area, so I'll wait for that PR instead. Closing this for now.

@hkwi hkwi closed this Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant