Core: Wire REST catalog table encryption#16672
Closed
hkwi wants to merge 1 commit into
Closed
Conversation
6cb5647 to
b14bc50
Compare
Author
|
I found #13225, which is already covering this area, so I'll wait for that PR instead. Closing this for now. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR wires Iceberg table encryption into REST catalog table operations.
The REST catalog now creates a
KeyManagementClientfromencryption.kms-typeorencryption.kms-impl, passes it toRESTTableOperations, and returns an encryption-awareEncryptionManager/EncryptingFileIOfor encrypted tables. It also adds generated encryptionkeys to REST commit requests as
AddEncryptionKeymetadata updates.Background
TableOperations.encryption()defaults to a plaintext encryption manager. REST table operationscurrently do not override that default, so tables configured with
encryption.key-idcan still usethe plaintext manager on the REST client write path.
Hive catalog already wires a KMS client into table operations and adds generated encryption keys to
table metadata before commit. This PR applies the same client-side encryption wiring to REST catalog
table operations.
Changes
KeyManagementClientinRESTSessionCatalogwhen KMS catalog properties are configuredEncryptionManagerandEncryptingFileIOfor encrypted REST tablesAddEncryptionKeymetadata updates to create, replace, and simple REST commitsencryption.key-idon existing encrypted tablesFileIOtracking wrapper so table loading does not initialize KMS stateTesting
./gradlew :iceberg-core:compileJava :iceberg-core:compileTestJava :iceberg-core:test --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedCreateTransactionAddsEncryptionKeys --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedCreateTransactionRequiresKmsClient --tests org.apache.iceberg.rest.TestRESTCatalog.testEncryptedTableLoadDoesNotRequireKmsClient --no-daemon./gradlew :iceberg-core:test --tests org.apache.iceberg.io.TestFileIOTracker --tests org.apache.iceberg.rest.TestRESTCatalog.testCustomTableOperationsInjection --tests org.apache.iceberg.rest.TestFreshnessAwareLoading.customTableOperationsWithFreshnessAwareLoading --no-daemon./gradlew :iceberg-core:spotlessCheck --no-daemongit diff --check