HDDS-14935. [STS] Handle Latent Inconsistencies in S3 API Acl Checks#10009
Open
fmorg-git wants to merge 1 commit into
Open
HDDS-14935. [STS] Handle Latent Inconsistencies in S3 API Acl Checks#10009fmorg-git wants to merge 1 commit into
fmorg-git wants to merge 1 commit into
Conversation
|
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. |
Contributor
Author
|
adding comment to remove stale label |
Contributor
Author
|
hi @ChenSammi @sodonnel - just a quick reminder that this PR is ready to review. Thanks! |
Comment on lines
+121
to
+129
| final OMRequest.Builder requestBuilder = getOmRequest().toBuilder() | ||
| .setUserInfo(getUserIfNotExists(ozoneManager)) | ||
| .setLayoutVersion(layoutVersion).build(); | ||
| .setLayoutVersion(layoutVersion); | ||
|
|
||
| if (requestBuilder.hasS3Authentication()) { | ||
| requestBuilder.setS3Authentication( | ||
| resolveS3Authentication(requestBuilder.getS3Authentication(), OzoneManager.getStsTokenIdentifier())); | ||
| } | ||
|
|
Comment on lines
+669
to
+688
| if (ozoneManager.isSecurityEnabled() && request.hasS3Authentication()) { | ||
| // STS token verification runs on the leader RPC path so we don't need to recheck here on the apply | ||
| // after the log is committed | ||
| STSSecurityUtil.ensureResolvedStsFieldsInvariants(request); | ||
|
|
||
| final OzoneManagerProtocolProtos.S3Authentication s3Auth = request.getS3Authentication(); | ||
| if (s3Auth.hasSessionToken() && !s3Auth.getSessionToken().isEmpty()) { | ||
| // ThreadLocal carries session policy for OmMetadataReader | ||
| final STSTokenIdentifier rehydratedTokenIdentifier = new STSTokenIdentifier( | ||
| s3Auth.hasResolvedStsTempAccessKeyId() ? s3Auth.getResolvedStsTempAccessKeyId() : "", | ||
| s3Auth.hasResolvedStsOriginalAccessKeyId() ? s3Auth.getResolvedStsOriginalAccessKeyId() : "", | ||
| s3Auth.hasResolvedStsRoleArn() ? s3Auth.getResolvedStsRoleArn() : "", | ||
| java.time.Instant.MAX, // ensure it deterministically is not expired | ||
| "", // no secretAccessKey needed | ||
| s3Auth.hasResolvedStsSessionPolicy() ? s3Auth.getResolvedStsSessionPolicy() : "", | ||
| null // no encryption key needed | ||
| ); | ||
| OzoneManager.setStsTokenIdentifier(rehydratedTokenIdentifier); | ||
| isStsThreadLocalSet = true; | ||
| } |
ChenSammi
reviewed
May 20, 2026
|
|
||
| return getOmRequest().toBuilder() | ||
| // super.preExecute resolves S3Authentication (STS) for Ratis apply. Merge SetAclRequest changes on top. | ||
| final OMRequest request = super.preExecute(ozoneManager); |
Contributor
There was a problem hiding this comment.
You mentioned OMBucketDeleteRequest in the JIRA description. Shall we call the super.preExecute in OMBucketDeleteRequest too?
Contributor
There was a problem hiding this comment.
There are two other requests, OMBucketAddAclRequest, OMBucketRemoveAclRequest. So we should consider move this to OMBucketAclRequest.
ChenSammi
reviewed
May 20, 2026
| OzoneManagerProtocolProtos.S3Authentication s3Auth, STSTokenIdentifier stsTokenIdentifier) { | ||
| final OzoneManagerProtocolProtos.S3Authentication.Builder s3AuthBuilder = s3Auth.toBuilder(); | ||
|
|
||
| if (s3Auth.hasSessionToken() && !s3Auth.getSessionToken().isEmpty() && stsTokenIdentifier != null) { |
Contributor
There was a problem hiding this comment.
Is "s3Auth.hasSessionToken() && !s3Auth.getSessionToken().isEmpty()" and null stsTokenIdentifier allowed?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please describe your PR in detail:
PutObject(i.e. OMKeyCreateRequest, OMAllocateBlockRequest, OMKeyCommitRequest),DeleteObject(i.e. OMKeyDeleteRequest),PutObjectTagging(i.e. S3PutObjectTaggingRequest), etc. perform their ACL checks inpreExecute()which is on the OM leader RPC thread.However, APIs like
DeleteBucket(i.e. OMBucketDeleteRequest),PutBucketAcl(i.e. OMBucketSetAclRequest), etc. perform their ACL checks invalidateAndUpdateCache()which is on the Ratis apply thread. This affects STS in that the STSTokenIdentifier ThreadLocal currently is not available on the Ratis apply thread, so if the STS token has an inline session policy, some ACL checks that should pass would fail. This ticket addresses the inconsistency by ensuring the ThreadLocal is always available on the Ratis apply thread via updates toOzoneManagerStateMachine.A separate PR is already open to move the checks to the correct place (#9653 and https://issues.apache.org/jira/browse/HDDS-13855), but this ticket is a fallback in case any future API has the check in the incorrect place, so it won't break STS.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14935
How was this patch tested?
unit tests, smoke tests