Skip to content

HDDS-9534. Support namespace summaries (du, dist & counts) for LEGACY buckets with file system disabled#5517

Merged
devmadhuu merged 19 commits into
apache:masterfrom
ArafatKhan2198:HDDS-9534
Mar 29, 2024
Merged

HDDS-9534. Support namespace summaries (du, dist & counts) for LEGACY buckets with file system disabled#5517
devmadhuu merged 19 commits into
apache:masterfrom
ArafatKhan2198:HDDS-9534

Conversation

@ArafatKhan2198
Copy link
Copy Markdown
Contributor

@ArafatKhan2198 ArafatKhan2198 commented Oct 31, 2023

What changes were proposed in this pull request?

Recon NSSummaries currently support FSO, OBS buckets, and Legacy buckets with the FSEnabled flag set to true. When FSEnabled is false, Legacy buckets act like OBS buckets, requiring a similar approach for managing keys. We've adapted our OBS bucket handling methods for these cases, maximizing code reuse.

Here's a breakdown of the changes :-

  1. NSSummaryTaskWithLegacy.java: This includes methods to process keys in Legacy buckets, distinguishing between filesystem and object store layouts to generate NSSummaries accurately.
  2. BucketHandler.java: Depending on whether FileSystemPaths is enabled, we assign the appropriate handler for each bucket. For Legacy buckets with disabled FileSystemPaths, we employ OBS bucket handler, thanks to their identical key management approach.
  3. EntityHandler.java: This class is activated by the NSSummaryEndpoint and determines which handler to assign based on the path provided by the user. For Legacy buckets with the FileSystemPaths option disabled, we directly employ the OBSHandler and its associated methods. We have introduced specific normalization methods for OBS keys. OBS keys follow the format volumeName/bucketName/keyName, where the keyName can include any characters, even multiple slashes, such as “////KeyName.” There's no need to normalize the keyName itself, but it's essential to ensure the path up to the bucket is correctly normalized. This is because volumeName and bucketName must not contain multiple slashes or spaces. Our standard normalization methods for the FSO layout normalize the entire path, including the keyName, which could alter the keyName and lead to invalid output. To address this, I've added a method in the OmUtils.java class that normalizes the path up to the bucket level.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-9534

How was this patch tested?

Manual Testing and Unit Testing

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants