Skip to content

HDDS-15304. DeleteObjects should enforce the request key limit#10309

Open
sreejasahithi wants to merge 1 commit into
apache:masterfrom
sreejasahithi:HDDS-15304
Open

HDDS-15304. DeleteObjects should enforce the request key limit#10309
sreejasahithi wants to merge 1 commit into
apache:masterfrom
sreejasahithi:HDDS-15304

Conversation

@sreejasahithi
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

The multi object delete iterates all request entries and forwards them to bucket.deleteKeys without checking the S3 1000 key maximum limit.
It should reject DeleteObjects requests with more than 1000 objects before execution and return the error.

What is the link to the Apache JIRA

HDDS-15304

How was this patch tested?

Added testcases
Green CI : https://github.com/sreejasahithi/ozone/actions/runs/26077522597

@sreejasahithi
Copy link
Copy Markdown
Contributor Author

@peterxcli could you please review this patch.

Copy link
Copy Markdown
Member

@peterxcli peterxcli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sreejasahithi sreejasahithi marked this pull request as ready for review May 19, 2026 16:25
@sreejasahithi
Copy link
Copy Markdown
Contributor Author

Sample output after the fix.

bash-5.1$ 
bash-5.1$ 
bash-5.1$ aws s3api delete-objects \
 --bucket test-delete-limit \
 --delete "$(python3 -c "
import json
keys = [{'Key': f'key-{i}'} for i in range(1, 1002)]
print(json.dumps({'Objects': keys, 'Quiet': False}))
")" \
 --endpoint-url http://localhost:9878/

An error occurred (MalformedXML) when calling the DeleteObjects operation: The XML you provided was not well-formed or did not validate against our published schema
bash-5.1$ 
bash-5.1$ 
bash-5.1$ aws s3api delete-objects \
 --bucket test-delete-limit \
 --delete "$(python3 -c "
import json
keys = [{'Key': f'key-{i}'} for i in range(1, 1000)]
print(json.dumps({'Objects': keys, 'Quiet': False}))
")" \
 --endpoint-url http://localhost:9878/
{
  "Deleted": [
    {
      "Key": "key-1"
    },
    {
      "Key": "key-2"
    },
    {
      "Key": "key-3"
    },
    {
      "Key": "key-4"
    },
    {
      "Key": "key-5"
    },
    {
      "Key": "key-6"
    },
    {
      "Key": "key-7"
    },
    {
      "Key": "key-8"
    },
    {
      "Key": "key-9"


if (request.getObjects() != null
&& request.getObjects().size() > S3Consts.S3_DELETE_OBJECTS_MAX_KEYS) {
throw newError(S3ErrorTable.MALFORMED_XML, bucketName);
Copy link
Copy Markdown
Member

@peterxcli peterxcli May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should throw clear exception

Suggested change
throw newError(S3ErrorTable.MALFORMED_XML, bucketName);
throw newError(S3ErrorTable.INVALID_ARGUMENT, "too many objects");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants