AWS CLI slow startup

[gaul]

Describe the bug

AWS CLI is three times as slow as s3cmd when tested on a local S3 implementation. s3fs-fuse invokes AWS CLI hundreds of times so this slows down integration tests. I tested on an Intel i7-1250U. I previously reported these symptoms in #6500.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

$ time s3cmd ls s3://

real    0m0.091s
user    0m0.073s
sys     0m0.016s

Current Behavior

$ time AWS_EC2_METADATA_DISABLED=true aws s3 ls --region us-east-1 --endpoint-url http://127.0.0.1:8081 --profile s3proxy

real    0m0.346s
user    0m0.284s
sys     0m0.055s

Reproduction Steps

See above

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.17.18 Python/3.13.0 Linux/6.11.4-300.fc41.x86_64 source/x86_64.fedora.41

Environment details (OS name and version, etc.)

Fedora 41

[tim-finnigan]

Thanks for reaching out. Changing this to a feature request for increasing the speed. Running CLI commands with the --debug parameter will show how long each step of that process is taking. There are several steps involved, such as loading models and processing configurations. Can you provide any more details regarding your specific use case? For more information on optimizing S3 performance you can refer to the S3 User Guide.

[gaul]

I attached the requested logs: aws-cli.log

My use case is running the CLI in CI hundreds of times per instance: https://github.com/s3fs-fuse/s3fs-fuse/blob/master/test/integration-test-main.sh . A typical operation is s3api head-object or s3 ls -- these are small numbers and sizes of objects.

[imSanko]

Can you assign this task to me so that I can work on the same @gaul.

[gaul]

I addressed my symptom by using curl --aws-sigv4 instead of the AWS CLI. This reduced CI run-time from 14-15 minutes to 10-11: s3fs-fuse/s3fs-fuse#2689