Docker images are pulled by anonymous user rather then authenticated

[kshpak]

I have logged in with docker login -u <username> and base64 encrypted record was written to ~/.docker/config.json as:

"https://index.docker.io/v1/": {
	"auth": "<base64encoded username:password>"
}

However, when I use Testcontainers it pulls images as an anonymous user rather than authenticated.
Examined by pull ratelimit-remaining check:

[anonymous] TOKEN=$(curl "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
[authenticated] TOKEN=$(curl --user 'username:password' "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl --head -H "Authorization: Bearer $TOKEN" https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest

I see that anonymous user pull ratelimit-remaining decreases, however for an authenticated user, it stays the same.

The problem leads to reaching Docker pull rate limit much faster

Error during callback: com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"toomanyrequests: You have reached your pull rate limit

[kiview]

Hi @kshpak, Testcontainers supports authenticated pull. Can you share more information about your environment? Does this happen on your workstation? You are on Linux? Which version of Docker are you using?

Also, please share your Testcontainers logs.

[kshpak]

Hi, @kiview
Docker version 20.10.12
RHEL8 Linux
Logs: https://pastebin.com/xL3EpBVc

[kiview]

Thanks, can you please share the logs again, using log level of DEBUG?
https://www.testcontainers.org/supported_docker_environment/logging_config/

[kshpak]

@kiview set everything for DEBUG level.
Workstation Fedora 34
Docker version 20.10.11

Issue can be reproduced
logs.txt

[rnorth]

@kshpak I notice that you've edited your comment above, presumably due to credentials appearing.

I've deleted the original revision of the comment so that it cannot be retrieved.

[kshpak]

hi, @kiview
what is the status with this ticket? is this an issue indeed?

[rnorth]

I think you've run into this existing issue: #4474

The key log lines are:

09:50:45.727 [build-33] INFO  🐳 [docker.io/postgres:14.1] - Pulling docker image: docker.io/postgres:14.1. Please be patient; this may take some time but only needs to be done once.
09:50:45.728 [build-33] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: docker.io/postgres:latest at registry: docker.io
09:50:45.728 [build-33] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/kshpak/.docker/config.json (exists) and commandPathPrefix: 
09:50:45.728 [build-33] DEBUG org.testcontainers.utility.RegistryAuthLocator - registryName [docker.io] for dockerImageName [docker.io/postgres:latest]
09:50:45.728 [build-33] DEBUG org.testcontainers.utility.RegistryAuthLocator - No matching Auth Configs - falling back to defaultAuthConfig [null]

If you were to remove the docker.io/ prefix from your image name, I think it should work...

[llowinge]

@rnorth To me it looks like a bug if docker auth is used based on image prefix existence. wdyt ?