[Bug]: Incompatibility in Ubuntu Github runners with Docker Engine 29.1.*

[rolandoskoulas-als]

Module

Core

Testcontainers version

2.0.3

Using the latest Testcontainers version?

Yes

Host OS

Linux

Host Arch

x86_64

Docker version

29.1.*

What happened?

Github's Ubuntu runners got upgraded to Docker 29.1. (https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md)

As a side effect, builds using testcontainers-java, even on the latest version, start failing.

I see the latest scheduled runs on your CI, possibly spot the issue: https://github.com/testcontainers/testcontainers-java/actions/runs/21969291388

Relevant log output

java.lang.IllegalStateException at DefaultCacheAwareContextLoaderDelegate.java:180
        Caused by: java.lang.IllegalStateException at SpringApplication.java:823
            Caused by: java.lang.ExceptionInInitializerError at Unsafe.java:1160
                Caused by: java.lang.IllegalStateException at DockerClientProviderStrategy.java:274

Additional Information

No response

[maksymsereda]

Same issue here

[k-go-gxp]

Same problem

[theozaurus]

We have the same using org.testcontainers:testcontainers-postgresql:2.0.3. We have worked around this by doing the following:

      - name: Force Docker API Version
        run: echo 'api.version=1.44' > ~/.docker-java.properties

The test logs when failing showed:

11:25:21.076 [Test worker] DEBUG o.t.jdbc.ContainerDatabaseDriver - Container not found in cache, creating new instance
11:25:21.164 [Test worker] DEBUG o.t.u.TestcontainersConfiguration - Testcontainers configuration overrides will be loaded from file:/home/runner/.testcontainers.properties
11:25:21.189 [Test worker] DEBUG o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/home/runner/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /home/runner/.testcontainers.properties (No such file or directory)
11:25:21.198 [Test worker] INFO  o.testcontainers.images.PullPolicy - Image pull policy will be performed by: DefaultPullPolicy()
11:25:21.204 [Test worker] INFO  o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
11:25:21.225 [Test worker] INFO  o.testcontainers.DockerClientFactory - Testcontainers version: 1.21.3
11:25:21.852 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - $XDG_RUNTIME_DIR is set but '/run/user/1001/docker.sock' does not exist.
11:25:21.853 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - '/home/runner/.docker/run' does not exist.
11:25:21.909 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - '/run/user/1001/docker.sock' does not exist.
11:25:21.909 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - Trying out strategy: UnixSocketClientProviderStrategy
11:25:22.233 [Test worker] DEBUG o.t.s.c.g.d.c.command.AbstrDockerCmd - Cmd: 
11:25:22.419 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - UnixSocketClientProviderStrategy: failed with exception BadRequestException (Status 400: {"message":"client version 1.32 is too old. Minimum supported API version is 1.44, please upgrade your client to a newer version"}
)
11:25:22.420 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - Trying out strategy: DockerDesktopClientProviderStrategy
11:25:22.421 [Test worker] DEBUG o.t.d.DockerDesktopClientProviderStrategy - '/home/runner/.docker/desktop' does not exist.
11:25:22.422 [Test worker] DEBUG o.t.d.DockerDesktopClientProviderStrategy - '/home/runner/.docker/run' does not exist.
11:25:22.423 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - DockerDesktopClientProviderStrategy: failed with exception NullPointerException (Cannot invoke "java.nio.file.Path.toString()" because the return value of "org.testcontainers.dockerclient.DockerDesktopClientProviderStrategy.getSocketPath()" is null)
11:25:22.426 [Test worker] INFO  o.t.d.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/usr/share/dotnet, /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/21.0.10-7/x64/bin, /snap/bin, /home/runner/.local/bin, /opt/pipx_bin, /home/runner/.cargo/bin, /home/runner/.config/composer/vendor/bin, /usr/local/.ghcup/bin, /home/runner/.dotnet/tools, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin, /usr/games, /usr/local/games, /snap/bin])
11:25:22.427 [Test worker] ERROR o.t.d.DockerClientProviderStrategy - Could not find a valid Docker environment. Please check configuration. Attempted configurations were:
	UnixSocketClientProviderStrategy: failed with exception BadRequestException (Status 400: {"message":"client version 1.32 is too old. Minimum supported API version is 1.44, please upgrade your client to a newer version"}
)
	DockerDesktopClientProviderStrategy: failed with exception NullPointerException (Cannot invoke "java.nio.file.Path.toString()" because the return value of "org.testcontainers.dockerclient.DockerDesktopClientProviderStrategy.getSocketPath()" is null)As no valid configuration was found, execution cannot continue.
See https://java.testcontainers.org/on_failure.html for more details.

With the above added to our Workflow we now get:

11:39:34.659 [Test worker] DEBUG o.t.jdbc.ContainerDatabaseDriver - Container not found in cache, creating new instance
11:39:34.735 [Test worker] DEBUG o.t.u.TestcontainersConfiguration - Testcontainers configuration overrides will be loaded from file:/home/runner/.testcontainers.properties
11:39:34.759 [Test worker] DEBUG o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/home/runner/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /home/runner/.testcontainers.properties (No such file or directory)
11:39:34.768 [Test worker] INFO  o.testcontainers.images.PullPolicy - Image pull policy will be performed by: DefaultPullPolicy()
11:39:34.772 [Test worker] INFO  o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
11:39:34.789 [Test worker] INFO  o.testcontainers.DockerClientFactory - Testcontainers version: 1.21.3
11:39:35.493 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - $XDG_RUNTIME_DIR is set but '/run/user/1001/docker.sock' does not exist.
11:39:35.494 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - '/home/runner/.docker/run' does not exist.
11:39:35.601 [Test worker] DEBUG o.t.d.RootlessDockerClientProviderStrategy - '/run/user/1001/docker.sock' does not exist.
11:39:35.601 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - Trying out strategy: UnixSocketClientProviderStrategy
11:39:35.884 [Test worker] DEBUG o.t.s.c.g.d.c.command.AbstrDockerCmd - Cmd: 
11:39:36.204 [Test worker] INFO  o.t.d.DockerClientProviderStrategy - Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
11:39:36.205 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - Transport type: 'httpclient5', Docker host: 'unix:///var/run/docker.sock'
11:39:36.205 [Test worker] DEBUG o.t.d.DockerClientProviderStrategy - Checking Docker OS type for local Unix socket (unix:///var/run/docker.sock)
11:39:36.210 [Test worker] INFO  o.testcontainers.DockerClientFactory - Docker host IP address is localhost
11:39:36.211 [Test worker] DEBUG o.testcontainers.DockerClientFactory - Docker info: {CDISpecDirs=[/etc/cdi, /var/run/cdi], NGoroutines=47, Name=runnervmjduv7, Swarm={ControlAvailable=false, Error=, LocalNodeState=inactive, NodeAddr=, NodeID=, RemoteManagers=null}, RuncCommit={Expected=v1.3.4-0-gd6d73eb8, ID=v1.3.4-0-gd6d73eb8}, OSVersion=24.04, Runtimes={io.containerd.runc.v2={path=runc, status={org.opencontainers.runtime-spec.features={"ociVersionMin":"1.0.0","ociVersionMax":"1.2.1","hooks":["prestart","createRuntime","createContainer","startContainer","poststart","poststop"],"mountOptions":["async","atime","bind","defaults","dev","diratime","dirsync","exec","iversion","lazytime","loud","mand","noatime","nodev","nodiratime","noexec","noiversion","nolazytime","nomand","norelatime","nostrictatime","nosuid","nosymfollow","private","ratime","rbind","rdev","rdiratime","relatime","remount","rexec","rnoatime","rnodev","rnodiratime","rnoexec","rnorelatime","rnostrictatime","rnosuid","rnosymfollow","ro","rprivate","rrelatime","rro","rrw","rshared","rslave","rstrictatime","rsuid","rsymfollow","runbindable","rw","shared","silent","slave","strictatime","suid","symfollow","sync","tmpcopyup","unbindable"],"linux":{"namespaces":["cgroup","ipc","mount","network","pid","time","user","uts"],"capabilities":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_DAC_READ_SEARCH","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_SETGID","CAP_SETUID","CAP_SETPCAP","CAP_LINUX_IMMUTABLE","CAP_NET_BIND_SERVICE","CAP_NET_BROADCAST","CAP_NET_ADMIN","CAP_NET_RAW","CAP_IPC_LOCK","CAP_IPC_OWNER","CAP_SYS_MODULE","CAP_SYS_RAWIO","CAP_SYS_CHROOT","CAP_SYS_PTRACE","CAP_SYS_PACCT","CAP_SYS_ADMIN","CAP_SYS_BOOT","CAP_SYS_NICE","CAP_SYS_RESOURCE","CAP_SYS_TIME","CAP_SYS_TTY_CONFIG","CAP_MKNOD","CAP_LEASE","CAP_AUDIT_WRITE","CAP_AUDIT_CONTROL","CAP_SETFCAP","CAP_MAC_OVERRIDE","CAP_MAC_ADMIN","CAP_SYSLOG","CAP_WAKE_ALARM","CAP_BLOCK_SUSPEND","CAP_AUDIT_READ","CAP_PERFMON","CAP_BPF","CAP_CHECKPOINT_RESTORE"],"cgroup":{"v1":true,"v2":true,"systemd":true,"systemdUser":true,"rdma":true},"seccomp":{"enabled":true,"actions":["SCMP_ACT_ALLOW","SCMP_ACT_ERRNO","SCMP_ACT_KILL","SCMP_ACT_KILL_PROCESS","SCMP_ACT_KILL_THREAD","SCMP_ACT_LOG","SCMP_ACT_NOTIFY","SCMP_ACT_TRACE","SCMP_ACT_TRAP"],"operators":["SCMP_CMP_EQ","SCMP_CMP_GE","SCMP_CMP_GT","SCMP_CMP_LE","SCMP_CMP_LT","SCMP_CMP_MASKED_EQ","SCMP_CMP_NE"],"archs":["SCMP_ARCH_AARCH64","SCMP_ARCH_ARM","SCMP_ARCH_MIPS","SCMP_ARCH_MIPS64","SCMP_ARCH_MIPS64N32","SCMP_ARCH_MIPSEL","SCMP_ARCH_MIPSEL64","SCMP_ARCH_MIPSEL64N32","SCMP_ARCH_PPC","SCMP_ARCH_PPC64","SCMP_ARCH_PPC64LE","SCMP_ARCH_RISCV64","SCMP_ARCH_S390","SCMP_ARCH_S390X","SCMP_ARCH_X32","SCMP_ARCH_X86","SCMP_ARCH_X86_64"],"knownFlags":["SECCOMP_FILTER_FLAG_TSYNC","SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_LOG"],"supportedFlags":["SECCOMP_FILTER_FLAG_TSYNC","SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_LOG"]},"apparmor":{"enabled":true},"selinux":{"enabled":true},"intelRdt":{"enabled":true},"mountExtensions":{"idmap":{"enabled":true}}},"annotations":{"io.github.seccomp.libseccomp.version":"2.5.5","org.opencontainers.runc.checkpoint.enabled":"true","org.opencontainers.runc.commit":"v1.3.4-0-gd6d73eb8","org.opencontainers.runc.version":"1.3.4\n"},"potentiallyUnsafeConfigAnnotations":["bundle","org.systemd.property.","org.criu.config"]}}}, runc={path=runc, status={org.opencontainers.runtime-spec.features={"ociVersionMin":"1.0.0","ociVersionMax":"1.2.1","hooks":["prestart","createRuntime","createContainer","startContainer","poststart","poststop"],"mountOptions":["async","atime","bind","defaults","dev","diratime","dirsync","exec","iversion","lazytime","loud","mand","noatime","nodev","nodiratime","noexec","noiversion","nolazytime","nomand","norelatime","nostrictatime","nosuid","nosymfollow","private","ratime","rbind","rdev","rdiratime","relatime","remount","rexec","rnoatime","rnodev","rnodiratime","rnoexec","rnorelatime","rnostrictatime","rnosuid","rnosymfollow","ro","rprivate","rrelatime","rro","rrw","rshared","rslave","rstrictatime","rsuid","rsymfollow","runbindable","rw","shared","silent","slave","strictatime","suid","symfollow","sync","tmpcopyup","unbindable"],"linux":{"namespaces":["cgroup","ipc","mount","network","pid","time","user","uts"],"capabilities":["CAP_CHOWN","CAP_DAC_OVERRIDE","CAP_DAC_READ_SEARCH","CAP_FOWNER","CAP_FSETID","CAP_KILL","CAP_SETGID","CAP_SETUID","CAP_SETPCAP","CAP_LINUX_IMMUTABLE","CAP_NET_BIND_SERVICE","CAP_NET_BROADCAST","CAP_NET_ADMIN","CAP_NET_RAW","CAP_IPC_LOCK","CAP_IPC_OWNER","CAP_SYS_MODULE","CAP_SYS_RAWIO","CAP_SYS_CHROOT","CAP_SYS_PTRACE","CAP_SYS_PACCT","CAP_SYS_ADMIN","CAP_SYS_BOOT","CAP_SYS_NICE","CAP_SYS_RESOURCE","CAP_SYS_TIME","CAP_SYS_TTY_CONFIG","CAP_MKNOD","CAP_LEASE","CAP_AUDIT_WRITE","CAP_AUDIT_CONTROL","CAP_SETFCAP","CAP_MAC_OVERRIDE","CAP_MAC_ADMIN","CAP_SYSLOG","CAP_WAKE_ALARM","CAP_BLOCK_SUSPEND","CAP_AUDIT_READ","CAP_PERFMON","CAP_BPF","CAP_CHECKPOINT_RESTORE"],"cgroup":{"v1":true,"v2":true,"systemd":true,"systemdUser":true,"rdma":true},"seccomp":{"enabled":true,"actions":["SCMP_ACT_ALLOW","SCMP_ACT_ERRNO","SCMP_ACT_KILL","SCMP_ACT_KILL_PROCESS","SCMP_ACT_KILL_THREAD","SCMP_ACT_LOG","SCMP_ACT_NOTIFY","SCMP_ACT_TRACE","SCMP_ACT_TRAP"],"operators":["SCMP_CMP_EQ","SCMP_CMP_GE","SCMP_CMP_GT","SCMP_CMP_LE","SCMP_CMP_LT","SCMP_CMP_MASKED_EQ","SCMP_CMP_NE"],"archs":["SCMP_ARCH_AARCH64","SCMP_ARCH_ARM","SCMP_ARCH_MIPS","SCMP_ARCH_MIPS64","SCMP_ARCH_MIPS64N32","SCMP_ARCH_MIPSEL","SCMP_ARCH_MIPSEL64","SCMP_ARCH_MIPSEL64N32","SCMP_ARCH_PPC","SCMP_ARCH_PPC64","SCMP_ARCH_PPC64LE","SCMP_ARCH_RISCV64","SCMP_ARCH_S390","SCMP_ARCH_S390X","SCMP_ARCH_X32","SCMP_ARCH_X86","SCMP_ARCH_X86_64"],"knownFlags":["SECCOMP_FILTER_FLAG_TSYNC","SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_LOG"],"supportedFlags":["SECCOMP_FILTER_FLAG_TSYNC","SECCOMP_FILTER_FLAG_SPEC_ALLOW","SECCOMP_FILTER_FLAG_LOG"]},"apparmor":{"enabled":true},"selinux":{"enabled":true},"intelRdt":{"enabled":true},"mountExtensions":{"idmap":{"enabled":true}}},"annotations":{"io.github.seccomp.libseccomp.version":"2.5.5","org.opencontainers.runc.checkpoint.enabled":"true","org.opencontainers.runc.commit":"v1.3.4-0-gd6d73eb8","org.opencontainers.runc.version":"1.3.4\n"},"potentiallyUnsafeConfigAnnotations":["bundle","org.systemd.property.","org.criu.config"]}}}}, BridgeNfIptables=false, DriverStatus=[[driver-type, io.containerd.snapshotter.v1]], ID=169aa517-19ec-4d49-bd61-4bb79f5886f6, LiveRestoreEnabled=false, SystemTime=2026-02-13T11:39:36.031324787Z, NoProxy=, Architecture=x86_64, HttpsProxy=, NEventsListener=0, Images=1, ContainersRunning=0, Labels=[], InitCommit={Expected=de40ad0, ID=de40ad0}, NFd=22, KernelVersion=6.14.0-1017-azure, LoggingDriver=json-file, OSType=linux, SecurityOptions=[name=apparmor, name=seccomp,profile=builtin, name=cgroupns], ContainerdCommit={Expected=dea7da592f5d1d2b7755e3a161be07f43fad8f75, ID=dea7da592f5d1d2b7755e3a161be07f43fad8f75}, GenericResources=null, OperatingSystem=Ubuntu 24.04.3 LTS, RegistryConfig={AllowNondistributableArtifactsCIDRs=null, AllowNondistributableArtifactsHostnames=null, IndexConfigs={docker.io={Mirrors=[], Name=docker.io, Official=true, Secure=true}}, InsecureRegistryCIDRs=[::1/128, 127.0.0.0/8], Mirrors=[]}, ServerVersion=29.1.5, OomKillDisable=false, CpuCfsQuota=true, Containers=0, Driver=overlayfs, IPv4Forwarding=true, ExperimentalBuild=false, DockerRootDir=/var/lib/docker, CPUSet=true, CpuCfsPeriod=true, DefaultRuntime=runc, CPUShares=true, Debug=false, NCPU=2, Plugins={Authorization=null, Log=[awslogs, fluentd, gcplogs, gelf, journald, json-file, local, splunk, syslog], Network=[bridge, host, ipvlan, macvlan, null, overlay], Volume=[local]}, SwapLimit=true, CgroupDriver=systemd, ContainersPaused=0, MemTotal=8329216000, IndexServerAddress=https://index.docker.io/v1/, HttpProxy=, MemoryLimit=true, InitBinary=docker-init, CgroupVersion=2, Isolation=, PidsLimit=true, BridgeNfIp6tables=false, Warnings=null, ContainersStopped=0}
11:39:36.216 [Test worker] DEBUG o.t.s.c.g.d.c.command.AbstrDockerCmd - Cmd: 
11:39:36.242 [Test worker] DEBUG o.testcontainers.DockerClientFactory - Docker version: {Components=[{Name=Engine, Version=29.1.5, Details={ApiVersion=1.52, Arch=amd64, BuildTime=2026-01-16T12:48:47.000000000+00:00, Experimental=false, GitCommit=3b01d64, GoVersion=go1.25.6, KernelVersion=6.14.0-1017-azure, MinAPIVersion=1.44, Os=linux}}, {Name=containerd, Version=v2.2.1, Details={GitCommit=dea7da592f5d1d2b7755e3a161be07f43fad8f75}}, {Name=runc, Version=1.3.4, Details={GitCommit=v1.3.4-0-gd6d73eb8}}, {Name=docker-init, Version=0.19.0, Details={GitCommit=de40ad0}}], KernelVersion=6.14.0-1017-azure, ApiVersion=1.52, Platform={Name=Docker Engine - Community}, Os=linux, GitCommit=3b01d64, Version=29.1.5, MinAPIVersion=1.44, Arch=amd64, GoVersion=go1.25.6, BuildTime=2026-01-16T12:48:47.000000000+00:00}
11:39:36.242 [Test worker] INFO  o.testcontainers.DockerClientFactory - Connected to docker: 
  Server Version: 29.1.5
  API Version: 1.52
  Operating System: Ubuntu 24.04.3 LTS
  Total Memory: 7943 MB

[jamf-software]

Same here

[adelinosousa]

Come across this issue as well and after some digging it seems that docker-machine is no longer an executable in the latest Ubuntu GH runners.

Checked docker API version and PATH on the Ubuntu runners as well and it all looks right.

Testcontainers team did push and release a change use 1.44 but I think what we need now is an update to the executable name to be able to run the latest Ubuntu GH runners.

[dsibilio]

Also looking for a long-term solution.

In the meanwhile, when running tests via maven-failsafe, this seems to solve the issue as well:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
        <configuration>
          <!-- temporary fix for https://github.com/testcontainers/testcontainers-java/issues/11491 -->
          <systemPropertyVariables>
            <api.version>1.44</api.version>
          </systemPropertyVariables>
        </configuration>
      </plugin>

[eddumelendez]

Hi, I have a project running with version 2.0.3 on a GH runner with engine 29.1 and it works. See https://github.com/eddumelendez/testcontainers-samples/actions/runs/21995183503/job/63553124230

Can you all please check there is no version conflict? maven: ./mvnw dependency:tree or gradle: ./gradlew dependencies

Please, remember all modules were prefixed with testcontainers-. So, starting with version 2.x we went from postgresql to testcontainers-postgresql

[dsibilio]

Hi, I have a project running with version 2.0.3 on a GH runner with engine 29.1 and it works. See https://github.com/eddumelendez/testcontainers-samples/actions/runs/21995183503/job/63553124230

Can you all please check there is no version conflict? maven: ./mvnw dependency:tree or gradle: ./gradlew dependencies

Please, remember all modules were prefixed with testcontainers-. So, starting with version 2.x we went from postgresql to testcontainers-postgresql

Thanks for looking into it, @eddumelendez. In my case, I can confirm a transitive dependency was overriding the testcontainers.version property to 1.21.3 so that's most likely the culprit.

[theozaurus]

I've removed the fix I suggested earlier and it now appears to be working fine. What's very odd is that the dependencies where setup with testcontainers-postgresql. However, my log except is clearly showing an older version.