Skip to content

Panic when re-bootstrapping driver in non Running state #2464

@sgarcez

Description

@sgarcez

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

Under very specific conditions bootstrapping a driver can panic here due to a nil progress.Writer.

The flow seems to be:

  • In build cmd drivers are resolved explicitly here via nodeResolver.Resolve
  • This calls nodeResolver.boot here with a valid progress.Writer
  • Later there is another boot call here via resolvedNode.Client
  • Which calls nodeResolver.boot here with a nil progress.Writer
  • The actual problem is that later on, here, driver.Boot calls driver.Info and, if the status is not Running, it tries to bootstrap it again.
  • This panics due to the nil progress.Writer

In order to reproduce this the driver must not be Running by the time the second boot call happens, which is a bit of an edge case, but possible AFAICT.
In order to force this to reproduce we can simply patch e.g. the remote driver to always return Inactive.

func (d *Driver) Info(ctx context.Context) (*driver.Info, error) {
	return &driver.Info{
		Status: driver.Inactive,
	}, nil
}

Expected behaviour

No panic. An error message related to the driver state not being Running.

Actual behaviour

◊ ./bin/build/buildx create --name remote --driver remote tcp://localhost:1234
◊ ./bin/build/buildx build --builder remote --progress plain .
#0 building with "remote" instance using remote driver

#1 [internal] waiting for connection
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x1017dce98]

goroutine 154 [running]:
github.com/docker/buildx/driver.Boot({0x102d4b880, 0x140003fee10}, {0x102d4b880, 0x140003fe5a0}, 0x14000581f90, {0x0?, 0x0})
        github.com/docker/buildx/driver/driver.go:96 +0x98
github.com/docker/buildx/build.(*nodeResolver).boot.func1.1({0x102d4b880?, 0x140003fee10?})
        github.com/docker/buildx/build/driver.go:244 +0x5c
github.com/moby/buildkit/util/flightcontrol.(*call[...]).run(0x102d58000)
        github.com/moby/buildkit@v0.13.0-rc3.0.20240417151852-71f99c52a669/util/flightcontrol/flightcontrol.go:121 +0x1e8
sync.(*Once).doSlow(0x0?, 0x100b95a54?)
        sync/once.go:74 +0x100
sync.(*Once).Do(0x1400061a0f0?, 0x140004d5200?)
        sync/once.go:65 +0x24
created by github.com/moby/buildkit/util/flightcontrol.(*call[...]).wait in goroutine 150
        github.com/moby/buildkit@v0.13.0-rc3.0.20240417151852-71f99c52a669/util/flightcontrol/flightcontrol.go:165 +0x504

Buildx version

github.com/docker/buildx v0.14.0-36-gafcb6099.m afcb609.m

Docker info

No response

Builders list

NAME/NODE                          DRIVER/ENDPOINT                                      STATUS     BUILDKIT   PLATFORMS
remote                             remote
 \_ remote0                         \_ tcp://localhost:1234                             running    v0.13.2    linux/arm64
default                            docker
 \_ default                         \_ default                                          running    v0.13.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
desktop-linux*                     docker
 \_ desktop-linux                   \_ desktop-linux                                    running    v0.13.1    linux/arm64, linux/amd64, linux/amd64/v2, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Configuration

buildx checkout

◊ ./bin/build/buildx create --name remote --driver remote tcp://localhost:1234
◊ ./bin/build/buildx build --builder remote --progress plain .

Build logs

No response

Additional info

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions