Add retry for os.rename in atomicFileWriter

[karman-docker]

Problem: Updating files using atomicFileWriter can fail during rename on Windows, due to AV software holding the file handle.

Reproduction steps:
Run this simple program on Windows

package main

import (
	"fmt"
	"log"
	//"time"

	"github.com/moby/sys/atomicwriter"
)

func main() {
	data := []byte(`{
  "builder": {
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "experimental": false
}`)
	var count uint64
        fmt.Println("starting the test")
	for {
		count++
		//time.Sleep(50 * time.Millisecond)
		err := atomicwriter.WriteFile("test_daemon.json", data, 0o644)
		if err != nil {
			log.Fatalf("write failed after %d iterations: %v\n", count, err)
		}
	}
}

It will fail with below error after 1 or 2 minutes:

2026/04/24 12:16:24 write failed after 7807 iterations: rename C:\Users\azureuser\tmp\.tmp-test_daemon.json854562021 C:\Users\azureuser\tmp\test_daemon.json: Access is denied.
exit status 1

I just have Microsoft Defender on this system with default configuration.

Some environments (finance sector customers) have strict security policy where customers cannot exclude folders from AV scans.

Adding a retry around os.rename at https://github.com/moby/sys/blob/main/atomicwriter/atomicwriter.go#L155 can avoid this transient rename failure.

[karman-docker]

For reference, output of that program:

[crazy-max]

Posting here after offline discussion.

Don't think retry on os.Rename is the best fix.

Today atomicwriter.Close() does a path-based rename only after the temp file has been closed:

sys/atomicwriter/atomicwriter.go

Lines 138 to 156 in 9d08d8f

	func (w *atomicFileWriter) Close() (retErr error) {
	defer func() {
	if err := os.Remove(w.f.Name()); !errors.Is(err, os.ErrNotExist) && retErr == nil {
	retErr = err
	}
	}()
	if err := w.f.Sync(); err != nil {
	_ = w.f.Close()
	return err
	}
	if err := w.f.Close(); err != nil {
	return err
	}
	if err := os.Chmod(w.f.Name(), w.perm); err != nil {
	return err
	}
	if w.writeErr == nil && w.written {
	return os.Rename(w.f.Name(), w.fn)
	}

On Windows, os.Rename goes through os/file_windows.go: https://github.com/golang/go/blob/go1.26.1/src/os/file_windows.go#L259-L265, which calls windows.Rename: https://github.com/golang/sys/blob/v0.1.0/windows/syscall_windows.go#L680-L690, and that's just MoveFileExW: https://github.com/golang/sys/blob/v0.1.0/windows/syscall_windows.go#L201 with MOVEFILE_REPLACE_EXISTING.

That means the current implementation has a close-then-rename race on the temp path. If Windows Defender or another filter grabs the temp file in that window, the rename fails. Retrying may paper over that, but it doesn't address the underlying race.

A better direction would be a Windows-specific, handle-based rename, similar to what newer Go internals do in internal/syscall/windows/Renameat: https://github.com/golang/go/blob/go1.26.1/src/internal/syscall/windows/at_windows.go#L364-L430.

That path uses NtSetInformationFile with FILE_RENAME_INFORMATION_EX, FILE_RENAME_REPLACE_IF_EXISTS, and FILE_RENAME_POSIX_SEMANTICS, then falls back for older filesystems / older Windows behavior. The relevant x/sys constants are already present in https://github.com/golang/sys/blob/v0.1.0/windows/types_windows.go#L2332-L2359 and https://github.com/golang/sys/blob/v0.1.0/windows/types_windows.go#L2638-L2648, and there is already a test using NtSetInformationFile for rename in: https://github.com/golang/sys/blob/v0.1.0/windows/syscall_windows_test.go#L938-L945.

Something to note is the temp file is currently opened by sequential.openSequential:

sys/sequential/sequential_windows.go

Lines 65 to 105 in 9d08d8f

	func openSequential(path string, mode int) (fd windows.Handle, err error) {
	if len(path) == 0 {
	return windows.InvalidHandle, windows.ERROR_FILE_NOT_FOUND
	}
	pathp, err := windows.UTF16PtrFromString(path)
	if err != nil {
	return windows.InvalidHandle, err
	}
	var access uint32
	switch mode & (windows.O_RDONLY | windows.O_WRONLY | windows.O_RDWR) {
	case windows.O_RDONLY:
	access = windows.GENERIC_READ
	case windows.O_WRONLY:
	access = windows.GENERIC_WRITE
	case windows.O_RDWR:
	access = windows.GENERIC_READ | windows.GENERIC_WRITE
	}
	if mode&windows.O_CREAT != 0 {
	access |= windows.GENERIC_WRITE
	}
	if mode&windows.O_APPEND != 0 {
	access &^= windows.GENERIC_WRITE
	access |= windows.FILE_APPEND_DATA
	}
	sharemode := uint32(windows.FILE_SHARE_READ | windows.FILE_SHARE_WRITE)
	var sa *windows.SecurityAttributes
	if mode&windows.O_CLOEXEC == 0 {
	sa = makeInheritSa()
	}
	var createmode uint32
	switch {
	case mode&(windows.O_CREAT|windows.O_EXCL) == (windows.O_CREAT | windows.O_EXCL):
	createmode = windows.CREATE_NEW
	case mode&(windows.O_CREAT|windows.O_TRUNC) == (windows.O_CREAT | windows.O_TRUNC):
	createmode = windows.CREATE_ALWAYS
	case mode&windows.O_CREAT == windows.O_CREAT:
	createmode = windows.OPEN_ALWAYS
	case mode&windows.O_TRUNC == windows.O_TRUNC:
	createmode = windows.TRUNCATE_EXISTING
	default:
	createmode = windows.OPEN_EXISTING

with FILE_SHARE_READ | FILE_SHARE_WRITE, not FILE_SHARE_DELETE, and without delete access. So if we want the cleaner non-retry fix, we likely need to adjust how the temp file is opened on Windows as part of the same change.

So The stronger fix is to use the better Windows rename primitive first, because that removes the current close-then-rename race instead of just sleeping and trying again.

[karman-docker]

import (
	"os"
	"unsafe"

	"golang.org/x/sys/windows"
)

// fileRenameInformation is the FILE_RENAME_INFORMATION structure used by
// NtSetInformationFile to rename a file. FileName is a variable-length
// field; callers must allocate a buffer large enough to hold the full name.
type fileRenameInformation struct {
	ReplaceIfExists uint32
	RootDirectory   windows.Handle
	FileNameLength  uint32
	FileName        [1]uint16
}

// fileRenameInformationEx is the FILE_RENAME_INFORMATION_EX structure used by
// NtSetInformationFile to rename a file.
type fileRenameInformationEx struct {
	Flags          uint32
	RootDirectory  windows.Handle
	FileNameLength uint32
	FileName       [1]uint16
}

// atomicwriterRenameAt renames oldpath to newpath using NtSetInformationFile with
// FILE_RENAME_POSIX_SEMANTICS, which allows atomic replacement of a file
// even when the destination is open by another process.
func atomicwriterRenameAt(oldpath, newpath string) error {
	// Open the source file requesting DELETE access so we can rename it.
	srcPtr, err := windows.UTF16PtrFromString(oldpath)
	if err != nil {
		return &os.PathError{Op: "rename", Path: oldpath, Err: err}
	}
	handle, err := windows.CreateFile(
		srcPtr,
		windows.DELETE,
		windows.FILE_SHARE_READ|windows.FILE_SHARE_WRITE|windows.FILE_SHARE_DELETE,
		nil,
		windows.OPEN_EXISTING,
		windows.FILE_ATTRIBUTE_NORMAL,
		0,
	)
	if err != nil {
		return &os.PathError{Op: "rename", Path: oldpath, Err: err}
	}
	defer windows.CloseHandle(handle)

	// NtSetInformationFile requires an absolute NT path (\??\C:\...) when
	// RootDirectory is NULL.
	ntNewPath := `\??\` + newpath
	newPathUTF16, err := windows.UTF16FromString(ntNewPath)
	if err != nil {
		return &os.PathError{Op: "rename", Path: newpath, Err: err}
	}

	fileNameLen := len(newPathUTF16)*2 - 2 // byte length, excluding null terminator
	renameInfoEx := fileRenameInformationEx{
		Flags: windows.FILE_RENAME_REPLACE_IF_EXISTS |
			windows.FILE_RENAME_POSIX_SEMANTICS,
	}
	var dummyEx fileRenameInformationEx
	bufferSizeEx := int(unsafe.Offsetof(dummyEx.FileName)) + fileNameLen
	bufferEx := make([]byte, bufferSizeEx)
	infoEx := (*fileRenameInformationEx)(unsafe.Pointer(&bufferEx[0]))
	infoEx.Flags = renameInfoEx.Flags
	infoEx.FileNameLength = uint32(fileNameLen)
	copy((*[windows.MAX_LONG_PATH]uint16)(unsafe.Pointer(&infoEx.FileName[0]))[:fileNameLen/2:fileNameLen/2], newPathUTF16)

	const (
		FileRenameInformation   = 10
		FileRenameInformationEx = 65
	)
	var iosbEx windows.IO_STATUS_BLOCK
	
	err = windows.NtSetInformationFile(handle, &iosbEx, &bufferEx[0], uint32(bufferSizeEx), FileRenameInformationEx,)
	if err == nil {
		return nil
	}
	
	// If the extended rename fails, fall back to the original FILE_RENAME_INFORMATION
	// which is supported on older versions of Windows. 
	var dummy fileRenameInformation
	bufferSize := int(unsafe.Offsetof(dummy.FileName)) + fileNameLen
	buffer := make([]byte, bufferSize)
	info := (*fileRenameInformation)(unsafe.Pointer(&buffer[0]))
	info.ReplaceIfExists = windows.FILE_RENAME_REPLACE_IF_EXISTS | windows.FILE_RENAME_POSIX_SEMANTICS
	info.FileNameLength = uint32(fileNameLen)
	copy((*[windows.MAX_LONG_PATH]uint16)(unsafe.Pointer(&info.FileName[0]))[:fileNameLen/2:fileNameLen/2], newPathUTF16)

	var iosb windows.IO_STATUS_BLOCK
	if err := windows.NtSetInformationFile(handle, &iosb, &buffer[0], uint32(bufferSize), FileRenameInformation); err != nil {
		return &os.PathError{Op: "rename", Path: newpath, Err: err}
	}
	return nil
}

I implemented atomicwriterRenameAt using NtSetInformationFile. However if try calling atomicwriterRenameAt in a loop like the repro program, I still get rename failure with access denied error.

So NtSetInformationFile can still fail if another external process has opened the file without specifying FILE_SHARE_DELETE.

[karman-docker]

I think we cannot avoid another external process open the file being renamed without specifying FILE_SHARE_DELETE (for e.g. Defender). Assuming that open operation is short lived(few milliseconds), I can't think of any other ways other than retry to give rename a chance to succeed. Apologies if I am overlooking anything here.