std.fs.test.test "file operations on directories" fails on Linux

[marnix]

Running the std tests on Ubuntu Linux 18.04.5, for recent master (12ce6eb) using matching precompiled zig-linux-x86_64-0.6.0+12ce6eb8f.tar.xz fails:

$ zig --override-lib-dir $PWD/lib --main-pkg-path $PWD/lib test $PWD/lib/std/std.zig 
Test [224/1523] std.progress.test "basic functionality"...SKIP
Test [605/1523] std.event.loop.test "std.event.Loop - basic"...SKIP
Test [631/1523] std.io.c_writer.test ""...SKIP
Test [648/1523] std.fs.test.test "file operations on directories"...expected error.IsDir, found 
/home/marnix/projects/zig/lib/std/testing.zig:27:24: 0x9f4e05 in std.testing.expectError (test)
        std.debug.panic("expected error.{}, found {}", .{ @errorName(expected_error), actual_payload });
                       ^
/home/marnix/projects/zig/lib/std/fs/test.zig:258:28: 0x40eb40 in std.fs.test.test "file operations on directories" (test)
        testing.expectError(error.IsDir, tmp_dir.dir.readFileAlloc(testing.allocator, test_dir_name, std.math.maxInt(usize)));
                           ^
/home/marnix/projects/zig/lib/std/special/test_runner.zig:61:28: 0x4872a1 in std.special.main (test)
        } else test_fn.func();
                           ^
/home/marnix/projects/zig/lib/std/start.zig:257:37: 0x450e4d in std.start.posixCallMainAndExit (test)
            const result = root.main() catch |err| {
                                    ^
/home/marnix/projects/zig/lib/std/start.zig:128:5: 0x450b8f in std.start._start (test)
    @call(.{ .modifier = .never_inline }, posixCallMainAndExit, .{});
    ^

Tests failed. Use the following command to reproduce the failure:
/media/marnux3data/data/projects/zig/zig-cache/o/VM696BU7V7eI0ftno9RVM7Y3OPoU2itihaW2KO0QnNOI0brEapB_8hJP-FZnUkif/test

(The line where things fail is

zig/lib/std/fs/test.zig

Line 258 in 12ce6eb

testing.expectError(error.IsDir, tmp_dir.dir.readFileAlloc(testing.allocator, test_dir_name, std.math.maxInt(usize)));

. Adding -target x86_64-linux-none or -target x86_64-linux-gnu or -target x86_64-linux-musl doesn't change the behavior.)

From what gdb debugging I could manage, the issue seems to be that Linux openat() on a directory only returns EISDIR if a write action is requested, not for read-only. (See https://man7.org/linux/man-pages/man2/openat.2.html, which says that EISDIR is returned if "pathname refers to a directory and the access requested involved writing (that is, O_WRONLY or O_RDWR is set).") And reading that directory file descriptor then results in a zero-length 'file'. So this test seems wrong?

However, on the other hand, I see the Linux pull request builds having nice success output lines like

2020-09-02T01:21:53.0021801Z 631/1506 fs.test.test "std-x86_64-linux-none-Debug-bare-multi file operations on directories"...OK

[squeek502]

Relevant PR where this test was added: #5717. See also #5732 and

zig/lib/std/fs/test.zig

Lines 260 to 262 in 12ce6eb

	// Note: The `.write = true` is necessary to ensure the error occurs on all platforms.
	// TODO: Add a read-only test as well, see https://github.com/ziglang/zig/issues/5732
	testing.expectError(error.IsDir, tmp_dir.dir.openFile(test_dir_name, .{ .write = true }));

Note that this test is meant to get an error at the read phase rather than the open phase. I'm unsure why that's not happening for you, though.

reading that directory file descriptor then results in a zero-length 'file'

According to the read docs:

ERRORS
    EISDIR fd refers to a directory.

[marnix]

@squeek502 OK, that helps, thanks!

I think I now understand what is going on: No read() is done because stat() says the directory has size zero.

I'm running these tests on a file system (BitLocker, don't ask) where stat() seems to say a directory entry is 0 bytes long, called in getEndPos()

zig/lib/std/fs/file.zig

Lines 234 to 239 in 12ce6eb

	pub fn getEndPos(self: File) GetPosError!u64 {
	if (builtin.os.tag == .windows) {
	return windows.GetFileSizeEx(self.handle);
	}
	return (try self.stat()).size;
	}

. Therefore readAll() at

zig/lib/std/io/reader.zig

Line 38 in 12ce6eb

pub fn readAll(self: Self, buffer: []u8) Error!usize {

is given a zero-size buffer, and no system read() is ever attempted. So no EISDIR so no error.IsDir but an empty buffer results.

---

As for fixing this, it seems helpful to make getEndPos() instead return error.IsDir if the entry is a directory?

---

Wait, this issue seems to be a kind of duplicate of #6232, which starts out, "Some files such as the ones in /proc report a st_size of zero, try to read the file anyway if we hit that case."

From how I read it, the fix proposed there should presumably also solve this issue, but in a different way. Update: Confirmed that lib/std/fs/test.zig works fine on the current version of that branch, LemonBoy@5f31d54.

[marnix]

Indeed, as expected, works fine with #6232's a496f94 merge commit. Closing. Thanks @LemonBoy !