std::process::Command's current_dir behaves differently on Unix and Windows, with respect to relative exe paths

[oconnor663]

On Unix, relative paths to the exe are interpreted after cd'ing to the current_dir, but on Windows they're interpreted before. Here's an example test (1 test file, and 1 little binary that the test needs to build) that demonstrates this if you run cargo test on the two different platforms:

src/lib.rs

#[test]
fn execute_binary_from_another_dir() {
    // Run `cargo build` to make sure the associated `foo` binary gets built.
    std::process::Command::new("cargo").arg("build").status().unwrap();

    // Shell out to `foo`, just to confirm that this works fine.
    std::process::Command::new("target/debug/foo")
        .status()
        .unwrap();

    // Now try to shell out to `foo` again while also setting the current_dir.
    // This fails on Unix, because the path to the binary is resolved *after*
    // changing directories, but it works on Windows.
    std::process::Command::new("target/debug/foo")
        .current_dir("..")
        .status()
        .unwrap();
}

src/bin/foo.rs

fn main() {
    println!("foo!");
}

I ran into a similar difference when I tried this with Python's subprocess.run(..., cwd=...). It looks like it's an artifact of Windows supporting an lpCurrentDirectory arg natively, while Unix has to fork and then chdir in the child before it calls exec. I prefer the semantics that we're getting on Windows right now*, but this issue is mainly that they're not the same.

* My main reason for preferring the current-Windows-style "current_dir does not affect the exe location" semantics: Emulating the current-Unix-style only requires a Path::join, which always works. But going the other way requires a Path::canonicalize, which can fail.

[alexcrichton]

Interesting! I believe your diagnosis is spot on as well.

A possible fix to this would be to always pass an absolute path on Unix but that'd involve probing which would inevitably go wrong somewhere... I agree though that the Windows semantics make more sense to me here.

[oconnor663]

@alexcrichton would it be possible to exec the new program with fexecve instead of execvp? I think would let us open the executable file before we change directories, and then execute it afterwards from its file descriptor. But it doesn't look like that's exposed in libc yet, and I don't know how portable it is?

[alexcrichton]

mind blown

Whoa I had no idea that API even existed! The portability does worry me though, especially this clause from the man page:

On Linux, fexecve() is implemented using the proc(5) file system, so /proc needs to be mounted and available at the time of the call.

We've had instances before of Rust binaries running without /proc and random things in glibc break (like thread info stuff). Putting this on Command::new may be a bit hindering :(

[oconnor663]

duct is currently working around this by doing a canonicalize under the covers. My reasoning here is that errors are acceptable, because in the situations that trigger this (relative program name, current_dir is in use) it's pretty much guaranteed that the invocation would fail otherwise anyway, unless the caller was actively prepared for the Unix semantics. But it's easier to say that as a higher level library, than it is as the stdlib.

[rzhikharevich]

I've just ran into this problem (specifically, the Unix behavior being rather unintuitive and undocumented).

@alexcrichton could you please describe what's wrong with making an absolute path with canonicalize before forking?

[alexcrichton]

@rzhikharevich I don't know of anything that could concretely go wrong right now, I'm just assuming that because this is low-level code used by tons of applications that the solution is bound to not be as simple as "just use canonicalize". In implementing a fix for this we basically just need to proceed with caution.

[oconnor663]

I'll note that I've seen canonicalize fail before on a VirtualBox shared drive, because the filesystem itself didn't support it. In that case I think a fallback of manually traversing .. entries would actually have worked, but I wouldn't be surprised if that created other issues. Just spitballing, it's possible that a manual traversal would give a different answer than canonicalize would have -- since I think the filesystem is allowed to implement canonicalize however it likes -- which could then lead to really weird behavior if canonicalize was failing say 50% of the time.

Even though it's not the semantics I prefer, doing a Path::join on Windows so that we always get the Unix behavior might be simpler. And in the defense of that behavior, relative paths passed to the child process as command line arguments will of course interact with current_dir, so having the binary path interact too isn't totally inconsistent.

But either way, this would probably constitute a breaking change at this point. Or at least, we'd have to be super careful to check the ecosystem and see who's depending on these details. I might go ahead and open a PR to document the current behavior at least.

[retep998]

We should never be using canonicalize on Windows unless we want to resolve symbolic links. If we just want to make a relative path absolute we should be using GetFullPathNameW.

[oconnor663]

@alexcrichton @retep998 @rzhikharevich I just added a docs PR above. Please take a look if you have time.