ACP: `process::resolve_exe` and `Command::resolve_in_parent_path`

[ChrisDenton]

Proposal

Problem statement

std::process::Command has the surprising behaviour of searching the child's PATH environment variable. Sometimes this is not wanted. Additionally rust on Windows has some further oddities in how it searches for executables whereas sometimes users do just want to look at the PATH.

Motivating examples or use cases

If you want to set child PATH such that it may not contain the application you want to run then you must manually find the binary. The following will panic on Unix systems:

std::process::Command::new("echo")
    .arg("hello world")
    .env("PATH", "")
    .spawn()
    .unwrap(); // panic!

Windows background

There are three basic search strategies people encounter on Windows (note here child_path is the PATH that's set on the child process, whereas PATH is from the parent's environment):

1. app_dir, system dirs, PATH
2. child_path, app dir, system dirs, PATH
3. PATH

The first is what happens when you use Command without setting a child PATH.
The second is what happens when you do.
The third can be kind of emulated via the second by using cmd.env(env::var("PATH").unwrap()).

People want the first for compatibility with other Windows applications and for when they do want to prioritise the app directory or find a system application.

People want the third when they want to do what powershell and Linux does or they explicitly don't want the system binary. For example, some varieties of Windows have WSL bash.exe in the system directory whereas people want to run git bash.exe.

The second can be used to kinda emulate searching PATH, if you don't mind giving up the ability to set the child PATH to something different.

Solution sketch

We could provide a way to get an absolute executable path from iterator over directories (e.g. as returned by split_paths).
This would enable inspecting and logging the path before the command is spawned.

// in std::process

/// Resolve the program to an absolute path by searching
/// the given paths for the executable.
pub fn resolve_exe(program: S, paths: I) -> io::Result<PathBuf>
where
S: AsRef<Path>.
I: IntoIterator<Item = P>,
P: AsRef<Path>,
{/*...*/

Alternatively have a function on Command that tells it the paths to use to resolve the executable. It is better for some platforms if Command can be given the list of paths to search:

// in std::process

impl Command {
    /// Resolve the program by searching only the given paths for the executable.
    /// `paths` should be in the same format as `PATH`.
    fn resolve_in_paths(&mut self, paths: impl AsRef<OsStr>) -> &mut Self;
}

Either way, it would be useful to have a helper for a common case:

impl Command {
    /// If `PATH` is set on the child and `program` is not a path
    /// then force the program to be resolved as if the child `PATH` wasn't set.
    fn resolve_in_parent_path(&mut self, use_parent: bool) -> &mut Self;
}

Alternatives

Don't provide a way to pass arbitrary paths and instead rely purely on boolean settings on Command, E.g. have a resolve_in_system_paths(bool) function on Command (maybe a Windows extension) which together with resolve_in_parent_path would allow selecting the wanted behaviour in many cases.

Links and related work

• which::which_in_global

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[programmerjake]

so basically you want which::which_in_global?

[ChrisDenton]

Kind of like that, yeah. The API proposed here has some differences though. It only returns a single exe path and it doesn't require using join_paths if you want to do any custom filtering or manually build a list of paths.

[ChrisDenton]

I've updated this ACP based on discussion.

• Command::resolve_in_parent_path takes a bool which, if true, forces spawn to use the parent process' PATH when searching for executables even if PATH is set on the child (whereas by default the child's PATH would be searched).
• process::resolve_exe searches the given paths for the executable and returns an absolute path to the executable (if found). This allows further customising the search paths as well as printing the absolute path for diagnostics or to otherwise inspect it before passing it to Command::new,

Names still open to bikeshedding.

[ChrisDenton]

Ok, I'm coming round to the idea of just having methods on Command but I'm not entirely sure.

I do like the idea of having a function that returns an absolute path because that's in some ways more flexible. You can log or inspect the path before you run anything.

However, the advantage of having a method on Command that takes the paths to search is that they can potentially be resolved in platform-specific ways (e.g. by trying to execute each path in turn). And this cannot be done outside of the standard library, at least not without implementing your own Command.

[ChrisDenton]

After more discussion I've expanded motivation to add some background for the Windows situation. I've also changed the suggested solution to include a function on Command. I liked the idea of having an absolute path to inspect but I'm not devoted to it and some people were keen for Command to be able to figure out how best to resolve the path (e.g. using system specific mechanism for that task).

[joshtriplett]

We had some preliminary discussion of this in today's @rust-lang/libs-api meeting.

Some things that came up, without definitive conclusions:

• Does this commit us, on every target, to always implement our own path resolution? (On some targets, we may be able to just defer to the OS for the actual execution, but this function would still require reimplementing path resolution.)
• What about chroot? Should our path resolution take that into account, and what should it return?
• What about future extensions letting us use execveat/fexecve, where there's a file descriptor but no path?
• After the path is resolved, should the Command avoid re-resolving it, by reusing the resolution (e.g. to avoid a TOCTTOU)?
• Some discussion on why this should be in Command.

[ChrisDenton]

What about chroot? Should our path resolution take that into account, and what should it return?

I think for Unix that's a major difference between setting PATH on the Command and not.
E.g. Command::new("ls").env("PATH", std::env::var("PATH").unwrap()).spawn() is not equivalent to Command::new("ls").spawn(). So it may make some sense to make it controllable. Which comes back to mara's point that we may need more options in the future.

What about future extensions letting us use execveat/fexecve, where there's a file descriptor but no path?

My feeling is that can be a new function. The fact that resolving options are ignored seems "obvious", I think?

[joshtriplett]

@ChrisDenton I'd propose that, if this is a function on Command, it should error if you've used an option we can't take into account in the resolution (e.g. before_exec/pre_exec/uid/gid/groups).

[joshtriplett]

We discussed this further in today's @rust-lang/libs-api meeting. We couldn't come to a consensus on whether this should be a free function or a function on Command. Some felt it should be a free function. Some felt it should be on Command so that it can take current and future builder methods into account.

[ChrisDenton]

I've created the tracking issue so discussion can move there: rust-lang/rust#141976