Revisit passing of strings: redesign and replace NixPath

[kamalmarhubi]

https://github.com/carllerche/nix-rust/blob/00c7905a8a85668d0993420e356a56002ab23905/src/lib.rs#L115

I'm trying to bring back mount (#85) and while I can make it work, I'm not 100% sure of what the purpose of NixPath is. Could do with a 1-2 sentence explanation of why and when to use it. Its name is confusing for mount because the filesystem type is passed as a NixPath, though it isn't really a path.

[arcnmx]

NixPath needs a major rehaul...

1. It's a weird Into<CString> hybrid-like thing that also happens to use a large part of the stack and checks len against MAXPATHLEN, which isn't really helpful anyway.
2. It doesn't have an implementation for CStr/CString/etc. so you can't even avoid the stack and memcpy cost if you wanted to.
3. The closure with_nix_path usage is super awkward (just look at mount's current commented-out implementation!).
4. And then there are parts of nix that use &CString!!! These need to be killed and changed to &CStr or AsRef as appropriate. AND the linked execve implementation does behind-the-scenes allocations to generate the null-terminated list of strings. That should be its own type instead so the caller can pay that cost up-front! Otherwise you're calling collect::<Vec<_>>() after a fork/clone/etc!

We should basically just use AsRef<CStr> instead. For easy integration with nix, maybe provide a impl From<std::ffi::NulError> for nix::Error when using try!(CString::new(...)). Nix can also provide a new explicit NixPath to use when you actually want the stack allocation and MAXPATHLEN check:

struct NixPath([u8; MAX_PATH]);

impl NixPath {
    fn new<T: AsRef<[u8]>>(t: T) -> Result<Self> { ... }
}

impl AsRef<CStr> for NixPath { ... }

[kamalmarhubi]

It's a weird Into<CString> hybrid-like thing that also happens to use a large part of the stack

The stack usage gave me a lot of confusion with "disappearing" strings! (This was "my fault" for returning the result of as_ptr from the passed in closure, but not quite intuitive...)

We should basically just use AsRef<CStr> instead.

Using AsRef<CStr> seems limited. Not even CString implements it. Perhaps AsRef<OsStr> would be better. This would also allow passing Path and PathBuf, str, and string. This would require a copy to ensure null termination though... hrm.

[arcnmx]

The stack usage gave me a lot of confusion with "disappearing" strings! (This was "my fault" for returning the result of as_ptr from the passed in closure, but not quite intuitive...)

Mm, that's the purpose of using the closure. It's not a rare Rust idiom, but shouldn't really be used unless strictly necessary. Lifetimes would normally catch that sort of misuse, but oop pointers!

Using AsRef seems limited. Not even CString implements it. Perhaps AsRef would be better. This would also allow passing Path and PathBuf, str, and string. This would require a copy to ensure null termination though... hrm.

Yeah as soon as I realised that I submitted rust-lang/rust#30616 because that's a huge missing trait impl for libstd. It's silly, it basically means we're stuck with using our own trait instead that's just a clone of AsRef<CStr>. I'd suggest naming it something more descriptive than NixPath this time around, and when the AsRef<CStr> for CString hits upstream we can always just switch over to it.

Alternatively, the trait could look more like so:

trait ToCStr {
    type Target: AsRef<CStr>;
    fn to_cstr(&self) -> Result<Self::Target> { ... }
}

... which would retain the current behaviour of being able to pass OsStrs, Paths, etc. without the closure weirdness. I'm personally not a fan of the hidden conversions though, it feels a bit too magical for a Rust API vs just using AsRef<CStr> on its own (IMO the InvalidPath variant probably shouldn't be part of nix::Error at all, which should only cover errno).

[kamalmarhubi]

Mm, that's the purpose of using the closure. It's not a rare Rust idiom, but shouldn't really be used unless strictly necessary. Lifetimes would normally catch that sort of misuse, but oop pointers!

Yeah! Working on mount is part of my first attempt at a Rust project, and I had a bit too much faith in the borrow checker. Took me several runs under gdb to see that I was clobbering the buffer, and to understand that the pointer doesn't have a tracked lifetime. Learning!

I'd suggest naming it something more descriptive than NixPath this time around, and when the AsRef<CStr> for CString hits upstream we can always just switch over to it.

This sounds like a good plan.

Alternatively, the trait could look more like so: [...]

Assuming this was the desired path, would Into<T: AsRef<CStr>> work? I'm still learning how stuff works, but t'd be nice to exploit std traits as much as possible.

I'm personally not a fan of the hidden conversions though, it feels a bit too magical for a Rust API vs just using AsRef<CStr> on its own

I see where this is coming from. Rust definitely favours explicitness! In #190, @carllerche's stated goal of nix is "*I do know that the goal of nix is comprehensive, safe, zero cost bindings to system APIs on nix OSes", so your feelings are in line with that.

[kamalmarhubi]

Given the direction of discussion here, I'm changing the title of the issue :-)

[kamalmarhubi]

One additional niggle of using AsRef<CStr> is that there's no way to safely have a null CStr for functions that allow null char* args. This is already a problem with NixPath, just another thing to be aware of. Taking Option<&T> where T: AsRef<CStr> kind of works, but I can't figure out how to use None without needing to be explicit about the concrete type T.

[arcnmx]

Yeah, that's a pain... Option<T> with a default type would be the way to do it, but
default types influencing inference doesn't seem to be (fully?) implemented yet... Need to use Option<&CStr> instead until then. Not too big a pain when your type impls Deref<Target=CStr> (as CString does) at least. At the worst it means you have to do an .as_ref() in the call to the method.

For the most part, the advantage of AsRef<CStr> over &CStr is that it allows you to call the method with arguments without being required to & borrow all of them :P

[arcnmx]

In fact, until impl AsRef<CStr> for CString lands in libstd, we could really just use &CStr everywhere instead. The one thing the trait buys you is being able to move types into the call instead of requiring them to be borrowed... Making a nix-specific trait isn't a huge deal, but may not even be worth it when &CStr works almost as well thanks to deref coercion.

Assuming this was the desired path, would Into<T: AsRef> work? I'm still learning how stuff works, but t'd be nice to exploit std traits as much as possible.

The difference is that Into<T> is generic over T, while my example uses an associated type. It's a strange difference but I don't think the type inference would be able to work out the Into<T> approach without explicitly annotating parts of the function calls.

I see where this is coming from. Rust definitely favours explicitness!

I think it makes for a cleaner API, nix::Error having its InvalidPath variant is just kind of a weird hidden feature... Result<T, Errno> makes more sense to me, and these APIs take CStrings so that's what they should be passed! It makes things more verbose when your data is coming from a str, OsStr, or Path though... Which is a fairly common scenario so Idunno what other peoples' opinions are...

@carllerche: opinions?

[kamalmarhubi]

Yeah, that's a pain... Option<T> with a default type would be the way to do it, but
default types influencing inference doesn't seem to be (fully?) implemented yet...

Yeah, as one of my multiple attempts at a semi-ergonomic mount in the current framework, I had this monstrosity:

pub fn mount<P1: ?Sized=[u8], P2: ?Sized=[u8], P3: ?Sized=[u8], P4: ?Sized=[u8]>(
        source: Option<&P1>,
        target: &P2,
        fstype: Option<&P3>,
        flags: MsFlags,
        data: Option<&P4>) -> Result<()>
where  P1: NixPath, P2: NixPath, P3: NixPath, P4: NixPath {

It didn't work.

[kamalmarhubi]

I think it makes for a cleaner API, nix::Error having its InvalidPath variant is just kind of a weird hidden feature...

Especially for the fstype and data arguments of mount!

These APIs take CStrings, and that's what they should be passed!

This comes back to functions that allow NULL to be passed in. NULL is not a CString or CStr, and that makes things a bit difficult. Do nix maintainers want to be responsible for reading all platforms' manpages to be sure every nullable string argument is passed as an Option<&CStr>?

[arcnmx]

This comes back to functions that allow NULL to be passed in. NULL is not a CString or CStr, and that makes things a bit difficult. Do nix maintainers want to be responsible for reading all platforms' manpages to be sure every nullable string argument is passed as an Option<&CStr>?

Yes, that's exactly what Option<&> is for! Whether NULL is valid or not is usually well specified, though I've never actually seen an explanation for when NULL is okay in the mount parameters >_>

[kamalmarhubi]

I've never actually seen an explanation for when NULL is okay in the mount parameters >_>

My justification for the existing (commented out) set of NULL-allowing parameters:

    // Supplying null should be fine for these arguments: source can be a dummy
    // for, eg, tmpfs; fstype and data can be ignored for bind mounts.

Still, requiring careful reading of manpages to get the nix bindings right seems a bit error prone. It's probably fine, but we'll end up with a few breaking changes as things get fixed. Platform-specific differences can be handled by conditional compilation.

[arcnmx]

Error prone maybe, but very necessary. In Rust, nullability is part of the type system. In C, it's just part of an API contract. It's usually pretty obvious when something can be NULL or not, but yes, it's up to nix to figure out what the proper argument types to any function should be.

My justification for the existing (commented out) set of NULL-allowing parameters

Well, there needs to be a guarantee that it won't try and read from the pointer... What if someone supplies source = None with a fstype that's not tmpfs? What if fstype = None when flags doesn't specify MS_BIND or MS_MOVE etc? Will implementations just spit out EINVAL? Will they segfault? You generally have these options:

1. Get a guarantee from the specification that NULL is a valid argument.
2. Pass an empty string instead, something like source.unwrap_or(CStr::from_bytes(b"\0"))
3. Reflect these invariants in the type system itself. Instead of fsType: Option<&CStr>, make it an enum with variants for known cases where some arguments are optional.
   • You still may need to do 2. behind the scenes in the cases where you don't have the guarantee of 1.