syscall for pipe(2)

[cfriedt]

Is your enhancement proposal related to a problem? Please describe.
While Zephyr has its own native pipe() call for IPC, the POSIX API could benefit by the addition of a pipe(2) system call to allow for better portability of userspace applications. In particular, in #24366, @pfalcon suggested (and I fully agree) that the socketpair(2) system call could be quite easily implemented with pipe(2).

The socketpair(2) implementation (#24367) is on hold until pipe(2) is implemented, tested and approved.

Describe the solution you'd like
I would be happy to implement pipe(2) and submit a PR.

Describe alternatives you've considered
The eventfd(2) syscall currently has an implementation in review #22863. However, AFAIK, it is a Linux-only syscall so applications written for some other OS that only supports the Berkeley Sockets API may not Just Work (TM) when being ported to Zephyr. Also, while eventfd(2) is better in terms of performance, it does not allow for arbitrary data to be passed between threads / processes.

Additional context

[cfriedt]

Incidentally, I opted out of using pipe(2) to implement socketpair(2) because it would duplicate the number of file descriptors used (UNIX pipes are only unidirectional, so you would need 2 pipes for 1 socketpair).

However, I see no harm in adding a pipe(2) syscall.

[pfalcon]

Incidentally, I opted out of using pipe(2) to implement socketpair(2) because it would duplicate the number of file descriptors used (UNIX pipes are only unidirectional, so you would need 2 pipes for 1 socketpair).

Yeah, a "small omission" in the grand plan I proposed, sorry about that ;-). On the other hand having 2 pairs of file descriptors isn't end of the world, at most it would cause issue with "edge cases" (in quotes, cab be "another small omission") of poll() implementation. Specifically, you have no problems when you poll for just read/write (can select a specific fd), and only issue comes when polling for both POLL_IN and POLL_OUT. All that assumes that socketpair() is implemented literally on top of pipe() ("in userspace"). It could also plug into it "under public API", effectively just reusing data transfer mechanism.

And let's look into what issues were raised re: initial implementation of socketpair() on top of networking subsys:

1. Security issue (re: binding sockets to specific endpoints).
2. Perhaps this went implied, by both myself and (I guess) Jukka hinted that such an implementation would "steal" resources from networking subsys. (And potentially complicate it with additional patching, and it's already complex with edge-case bugs still lurking).

The bottom line: whatever you do, it would cost you (or system) some resources. The point is trying to find a sweet point of compromise between code duplication and resource use. It's indeed hard to imagine where it may be, until you start to code it up. In the end, it yet may occur that patching networking subsys was the best choice, if turns out that you just duplicate its code (using existing IPC primitives).

And in that regard, having 4 instead of 2 file descriptors ain't that big price to pay, that's some 16 bytes of overhead, any realistic communication buffer would be much more than that.

(To elaborate, I don't try to keep nudging you into direction of pipe(). Instead, trying to explain how it comes I suggested such a "stupid idea" (which it may very well be).)

[pfalcon]

However, I see no harm in adding a pipe(2) syscall.

Well, you original comment as arrived by email notification didn't have phrase, and that's actually what I wanted to ask: With the prelude above, is it still in your interests to follow with any pipe()-related work? If not, let's "fix the losses", and close this issue (letting folks who need exactly pipe() to re-raise it later). And sorry again for "leading you astray".

[cfriedt]

Incidentally, I opted out of using pipe(2) to implement socketpair(2) because it would duplicate the number of file descriptors used (UNIX pipes are only unidirectional, so you would need 2 pipes for 1 socketpair).

Yeah, a "small omission" in the grand plan I proposed, sorry about that ;-). On the other hand having 2 pairs of file descriptors isn't end of the world, at most it would cause issue with "edge cases" (in quotes, cab be "another small omission") of poll() implementation. Specifically, you have no problems when you poll for just read/write (can select a specific fd), and only issue comes when polling for both POLL_IN and POLL_OUT. All that assumes that socketpair() is implemented literally on top of pipe() ("in userspace"). It could also plug into it "under public API", effectively just reusing data transfer mechanism.

And let's look into what issues were raised re: initial implementation of socketpair() on top of networking subsys:

1. Security issue (re: binding sockets to specific endpoints).

Socketpair fd's cannot be bound, and set errno to EISCONN.

2. Perhaps this went implied, by both myself and (I guess) Jukka hinted that such an implementation would "steal" resources from networking subsys. (And potentially complicate it with additional patching, and it's already complex with edge-case bugs still lurking).

Doesn't seem to be the case in terms of patching as I've implemented it in now a separate file altogether. The only resources that it may be stealing are the file descriptor pairs.

The bottom line: whatever you do, it would cost you (or system) some resources. The point is trying to find a sweet point of compromise between code duplication and resource use. It's indeed hard to imagine where it may be, until you start to code it up. In the end, it yet may occur that patching networking subsys was the best choice, if turns out that you just duplicate its code (using existing IPC primitives).

And in that regard, having 4 instead of 2 file descriptors ain't that big price to pay, that's some 16 bytes of overhead, any realistic communication buffer would be much more than that.

(To elaborate, I don't try to keep nudging you into direction of pipe(). Instead, trying to explain how it comes I suggested such a "stupid idea" (which it may very well be).)

Not at all, at no time did I say it was a "stupid idea", I completely agreed with the suggestion ;-) It may be the direction to go in at a later time.

[cfriedt]

However, I see no harm in adding a pipe(2) syscall.

Well, you original comment as arrived by email notification didn't have phrase, and that's actually what I wanted to ask: With the prelude above, is it still in your interests to follow with any pipe()-related work? If not, let's "fix the losses", and close this issue (letting folks who need exactly pipe() to re-raise it later). And sorry again for "leading you astray".

No need to apologize at all. I implemented socketpair(2) the same way using pipe(2) many years ago when I wrote a small RTOS. It's not a bad idea at all. However, my current focus is to just get socketpair(2) to work.

Yes, I would be fine to "cut the losses" and close this ticket. It can always be re-opened later if someone needs pipe.