fs.readSync throws EOF Error instead of returning 0 (Windows, if STDIN is a pipe)

[RomainMuller]

• Version: 14.15.0
• Platform: Windows 10 x86_64
• Subsystem: fs

What steps will reproduce the bug?

Given the following node script:

/// script.js
const fs = require('fs');

console.error('Node: ' + process.execPath);
console.error('Version: ' + process.version);
console.error('Platform: ' + process.platform);

console.error('process.stdin: ' + process.stdin.constructor.name);

const fd = 0; // STDIN
const buffer = Buffer.alloc(4_096);

let read;
do {
  read = fs.readSync(fd, buffer, 0, buffer.length, null);
  if (read != 0) {
      console.log('STDIN: ' + buffer.slice(0, read).toString('utf-8'))
  }
} while (read != 0);

console.log('EOF was (normally) reached.')

Will behave differently based on how it is invoked:

$ node script.js
Node: C:\Program Files\nodejs\node.exe
Version: v14.15.0
Platform: win32
process.stdin: ReadStream
Data!
STDIN: Data!
^Z
EOF was (normally) reached.


$ echo 'Data!' | node script.js
Node: C:\Program Files\nodejs\node.exe
Version: v14.15.0
Platform: win32
process.stdin: Socket
STDIN: Data!
internal/fs/utils.js:308
    throw err;
    ^

Error: EOF: end of file, read
    at Object.readSync (fs.js:592:3)
    at Object.<anonymous> (H:\repro\test\script.js:14:13)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47 {
  errno: -4095,
  syscall: 'read',
  code: 'EOF'
}

How often does it reproduce? Is there a required condition?

This issue happens 100% of the time when process.stdin is a Socket instance.

What is the expected behavior?

The expected and documented behavior is that fs.readSync returns 0 when EOF is reached.

What do you see instead?

Instead, an Error: EOF is thrown, which is unexpected.

Additional information

This problem was surfaced as users are starting to see failures on Windows at aws/aws-cdk#11314.

We have additionally seen issues where in similar conditions (process.stdin is a socket/pipe), attempting to perform fs.readSync on file descriptor 0 (aka STDIN), occasionally results in unexpected EAGAIN errors being thrown, which might be related. That was highlighted in aws/aws-cdk#5187 and we are currently working with a pretty ugly workaround that would be nice if removed (but that other issue is a lot more difficult to steadily reproduce, as it does not always trigger, and may be caused by side-effects of another operation).

[RomainMuller]

The stack trace of the error points at the following code:

node/lib/fs.js

Lines 590 to 592 in c1da528

	const result = binding.read(fd, buffer, offset, length, position,
	undefined, ctx);
	handleErrorFromBinding(ctx);

I basically expect the windows binding returns different codes for Socket than for ReadStream and in turn causes the difference in behavior.

[mmomtchev]

You will get much better results with:

/// script.js
const fs = require('fs');

console.error('Node: ' + process.execPath);
console.error('Version: ' + process.version);
console.error('Platform: ' + process.platform);

console.error('process.stdin: ' + process.stdin.constructor.name);

const buffer = Buffer.alloc(4_096);

let read;
do {
    read = fs.readFileSync('/dev/stdin', buffer, 0, buffer.length, null);
    if (read != 0) {
        console.log('STDIN: ' + buffer.slice(0, read).toString('utf-8'));
    }
} while (read != 0);

console.log('EOF was (normally) reached.');

Don't fiddle with the 0, 1 and 2 file descriptors. They are open in non-blocking mode and libuv doesn't expect this when you are passing a file descriptor.

[RomainMuller]

Hey @mmomtchev,

read = fs.readFileSync('/dev/stdin', buffer, 0, buffer.length, null);

This is interesting and all... But /dev/stdin is not a thing on Windows, and I cannot require WSL... 🤷🏻‍♂️
Or does fs have some magic tricks to make it look like this is a file that exists?

[mmomtchev]

No, you are right, I can't think of any easy way to synchronously read from stdin on Windows
There are npm modules or you could go the async way
There is an awful solution here http://corpus.hubwiz.com/2/node.js/3430939.html that will simulate blocking access, repeating the call on EAGAIN and breaking the loop on EOF

But the real answer is that Node is built for async I/O

[RomainMuller]

We already have the ugly code to loop on EAGAIN...
Unfortunately, the use-case I have does not lend itself to async exchange here... As there are ordering guarantees I must maintain.

The real thing I'm concerned with here, is how fs.readSync throws an EOF error... When it is documented to never do this and instead return 0.

[mmomtchev]

It says that the descriptor you are passing must be opened in blocking mode.

[RomainMuller]

It says that the descriptor you are passing must be opened in blocking mode.

Where though?

I'm looking at: https://nodejs.org/api/fs.html#fs_fs_readsync_fd_buffer_offset_length_position, and the only place on this page where a requirement for fd to be in blocking (or non-blocking for that matters) mode is on createReadStream and createWriteStream, none of which I am using...

---

In any case, in light of your confirming that we are doing things likely to trip libuv in weird and awkward ways, I've been implementing a more comprehensive (dirty) workaround in aws/jsii#2238 that also catches the EOF error; which would buy us time to figure out if/how we can proceed without needing to do synchronous I/O on our STD{IN,OUT,ERR}...

[mmomtchev]

@RomainMuller in fact, the stdin is not opened in non-blocking mode on Windows, that is the case only on Linux and OSX, so maybe there is a solution to your problem

[mmomtchev]

On Windows libuv does not properly handle pipes when operating in file mode. I wonder if it is supposed to, because normally, there is a separate pipe mode, but it is a trivial change. In any case, this issue should be taken to https://github.com/libuv/libuv
It happens because the writing process exits and closes the anonymous pipe.
Feel free to open it there, referencing this issue.

This is everything that is needed to make it work:

--- a/deps/uv/src/win/fs.c
+++ b/deps/uv/src/win/fs.c
@@ -918,7 +918,7 @@ void fs__read(uv_fs_t* req) {
     SET_REQ_RESULT(req, bytes);
   } else {
     error = GetLastError();
-    if (error == ERROR_HANDLE_EOF) {
+    if (error == ERROR_HANDLE_EOF || error == ERROR_BROKEN_PIPE) {
       SET_REQ_RESULT(req, bytes);
     } else {
       SET_REQ_WIN32_ERROR(req, error);

[mmomtchev]

But then again, you will end up with Windows-specific JS code, which is not the Node way

[mmomtchev]

@ronag is this supposed to work?

[ronag]

Sorry, sync api's is not my area of expertise.

[mmomtchev]

It is not specific to synchronous mode. This is getting stranger by the minute.
On Linux this leaves stdin in non-blocking mode:

const fs = require('fs');
console.error('process.stdin: ' + process.stdin.constructor.name);
(async () => {
    let read;
    read = fs.createReadStream(null, { fd: 0 })
    for await (const r of read)
        console.log(r)
    console.log('EOF was (normally) reached.')
})();

but this leaves it in blocking mode:

const fs = require('fs');

(async () => {
    let read;
    read = fs.createReadStream(null, { fd: 0 })
    for await (const r of read)
        console.log(r)
    console.log('EOF was (normally) reached.')
})();

The difference is accessing process.stdin.constructor.name. Adding a simple console.error doesn't change anything.

[mmomtchev]

Easiest way to check if a file descriptor is in blocking or non-blocking mode is cat /proc/<pid>/fdinfo/<fd> - look at flags, the fourth digit from the right to the left will be 4 for non-blocking mode