- 
                Notifications
    
You must be signed in to change notification settings  - Fork 15
 
Closed
Labels
enhancementNew feature or requestNew feature or request
Milestone
Description
When hard killing restate and then recovering, it tries to recover previously undelivered invocations (I think), and it causes our SDK to log a lot ERRORs and WARNs like the snippet below.
The error message is not very informative, and most likely to appear every time restate is re-started.
Since that is a common situation, can we improve that log message?
[restate] [2023-05-20T15:23:34.073Z] WARN: Error in input stream: Error: State machine is closed. Canceling all execution
    at DurableExecutionStateMachine.failIfClosed (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:606:19)
    at DurableExecutionStateMachine.checkIfInReplay (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:594:14)
    at DurableExecutionStateMachine.handleSetStateMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:561:14)
    at DurableExecutionStateMachine.onIncomingMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:485:22)
    at Transform.<anonymous> (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/connection/restate_duplex_stream.js:26:13)
    at Transform.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Transform.transform [as _transform] (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/io/decoder.js:64:38)
[restate] [2023-05-20T15:23:34.074Z] WARN: Error in input stream: Error: State machine is closed. Canceling all execution
    at DurableExecutionStateMachine.failIfClosed (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:606:19)
    at DurableExecutionStateMachine.checkIfInReplay (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:594:14)
    at DurableExecutionStateMachine.handleSetStateMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:561:14)
    at DurableExecutionStateMachine.onIncomingMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:485:22)
    at Transform.<anonymous> (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/connection/restate_duplex_stream.js:26:13)
    at Transform.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Transform.transform [as _transform] (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/io/decoder.js:64:38)
[restate] [2023-05-20T15:23:34.124Z] WARN: Error in input stream: Error: State machine is closed. Canceling all execution
    at DurableExecutionStateMachine.failIfClosed (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:606:19)
    at DurableExecutionStateMachine.checkIfInReplay (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:594:14)
    at DurableExecutionStateMachine.handleSetStateMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:561:14)
    at DurableExecutionStateMachine.onIncomingMessage (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/state_machine.js:485:22)
    at Transform.<anonymous> (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/connection/restate_duplex_stream.js:26:13)
    at Transform.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:297:9)
    at Readable.push (node:internal/streams/readable:234:10)
    at Transform.transform [as _transform] (/home/igal/work/jepsen.restate/service/node_modules/@restatedev/restate-sdk/dist/io/decoder.js:64:38)
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request