-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Description
Issue Title: [Backend] Add {"done": true} identifier to SSE response stream
Description
Problem:
Currently, the frontend can only determine the end of a Server-Sent Events (SSE) response stream from the backend LLM by detecting the disconnection of the interface. This method is unreliable and not semantically clear. It makes frontend logic more complex and error-prone, as it must handle connection drops for both completion and errors in the same way.
Proposed Solution:
Modify the backend SSE response to include a final, explicit termination signal. Before closing the stream, the backend should send a final data message containing a JSON object: {"done": true}.
This allows the frontend to:
- Reliably detect successful completion: The frontend can now wait for the
{"done": true}message to know the stream has ended successfully. - Distinguish between completion and error: A connection drop without receiving the
donesignal can now be confidently treated as a network error or backend failure. - Simplify frontend logic: The event handling code becomes cleaner and more intentional.
Example of new stream flow:
data: {"text": "This is the first chunk..."}
data: {"text": "This is the second chunk..."}
data: {"done": true}
Changes Required:
- Update the backend LLM response stream logic to append the
{"done": true}message as the final event. - Ensure the change is compatible with the existing frontend code that listens for
{"text": "..."}messages.
Acceptance Criteria:
- The SSE stream from the LLM ends with a
{"done": true}message. - The existing
{"text": "..."}messages remain unchanged and are not affected. - The frontend team can use this signal to accurately determine the end of a successful stream.
This change is crucial for building a robust and predictable user experience, especially in scenarios with unstable network conditions.