Remove omitempty tag from content field in ChatCompletionStreamChoice… #1058
+1
−1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe the change
This PR removes the omitempty json tag from Content field in ChatCompletionStreamChoiceDelta for chat/completions streaming response. This is needed cause SDK which consume this response expects content field to be in the response always even when its empty/null and if this field is not present these client SDKs break. For streaming responses from VLLM the first stream chunk is:
So in delta only role is filled and content is empty, when this is unmarhsalled by the go-openai client it omits the content field and returns where content is missing:
to resolve this issue have removed omitempty tag
Provide OpenAI documentation link
https://platform.openai.com/docs/api-reference/chat_streaming/streaming
https://github.com/openai/openai-go/blob/3fb5d8d68e5e7a74fa5d666550904d26d3f34bc8/chatcompletion.go#L661
https://github.com/openai/openai-python/blob/a3315d9fcc17d7583603476f088929fb2b9e71ca/src/openai/types/chat/chat_completion_chunk.py#L60
In the official OpenAI API the content field is nullable but not omitted from the response. Also tested the official python client they do not omit the content field.
Describe your solution
Removed the omitempty json tag from Content field in ChatCompletionStreamChoiceDelta for chat/completions streaming response
Tests
After updating the go-client ran chat completions streaming response and got desired output:
Additional context
Add any other context or screenshots or logs about your pull request here. If the pull request relates to an open issue, please link to it.
Issue: #XXXX