Skip to content

Remove omitempty tag from content field in ChatCompletionStreamChoice… #1058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

AyushSawant18588
Copy link
Contributor

@AyushSawant18588 AyushSawant18588 commented Aug 5, 2025

Describe the change
This PR removes the omitempty json tag from Content field in ChatCompletionStreamChoiceDelta for chat/completions streaming response. This is needed cause SDK which consume this response expects content field to be in the response always even when its empty/null and if this field is not present these client SDKs break. For streaming responses from VLLM the first stream chunk is:

data: {"id":"chatcmpl-25f9188232674f0bbcc81efa9cb73fc0","object":"chat.completion.chunk","created":1752484006,"model":"llama3-2","choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

So in delta only role is filled and content is empty, when this is unmarhsalled by the go-openai client it omits the content field and returns where content is missing:

data: {"id":"chatcmpl-d11a2823d2af48e09cf61b8584c5003d","object":"chat.completion.chunk","created":1752483769,"model":"llama3-2","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null,"content_filter_results":{"hate":{"filtered":false},"self_harm":{"filtered":false},"sexual":{"filtered":false},"violence":{"filtered":false},"jailbreak":{"filtered":false,"detected":false},"profanity":{"filtered":false,"detected":false}}}],"system_fingerprint":""}

to resolve this issue have removed omitempty tag

Provide OpenAI documentation link
https://platform.openai.com/docs/api-reference/chat_streaming/streaming
https://github.com/openai/openai-go/blob/3fb5d8d68e5e7a74fa5d666550904d26d3f34bc8/chatcompletion.go#L661
https://github.com/openai/openai-python/blob/a3315d9fcc17d7583603476f088929fb2b9e71ca/src/openai/types/chat/chat_completion_chunk.py#L60
In the official OpenAI API the content field is nullable but not omitted from the response. Also tested the official python client they do not omit the content field.

ChoiceDelta(content='', function_call=None, refusal=None, role='assistant', tool_calls=None)

Describe your solution
Removed the omitempty json tag from Content field in ChatCompletionStreamChoiceDelta for chat/completions streaming response

Tests
After updating the go-client ran chat completions streaming response and got desired output:

data: {"id":"chatcmpl-f6c03344ce134aaca4e87670c688aa74","object":"chat.completion.chunk","created":1754373814,"model":"llama3-3b","choices":[{"index":0,"delta":{"content":"","role":"assistant"},"finish_reason":null,"content_filter_results":{"hate":{"filtered":false},"self_harm":{"filtered":false},"sexual":{"filtered":false},"violence":{"filtered":false},"jailbreak":{"filtered":false,"detected":false},"profanity":{"filtered":false,"detected":false}}}],"system_fingerprint":""}

Additional context
Add any other context or screenshots or logs about your pull request here. If the pull request relates to an open issue, please link to it.

Issue: #XXXX

Copy link

codecov bot commented Aug 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.59%. Comparing base (c4273cb) to head (88aa732).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1058   +/-   ##
=======================================
  Coverage   99.59%   99.59%           
=======================================
  Files          34       34           
  Lines        2205     2205           
=======================================
  Hits         2196     2196           
  Misses          6        6           
  Partials        3        3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sashabaranov sashabaranov requested a review from Copilot August 7, 2025 14:59
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes the omitempty JSON tag from the Content field in ChatCompletionStreamChoiceDelta to ensure compatibility with OpenAI client SDKs that expect the content field to always be present in streaming responses, even when empty.

  • Removes omitempty tag from the Content field in the streaming response structure
  • Ensures the content field is always included in JSON serialization, matching OpenAI API behavior
  • Fixes compatibility issues with client SDKs that break when the content field is missing

@@ -6,7 +6,7 @@ import (
)

type ChatCompletionStreamChoiceDelta struct {
Content string `json:"content,omitempty"`
Content string `json:"content"`
Copy link
Preview

Copilot AI Aug 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a pointer type *string for the Content field to properly distinguish between empty string and null values, which would better match the OpenAI API specification that describes content as nullable.

Suggested change
Content string `json:"content"`
Content *string `json:"content"`

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants