Skip to content

Conversation

streamnsight
Copy link
Member

This PR fixes a bug in the langchain integration as ToolMessages are NOT included in the chat history.
This leads to issues with follow-up questions where the ToolMessage content is not available in the history context for the LLM to infer from.

HOWEVER, note that there is another issue with the GenAI inference service that currently would make this bugfix FAIL with Cohere Command R/R+ (cohere.command-r-plus-08-2024) and Command A (cohere.command-a-03-2025) models, in the case where a HumanMessage follows a ToolMessage.
This is a specific use-case when using the @tool(return_direct=True) annotation for the tool to return direct (i.e. without another pass to the LLM to rephrase the content of the tool output). This error is not currently raised because the ToolMessages are being stripped out, so it is never the case.

With this bugfix, and in the conditions above, we get:

oci.exceptions.ServiceError: {'target_service': 'generative_ai_inference', 'status': 400, 'code': '400', 'opc-request-id': '', 'message': '{"message":"invalid request: cannot specify message if the last entry in chat history contains tool results"}

This secondary issue is being worked on by OCI and should be resolved by mid August, so this fix can be safely applied after that date for all use-cases.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Jul 30, 2025
@YouNeedCryDear YouNeedCryDear requested a review from lu-ohai July 31, 2025 06:26
@YouNeedCryDear YouNeedCryDear merged commit 587f785 into oracle:main Aug 1, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants