langchain_oci bugfix: include ToolMessages in chat history #8
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes a bug in the langchain integration as ToolMessages are NOT included in the chat history.
This leads to issues with follow-up questions where the ToolMessage content is not available in the history context for the LLM to infer from.
HOWEVER, note that there is another issue with the GenAI inference service that currently would make this bugfix FAIL with Cohere Command R/R+ (
cohere.command-r-plus-08-2024
) and Command A (cohere.command-a-03-2025
) models, in the case where a HumanMessage follows a ToolMessage.This is a specific use-case when using the
@tool(return_direct=True)
annotation for the tool to return direct (i.e. without another pass to the LLM to rephrase the content of the tool output). This error is not currently raised because the ToolMessages are being stripped out, so it is never the case.With this bugfix, and in the conditions above, we get:
This secondary issue is being worked on by OCI and should be resolved by mid August, so this fix can be safely applied after that date for all use-cases.