ToolMessages in chat history - discrepancies between Command R/R+ and Command A

[streamnsight]

For context, let's state the current situation:

1. There are 2 Cohere APIs: v1 and v2. At this time (08-2025), the OCI SDK, leveraged by this integration, only uses V1. The langchain_cohere integration uses v2.

2. The integration strips ToolMessages from chat history. This, to me, is a bug: ToolMessage content usually contains useful context that may be needed for follow-up questions.

3. Command R and R+ models do not require a tool_result following a tool_call. It does not error on that, so when ToolMessages are stripped away (as it is now), the API doesn't complain.

4. Command A, however, now requires a tool_result matching a tool_call when a tool_call is present in history, so tool calls fails with Command A because ToolMessages are stripped from history. This fails with the error:

   "message":"invalid request: An assistant message with \'tool_calls\' must be followed by tool messages responding 
   to each \'tool_call_id\'.     The following tool_call_ids did not have response messages: "
   

   -> To make this integration work with Command A tool calls, we need to pass the ToolMessages in history.

5. When adding the ToolMessages in history, for Command R, R+ and A, the integration works when the history is like:

   [
     HumanMessage("give me the list of users that match X"),
     AIMessage("I will use the tool 'list_users' with parameters 'criteria' = X"),
     ToolMessage("I found the following users: \n- John Does (id 123) \n- Jane Doe (id 345)")
   ]
   

   where the AI is now interpreting the ToolMessage and providing a response (i.e. no return_direct flag)

Now,

6. LangChain has a feature for tools which is to return directly (i.e. without an extra LLM call to interpret the tool_result). This is activated by providing the @tool decorator with the parameter @tool(return_direct=True).
   In that case, and in the case of a chat conversation, we then typically have a history that looks like:

   [
     HumanMessage("give me the list of users that match X"),
     AIMessage("I will use the tool 'list_users' with parameters 'criteria' = X"),
     ToolMessage("I found the following users: \n- John Does (id 123) \n- Jane Doe (id 345)"),   # <--- ToolMessage returned directly
     HumanMessage("give me the contact info for John Doe"),  # <-- follow up HumanMessage
   ]
   

   In this case, if the ToolMessage had been removed, the follow up question from the human could not be answered. This is why I say it's a bug, and this is why Command A requires those.

7. However, when the message following the ToolMessage is a HumanMessage, as in 6) we get this error (for all models):

   "message":"invalid request: cannot specify message if the last entry in chat history contains tool results"
   

   This error / validation is present in Cohere API v1, but not v2 (The langchain_cohere integration uses v2, and Cohere confirmed that this validation is not present in v2)

8. So, to make the use-case of a @tool(return_direct=True) work, we need to hack this. The solution is to insert a dummy AIMessage before our HumanMessage. We can detect if a HumanMessage follows a ToolMessage and insert an empty message " ".
   This fixes the issue with the Command A model, but with Command R/R+, it raises the following error:
   "message":"No valid tool call or response generated"

9. Cohere lets me know that whether v1 or v2, for Command R/R+ model, its API strips the tool_results messages before the last human message (to preserve the more limited context length of those models and not overflow them with older context). That is not the case for Command A.
   So, essentially adding the ToolMessages in the history in this integration is useless for Command R/R+ because those messages get stripped downstream anyway by the Cohere API, but this is still required for Command A.

10. Without this context, the empty " " message throughs Command R/R+ out because it thinks the prompt is malformed.
    So, the solution, to preserve the context of the ToolMessage for this use-case, would be to repeat the ToolMessage content as an AIMessage, so as to circumvent the issue of the HumanMessage not being allowed after a ToolMessage, AND preserving the ToolMessage context for the follow up call.

[streamnsight]

Solved with the PR#11