Fix handling of multiple tool calls in single LLM response #241
Conversation
Previously, the library only processed the first tool call when an LLM returned multiple tool_use blocks in a single response. This limitation prevented parallel function execution, a key feature of modern LLMs. Changes: - Update Anthropic provider to extract ALL tool_use blocks from responses - Modify tool parser to handle arrays of content blocks - Ensure all tools execute before continuing the conversation - Add comprehensive tests for multi-tool scenarios across all providers This enables use cases like: - Executing multiple independent operations in parallel - Rolling dice multiple times in one request - Fetching data from multiple sources simultaneously The implementation maintains backward compatibility while extending support for advanced parallel tool calling capabilities. Fixes the limitation where only the first tool call was processed when multiple were requested.
|
One note here is that I couldn't get bedrock credentials to work, try as I might. So the bedrock VCR is faked for the multi tool calls. Every other VCR is real. |
tpaulshippy
left a comment
tpaulshippy
left a comment
There was a problem hiding this comment.
I just tested this against Bedrock. Looks good.
chat_function_calling_bedrock_anthropic_claude-3-5-haiku-20241022-v1_0_can_handle_multiple_tool_calls_in_a_single_response.yml.zip |
|
Thank you @tpaulshippy I added the real cassette. |
|
Merging this into my fork to see if it helps with intermittent "text content blocks must be non-empty" errors. |
|
@finbarr Do you have time to address the feedback? If not, I can start a separate PR. |
|
@tpaulshippy feel free to go ahead. Thank you! |
|
@tpaulshippy I made a change. Feel free to modify further as needed. |
- Updated test skip logic to check if provider is local using provider.local? method - Consolidated separate skip statements for Ollama and GPUStack into single check - Both providers have local? => true and share similar limitations with tool usage
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #241 +/- ##
==========================================
+ Coverage 87.67% 87.72% +0.04%
==========================================
Files 78 78
Lines 3001 3005 +4
Branches 564 567 +3
==========================================
+ Hits 2631 2636 +5
+ Misses 370 369 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@finbarr great PR, thank you! Merging |
3 participants
Summary
Context
This PR addresses issue #236 where the Anthropic provider was only processing the first tool call when multiple tool calls were returned in a single LLM response.
Changes
parse_tool_callsmethod to handle arrays of tool blocksformat_tool_callto iterate through all tool callsselectinstead offindto get all tool_use blocksTest plan
🤖 Generated with Claude Code