Incorrect Error handling when calling MCP tools

[Ga-Ol-St]

Codebase: M7
Issue Description:
When we execute MCP Tools it's normal that tool can report errors, for example:
MCP Tool is handling renaming files, we fire this tool and trying to rename a file which don't exist, Tool will report an error.

Current state:
When LLM execute some MCP tool and MCP tool reports error - this will terminate execution flow with following error:

java.lang.IllegalStateException: Error calling tool: [TextContent[audience=null, priority=null, text=file not found]]
at org.springframework.ai.mcp.SyncMcpToolCallback.call(SyncMcpToolCallback.java:117) ~[spring-ai-mcp-1.0.0-M7.jar:1.0.0-M7]
at org.springframework.ai.mcp.SyncMcpToolCallback.call(SyncMcpToolCallback.java:125) ~[spring-ai-mcp-1.0.0-M7.jar:1.0.0-M7]
at org.springframework.ai.model.tool.DefaultToolCallingManager.executeToolCall(DefaultToolCallingManager.java:227) ~[spring-ai-model-1.0.0-M7.jar:1.0.0-M7]
at org.springframework.ai.model.tool.DefaultToolCallingManager.executeToolCalls(DefaultToolCallingManager.java:139) ~[spring-ai-model-1.0.0-M7.jar:1.0.0-M7]
at com.example.demo.MyController.processRequest(MyController.java:234) ~[main/:na]

Expected Behavior:
Error should be processed by ToolExecutionExceptionProcessor and result should be returned to LLM

Issue analysis:
DefaultToolCallingManager should catch this kind of errors but in current code state if MCP Tool calls produce an error they will throw IllegalStateException but DefaultToolCallingManager is expecting ToolExecutionException in case of error in Tool execution.

Reference points:

spring-ai/spring-ai-model/src/main/java/org/springframework/ai/model/tool/DefaultToolCallingManager.java

Lines 204 to 210 in 81b715b

	String toolResult;
	try {
	toolResult = toolCallback.call(toolInputArguments, toolContext);
	}
	catch (ToolExecutionException ex) {
	toolResult = this.toolExecutionExceptionProcessor.process(ex);
	}

spring-ai/mcp/common/src/main/java/org/springframework/ai/mcp/SyncMcpToolCallback.java

Lines 111 to 120 in 81b715b

	public String call(String functionInput) {
	Map<String, Object> arguments = ModelOptionsUtils.jsonToMap(functionInput);
	// Note that we use the original tool name here, not the adapted one from
	// getToolDefinition
	CallToolResult response = this.mcpClient.callTool(new CallToolRequest(this.tool.name(), arguments));
	if (response.isError() != null && response.isError()) {
	throw new IllegalStateException("Error calling tool: " + response.content());
	}
	return ModelOptionsUtils.toJsonString(response.content());
	}

spring-ai/mcp/common/src/main/java/org/springframework/ai/mcp/AsyncMcpToolCallback.java

Lines 107 to 117 in 81b715b

	public String call(String functionInput) {
	Map<String, Object> arguments = ModelOptionsUtils.jsonToMap(functionInput);
	// Note that we use the original tool name here, not the adapted one from
	// getToolDefinition
	return this.asyncMcpClient.callTool(new CallToolRequest(this.tool.name(), arguments)).map(response -> {
	if (response.isError() != null && response.isError()) {
	throw new IllegalStateException("Error calling tool: " + response.content());
	}
	return ModelOptionsUtils.toJsonString(response.content());
	}).block();
	}

[zikos2]

Same problem, an IllegalStateException is thrown and the program execution is terminated.

[down-to-earth1994]

Temporary Solution

1. Customize CustomSyncMcpToolCallback and CustomSyncMcpToolCallbackProvider to override the call and getToolCallbacks methods.
2. Use CustomSyncMcpToolCallbackProvider to provide tools when building the chatModel.

package com.pig4cloud.pig.mcp.client.chat;

import cn.hutool.json.JSONUtil;
import io.modelcontextprotocol.client.McpSyncClient;
import io.modelcontextprotocol.spec.McpSchema;
import org.springframework.ai.mcp.SyncMcpToolCallback;
import org.springframework.ai.model.ModelOptionsUtils;
import org.springframework.ai.tool.definition.DefaultToolDefinition;
import org.springframework.ai.tool.execution.ToolExecutionException;

import java.util.Map;
import lombok.extern.slf4j.Slf4j;

@Slf4j
public class CustomSyncMcpToolCallback extends SyncMcpToolCallback {
    private final McpSyncClient mcpClient;

    private final McpSchema.Tool tool;
    /**
     * Creates a new {@code SyncMcpToolCallback} instance.
     *
     * @param mcpClient the MCP client to use for tool execution
     * @param tool      the MCP tool definition to adapt
     */
    public CustomSyncMcpToolCallback(McpSyncClient mcpClient, McpSchema.Tool tool) {
        super(mcpClient, tool);
        this.mcpClient = mcpClient;
        this.tool = tool;
    }

    public String call(String functionInput) {
        Map<String, Object> arguments = ModelOptionsUtils.jsonToMap(functionInput);
        // Note that we use the original tool name here, not the adapted one from
        // getToolDefinition
        McpSchema.CallToolResult response = this.mcpClient.callTool(new McpSchema.CallToolRequest(this.tool.name(), arguments));
        if (response.isError() != null && response.isError()) {
            log.warn("tools exec response error: {}", response);
            throw new ToolExecutionException(new DefaultToolDefinition(tool.name(),tool.description(), JSONUtil.toJsonStr(tool.inputSchema())),new IllegalStateException("Error calling tool: " + response.content()));
        }
        return ModelOptionsUtils.toJsonString(response.content());
    }


}

package com.pig4cloud.pig.mcp.client.chat;

import io.modelcontextprotocol.client.McpSyncClient;
import io.modelcontextprotocol.spec.McpSchema;
import org.springframework.ai.mcp.SyncMcpToolCallbackProvider;
import org.springframework.ai.tool.ToolCallback;
import org.springframework.ai.tool.util.ToolUtils;
import org.springframework.util.Assert;

import java.util.ArrayList;
import java.util.List;
import java.util.function.BiPredicate;

public class CustomSyncMcpToolCallbackProvider extends SyncMcpToolCallbackProvider {

    private final List<McpSyncClient> mcpClients;

    private final BiPredicate<McpSyncClient, McpSchema.Tool> toolFilter;

    /**
     * Constructor for CustomSyncMcpToolCallbackProvider with tool filter.
     *
     * @param toolFilter The filter to apply to tools.
     * @param mcpClients The list of MCP clients.
     */
    public CustomSyncMcpToolCallbackProvider(BiPredicate<McpSyncClient, McpSchema.Tool> toolFilter, List<McpSyncClient> mcpClients) {
        Assert.notNull(mcpClients, "MCP clients must not be null");
        Assert.notNull(toolFilter, "Tool filter must not be null");
        this.mcpClients = mcpClients;
        this.toolFilter = toolFilter;
    }

    /**
     * Constructor for CustomSyncMcpToolCallbackProvider without tool filter.
     *
     * @param mcpClients The list of MCP clients.
     */
    public CustomSyncMcpToolCallbackProvider(List<McpSyncClient> mcpClients) {
        this((mcpClient, tool) -> true, mcpClients);
    }

    /**
     * Get the tool callbacks.
     *
     * @return An array of ToolCallback objects.
     */
    @Override
    public ToolCallback[] getToolCallbacks() {

        var toolCallbacks = new ArrayList<>();

        this.mcpClients.stream().forEach(mcpClient -> {
            toolCallbacks.addAll(mcpClient.listTools()
                    .tools()
                    .stream()
                    .filter(tool -> toolFilter.test(mcpClient, tool))
                    .map(tool -> new CustomSyncMcpToolCallback(mcpClient, tool))
                    .toList());
        });
        var array = toolCallbacks.toArray(new ToolCallback[0]);
        validateToolCallbacks(array);
        return array;
    }

    /**
     * Validate the tool callbacks to ensure there are no duplicate tool names.
     *
     * @param toolCallbacks An array of ToolCallback objects.
     * @throws IllegalStateException if there are duplicate tool names.
     */
    private void validateToolCallbacks(ToolCallback[] toolCallbacks) {
        List<String> duplicateToolNames = ToolUtils.getDuplicateToolNames(toolCallbacks);
        if (!duplicateToolNames.isEmpty()) {
            throw new IllegalStateException(
                    "Multiple tools with the same name (%s)".formatted(String.join(", ", duplicateToolNames)));
        }
    }
}

[down-to-earth1994]

Validation Result：

[zzhujing]

i think then Async/SyncMcpToolCallback#call should return ToolExecutionException not IllegalStateException , Some mcp server tools calls fail. The error information should be returned to the AI model to read the exception information and then try to solve the exception

[ls-rein-martha]

Another possible solution is to make ToolExecutionExceptionProcessor able to process all Exceptions instead of only ToolExecutionException, allowing it to be fine-grained further. Thus, any other edge case exception can be gracefully handled as well. LLM doesn't really need the toolDefinition again anyway, since it already has it.

[emaarco]

I'm dealing with the same problem. In my case, I’m using the FunctionToolCallback. This callback is also performed by the DefaultToolCallingManager as described by @Ga-Ol-St.

Performing the apply-function, it executes a specific tool that was registered before. However, imagining that the tool can throw errors (no matter whether they are thrown explicitly or implicitly), they are not handled by the ExceptionProcessor as well.

	@Override
	public String call(String toolInput, @Nullable ToolContext toolContext) {
	    Assert.hasText(toolInput, "toolInput cannot be null or empty");

	    logger.debug("Starting execution of tool: {}", this.toolDefinition.name());

	    I request = JsonParser.fromJson(toolInput, this.toolInputType);
	    O response = this.toolFunction.apply(request, toolContext);

	    logger.debug("Successful execution of tool: {}", this.toolDefinition.name());

	    return this.toolCallResultConverter.convert(response, null);
	}

@Bean
fun exampleTool(): FunctionToolCallback<Void, List<String>> {
    return FunctionToolCallback
        .builder("ExampleTool", exampleTool::performCall)
        .description("This is an example tool that returns a list of example strings.")
        .build()
}

class ExampleTool {
    fun performCall(): List<String> {
        val response = client.execute("query { exampleField }")
        if (response.hasErrors()) {
            throw RuntimeException("Error fetching example data: $response")
        } else {
            val data = response.getData("exampleField")
            return data as List<String>
        }
    }
}

Therefore, I also thought about how to solve this. First, I thought about throwing the ToolExecutionException myself. But this would be quite hacky, because within the ExampleTool function, I don’t have access to the associated toolDefinition. Therefore, I can’t satisfy the constructor without mocking a definition or passing it lazily.

public class ToolExecutionException extends RuntimeException {
    // ...
    public ToolExecutionException(ToolDefinition toolDefinition, Throwable cause) {
        super(cause.getMessage(), cause);
        this.toolDefinition = toolDefinition;
    }
}

Another option, in my opinion, could be to wrap the function-call of FunctionToolCallback in a try-catch and throw it .
This would be equivalent to the implementation in SyncMcpToolCallback.

And since this is the same class as mentioned by @Ga-Ol-St, I’m wondering if his initial issue is (no matter the workaround) still active or solved by now (maybe you can provide some feedback on this? 😄 ).

try {
    response = this.toolFunction.apply(request, toolContext);
} catch (Exception e) {
    logger.error("Error executing tool: {}", this.toolDefinition.name(), e);
    throw new ToolExecutionException(this.toolDefinition, e);
}

If it’s not solved, then I would also support the idea of @ls-rein-martha to make the errorProcessor able to process all kinds of exceptions. Despite that, I'm happy for any feedback: Any other ideas, or things I could make better in my case (I’m happy to offer more details)?

No matter the preferred solution, I would offer to contribute for this 😄 👋