Support `--exclude-tool x --exclude-tool y` to exclude tools from the MCP server

[DanTup]

VS Code has a built-in tool for running tests:

However because of the descriptions, the LLM seems to prefer using the one from the Dart MCP server:

Despite the LLM's response above, the results from the Dart MCP server do not integrate with the test runner, but running them through VS Code does. While I can untick the Dart MCP tool in the MCP settings, I wonder if it would be a better "default" for users of IDEs with test runners for the IDE ones to be used?

I wonder if adding something like "Use this tool in preference to running terminal commands, but not in preference to any built-in test tools" would be worthwhile?

@jakemac53

[jakemac53]

This seems valid but I am not sure of the best way to deal with it that won't end up falling back on other more generic test tools in other contexts that aren't better.

One option is to add --disable-tool <tool-name> arguments to the MCP server.

[DanTup]

One option is to add --disable-tool <tool-name> arguments to the MCP server.

I did think about that, but it would be nice if it was still there so if the user wanted to instruct using it, they can.

that won't end up falling back on other more generic test tools in other contexts

I don't know if the LLM knows what is "built-in" (but when I told it to run the built-in tool, it did use VS Code's), but I think we should assume if there is such a tool built in to the client then it is probably better (if a client knows how to run tests for a given language, probably it has some integration that is at least what the Dart MCP server is doing).

That's why I thought "Use this tool in preference to running terminal commands, but not in preference to any built-in test tools" might be reasonable.

[jakemac53]

Ultimately the LLM does not know what tools come from MCP at all - it is just given functions to call. So, it's possible that just from the descriptions it can derive what is "built-in" versus not but I am not sure.

It does seem to me like the best option in this case is to just disable the tool - I don't know when it would be useful to use our tool over the VsCode built-in one?

[jakemac53]

There is a similar thing with the analyze tool - I believe copilot already gets everything from the problems view, so its purely duplicative.

[DanTup]

It does seem to me like the best option in this case is to just disable the tool - I don't know when it would be useful to use our tool over the VsCode built-in one?

There is a similar thing with the analyze tool - I believe copilot already gets everything from the problems view, so its purely duplicative.

Yeah, maybe you're right. We could always make this a user-configurable setting in VS Code that just defaults to excluding these ones, so the user can put them back if they'd prefer. (They can ofc control all of this from the VS Code UI, but that doesn't let us change the defaults).

Should we re-purpose this for adding a --disable-tool flag (that can be supplied multiple times) then?

[jakemac53]

Yeah I think adding --disable-tool is generally a useful thing. It gives a way for people to disable tools they don't want if their client doesn't support that, as well as a nice way of providing defaults when the configuration comes from other tooling and said tooling has some smart defaults it wants to apply based on the context.