WebUI - Mcp SSE server support in client

[brucepro]

This is intended to be a discussion on adding MCP SSE Client support to the current WebUI. I have not started migrating the demo included into the WebUI, but the demo is written as a React app to enable ease of migration. I wanted to discuss the path forward first before continuing.
I have a working demo of the MCP support for Tools, Resources and Prompts. This is not merged into the WebUI yet as I wanted to have a discussion on how the WebUI should handle the specific MCP items.
For context: https://modelcontextprotocol.io/introduction
Some Examples of MCP Servers: https://glama.ai/mcp/servers
Since we are using a Web Interface SSE support is used. There are a few Stdio to SSE proxies that can allow the vast majority of MCP servers to run. I have included two basic SSE python MCP servers in the demo code.

These features will work if tools support is enabled with the --jinja flag and a model that supports function use. Potentially adding a check at /props to verify support might be good.

Tool Use:
Other clients have varying UI's when it comes to tool use. Deciding to allow tool use by the llm without user interaction is one option, having a pop-up to verify the tool should run is another. We could potential have a checkbox in config that turns user prompting for tools on or off. I have also experimented with having tools available to the user for running manually.

Prompts:
I haven't tested to many MCP Servers with prompts, but user interaction is usually needed to add the various fields.

Resources:
I am currently adding the resources selected by the user to be included in the context.

From a UI perspective, I was thinking of adding the tools, prompts and resources config box next to the send button.

Something to also consider. MCP Servers are varied and some of them provide some pretty powerful tools to the agent. Does this entire feature set need to be migrated to it's own WebUI client instead of tacking the features on top of the existing.

Thoughts?

[ngxson]

Sorry I didn't have much time to look into this.

AFAIU in this PR, you're implementing it on top of the old vuejs frontend. This can be quite messy.

I'm wondering, can you make another demo with less code, maybe just start with a simple HTML, minimal JS/CSS, no vitejs or npm is needed. Once the simple demo works, I'll see how the current reactjs code base can be adapted

[brucepro]

This PR was mostly just about questions. I have a working demo using React that I built out once the vue was replaced with react. https://github.com/brucepro/llamacppMCPClientDemo

I should of done a better job of a better pr. I don't mind migrating the basic react code to the webui, just wanted the guidance on ui before I do it.

[ngxson]

I had a quick look on your demo https://github.com/brucepro/llamacppMCPClientDemo , it looks nice overall (in term of minimalist). This is a good point to start planning UI structure and most importantly, how hooks can be organized.

Indeed, what I'm worry about the most is how to organize hooks. This is a powerful thing in react but often time, it makes the code become spaghetti if used incorrectly.

I'll have a deeper look later, will open an issue on your repo if I need to discuss more.

[brucepro]

Sounds good. The mcpsse class is the main part. Rest just implements the code to work with the completion server.
A big thing for tool use is not having streaming for text turned on and the way it changes the prompt flow by responding with role function. Looking forward to getting it into webui.

[brucepro]

So far I have the WebUI code imported into the repo and am working on features.
Completed:

• Import MCPSSE Class that supports multiple SSE servers (Need to add some code to fix tool names being the same across multiple servers, thinking of appending tool name with the server name in the available tools list)
• UI element in settings dialog in experimental to turn on MCP Servers
• When MCP Servers is enabled, MCP Toolbox appears next to the send button.
• MCP Config Dialog handles adding and deleting Servers. (Still need to add the tools,prompts, resources available for each connected server here to allow the User to interact with Prompts and Resources and Tools manually.)
  Todo:
  -Finish up the tool call code. Including the UI approve tool call dialog. Wanted to get all the UI elements in place before finishing up the tool_call loop.
• Add UI element in MCP Config Dialog to turn on/off Prompting of tool use. That way tools can be called with or without user interaction.
• Add dialog screen to approve tool use into tool call loop.
• Add resource handling code to add resources to additional context when selected.
• Create workflow dialog for Prompt workflows. Most prompts seem to have input requirements. Once I install a few non-testing MCP Servers will get the workflow laid out.

Would it be better to continue working in the https://github.com/brucepro/llamacppMCPClientDemo, or go ahead and migrate it to an actual PR?