Add Langfuse diagnostics and GiteeAI plugin

[geffzhang]

User description

Introduced OpenTelemetry-based model diagnostics with Langfuse integration, including new helper classes and activity tracing for agent and function execution. Added BotSharp.Plugin.GiteeAI with chat and embedding providers, and updated solution/project files to register the new plugin. Enhanced tracing in routing, executor, and controller logic for improved observability.

---

PR Type

Enhancement, Documentation

---

Description

• Implemented OpenTelemetry-based model diagnostics with Langfuse integration for improved observability

• Added GiteeAI plugin with chat completion and text embedding providers

• Enhanced tracing in routing, executor, and controller logic for agent and function execution

• Integrated activity tracking with semantic conventions for GenAI operations

---

Diagram Walkthrough

flowchart LR
  A["OpenTelemetry Setup"] --> B["ModelDiagnostics Helper"]
  B --> C["Activity Tracing"]
  C --> D["Chat Providers"]
  C --> E["Function Executors"]
  C --> F["Routing Service"]
  G["GiteeAI Plugin"] --> D
  H["Langfuse Config"] --> A

Loading

File Walkthrough

	Relevant files
Configuration changes	6 files Extensions.cs Configure OpenTelemetry with Langfuse exporter                      +49/-3    BotSharp.Plugin.GiteeAI.csproj Create GiteeAI plugin project file                                              +31/-0    BotSharp.sln Register GiteeAI plugin in solution                                            +11/-0    WebStarter.csproj Add GiteeAI plugin project reference                                          +1/-0      appsettings.json Add Langfuse and GiteeAI model configurations                        +85/-40  Program.cs Comment out MCP service configuration                                        +2/-2
Enhancement	17 files LangfuseSettings.cs Add Langfuse configuration settings model                                +19/-0    ModelDiagnostics.cs Implement model diagnostics with semantic conventions        +394/-0  ActivityExtensions.cs Add activity extension methods for tracing                              +119/-0  AppContextSwitchHelper.cs Helper to read app context switch values                                  +35/-0    FunctionCallbackExecutor.cs Add activity tracing to function execution                              +16/-2    MCPToolExecutor.cs Add activity tracing to MCP tool execution                              +34/-17  RoutingService.InvokeAgent.cs Add agent invocation activity tracing                                        +4/-1      RoutingService.InvokeFunction.cs Import diagnostics for function invocation                              +1/-0      RoutingService.cs Add System.Diagnostics import for tracing                                +1/-0      ConversationController.cs Add activity tracing to conversation endpoints                      +35/-19  ChatCompletionProvider.cs Integrate model diagnostics into chat completion                  +78/-66  ChatCompletionProvider.cs Integrate model diagnostics into chat completion                  +65/-52  GiteeAiPlugin.cs Create GiteeAI plugin with DI registration                              +19/-0    ChatCompletionProvider.cs Implement GiteeAI chat completion provider                              +496/-0  TextEmbeddingProvider.cs Implement GiteeAI text embedding provider                                +73/-0    ProviderHelper.cs Add helper to create GiteeAI client instances                        +16/-0    Using.cs Add global using statements for GiteeAI plugin                      +15/-0
Documentation	1 files README.md Add GiteeAI plugin documentation                                                  +8/-0
Formatting	1 files Program.cs Reorder using statements for clarity                                          +2/-3

---

[qodo-merge-pro]

PR Compliance Guide 🔍

Below is a summary of compliance checks for this PR:

Security Compliance
⚪	Sensitive data in traces Description: Diagnostic switches enable sensitive events which may capture and export user prompts/messages and agent inputs/outputs in traces, risking PII exposure if telemetry backend is not properly secured. Extensions.cs [52-55] Referred Code // Enable model diagnostics with sensitive data. AppContext.SetSwitch("BotSharp.Experimental.GenAI.EnableOTelDiagnostics", true); AppContext.SetSwitch("BotSharp.Experimental.GenAI.EnableOTelDiagnosticsSensitive", true);
	Insecure auth transport Description: Constructs Basic Auth header for OTLP exporter to Langfuse and sends it over HTTP Protobuf; if the host is misconfigured to non-TLS, credentials could be exposed. Extensions.cs [139-151] Referred Code var publicKey = langfuseSection.GetValue<string>(nameof(LangfuseSettings.PublicKey)) ?? string.Empty; var secretKey = langfuseSection.GetValue<string>(nameof(LangfuseSettings.SecretKey)) ?? string.Empty; var host = langfuseSection.GetValue<string>(nameof(LangfuseSettings.Host)) ?? string.Empty; var plainTextBytes = System.Text.Encoding.UTF8.GetBytes($"{publicKey}:{secretKey}"); string base64EncodedAuth = Convert.ToBase64String(plainTextBytes); builder.Services.ConfigureOpenTelemetryTracerProvider(tracing => tracing.AddOtlpExporter(options => { options.Endpoint = new Uri(host); options.Protocol = OtlpExportProtocol.HttpProtobuf; options.Headers = $"Authorization=Basic {base64EncodedAuth}"; }) );
Ticket Compliance
⚪	🎫 No ticket provided - [ ] Create ticket/issue <!-- /create_ticket --create_ticket=true --> </details></td></tr>
Codebase Duplication Compliance
⚪	Codebase context is not defined Follow the guide to enable codebase context checks.
Custom Compliance
⚪	No custom compliance provided Follow the guide to enable custom compliance check.
Update

Compliance status legend

🟢 - Fully Compliant
🟡 - Partial Compliant
🔴 - Not Compliant
⚪ - Requires Further Human Verification
🏷️ - Compliance label

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
High-level	Refactor providers to reduce duplication The ChatCompletionProvider for the new GiteeAI plugin duplicates code from the OpenAI and AzureOpenAI providers. To improve maintainability, refactor this by creating a shared base class for providers with OpenAI-compatible APIs to house the common logic. Examples: src/Plugins/BotSharp.Plugin.GiteeAI/Providers/Chat/ChatCompletionProvider.cs [15-100] public class ChatCompletionProvider( ILogger<ChatCompletionProvider> logger, IServiceProvider services) : IChatCompletion { protected string _model = string.Empty; public virtual string Provider => "gitee-ai"; public string Model => _model; ... (clipped 76 lines) src/Plugins/BotSharp.Plugin.OpenAI/Providers/Chat/ChatCompletionProvider.cs [35-119] public async Task<RoleDialogModel> GetChatCompletions(Agent agent, List<RoleDialogModel> conversations) { var contentHooks = _services.GetHooks<IContentGeneratingHook>(agent.Id); var convService = _services.GetService<IConversationStateService>(); // Before chat completion hook foreach (var hook in contentHooks) { await hook.BeforeGenerating(agent, conversations); } ... (clipped 75 lines) Solution Walkthrough: Before: // In GiteeAI/ChatCompletionProvider.cs public class ChatCompletionProvider : IChatCompletion { public async Task<RoleDialogModel> GetChatCompletions(...) { // ... setup ... var (prompt, messages, options) = PrepareOptions(agent, conversations); using (var activity = ModelDiagnostics.StartCompletionActivity(...)) { // ... call API, process response, set tags ... } } protected (string, ...) PrepareOptions(...) { /* ... complex logic ... */ } } // In OpenAI/ChatCompletionProvider.cs public class ChatCompletionProvider : IChatCompletion { // ... Nearly identical implementation to GiteeAI ... } After: // New Base Class public abstract class OpenAiCompatibleChatCompletionProvider : IChatCompletion { public abstract string Provider { get; } public async Task<RoleDialogModel> GetChatCompletions(Agent agent, List<RoleDialogModel> conversations) { // ... common setup ... var (prompt, messages, options) = PrepareOptions(agent, conversations); using (var activity = ModelDiagnostics.StartCompletionActivity(..., Provider, ...)) { // ... common logic for API call, response processing, tags ... } } protected virtual (string, ...) PrepareOptions(...) { /* ... common complex logic ... */ } } // Refactored GiteeAI provider public class GiteeAiChatCompletionProvider : OpenAiCompatibleChatCompletionProvider { public override string Provider => "gitee-ai"; // ... minimal overrides if needed ... } Suggestion importance[1-10]: 9 __ Why: The suggestion correctly identifies significant code duplication across the new GiteeAI provider and the existing OpenAI and AzureOpenAI providers, which this PR exacerbates, and proposes a valid refactoring that would greatly improve maintainability.	High
Possible issue	Fix incorrect null check Replace the incorrect null check on langfuseSection with langfuseSection.Exists() to correctly determine if the configuration section is present. src/BotSharp.ServiceDefaults/Extensions.cs [129-130] var langfuseSection = builder.Configuration.GetSection("Langfuse"); -var useLangfuse = langfuseSection != null; +var useLangfuse = langfuseSection.Exists(); Apply / Chat Suggestion importance[1-10]: 8 __ Why: The suggestion correctly identifies that GetSection never returns null, fixing a bug where useLangfuse would always be true, which would lead to a UriFormatException if the configuration section is missing.	Medium
	Validate Langfuse configuration values Add validation to ensure publicKey, secretKey, and host from the Langfuse configuration are not empty before attempting to create and use them for authentication. src/BotSharp.ServiceDefaults/Extensions.cs [139-143] var publicKey = langfuseSection.GetValue<string>(nameof(LangfuseSettings.PublicKey)) ?? string.Empty; var secretKey = langfuseSection.GetValue<string>(nameof(LangfuseSettings.SecretKey)) ?? string.Empty; var host = langfuseSection.GetValue<string>(nameof(LangfuseSettings.Host)) ?? string.Empty; + +if (string.IsNullOrEmpty(publicKey) || string.IsNullOrEmpty(secretKey) || string.IsNullOrEmpty(host)) +{ + return builder; +} + var plainTextBytes = System.Text.Encoding.UTF8.GetBytes($"{publicKey}:{secretKey}"); string base64EncodedAuth = Convert.ToBase64String(plainTextBytes); Apply / Chat Suggestion importance[1-10]: 7 __ Why: The suggestion correctly points out that missing configuration values will lead to a UriFormatException and adds necessary validation to prevent this runtime error, improving the code's robustness.	Medium
General	Remove duplicate tag assignment Remove the duplicate line that sets the OutputTokens tag. src/Plugins/BotSharp.Plugin.AzureOpenAI/Providers/Chat/ChatCompletionProvider.cs [128-130] activity?.SetTag(ModelDiagnosticsTags.InputTokens, (tokenUsage?.InputTokenCount ?? 0) - (inputTokenDetails?.CachedTokenCount ?? 0)); activity?.SetTag(ModelDiagnosticsTags.OutputTokens, tokenUsage?.OutputTokenCount ?? 0); -activity?.SetTag(ModelDiagnosticsTags.OutputTokens, tokenUsage?.OutputTokenCount ?? 0); Apply / Chat Suggestion importance[1-10]: 3 __ Why: The suggestion correctly identifies a duplicated line of code that sets the OutputTokens tag twice and proposes removing the redundant line, which cleans up the code.	Low
	Remove unnecessary semicolon Remove the unnecessary semicolon after the foreach loop's closing brace. src/Infrastructure/BotSharp.Abstraction/Diagnostics/ActivityExtensions.cs [43-45] activity.SetTag(tag.Key, tag.Value); } -; return activity; [To ensure code accuracy, apply this suggestion manually] Suggestion importance[1-10]: 2 __ Why: The suggestion correctly identifies and removes a stray semicolon, which is a minor code style and readability improvement.	Low
More

[Oceania2018]

Should this class be moved to the implementation project of BotSharp.Plugin.GiteeAI?

[geffzhang]

this class is otel semantic conventions https://github.com/open-telemetry/semantic-conventions

[geffzhang]

This is the result of integrating OpenTelemetry with Langfuse.
The integration enables real-time tracing and streaming collection of inputs, outputs, execution states, and exceptions for each node in complex workflows, enhancing observability for debugging and performance optimization