Comprehensive overview for difficulties with security analyzer

[malhotra5]

This is a comprehensive issue which describes the current architecture specifically for the security analyzer. It also describes the difficulties client applications of the agent-sdk face when attempting to utilize this feature. The complexity can be broken into three categories as follows

(UNSOLVED): Security analyzer <> security policy relation

The security analyzer and security policy are configured independently yet closely related. The idea being that we introduce new fine-grained client defined policies when using the same analyzer. However, currently it adds extra overhead for clients to maintain a reasonable mapping

1. Always confirm <> No analyzer required (all actions by default require user confirmation)
2. Never confirm <> No analyzer required (all actions by default require no user confirmation)
3. Risk based confirmation <> LLM analyzer (LLM determines high risk actions which require user confirmation)

This is the expected set of mapping between security policies and analyzers at the moment. Unfortunately, clients can choose other combinations such as Never Confirm <> LLM Analyzer. This scenario is especially likely to occur when switching analyzers. In the V1 CLI, the user may enable the security analyzer mid conversation. This requires the client to enable the LLM security analyzer. If the client forgets to update the security policy as well, then confirmation mode will not work as expected.

Related PRs:

• Security analyzer framework #305

(SOLVED): Reconciliation with persistent state

This related to the difficulty of toggling the security analyzer. We used to have logic that is triggered when the agent is initialized. Once initialized, updating the security analyzer would fail to update other parts of the agent (tools, prompts, etc). The workaround was to save everything to persistent state, restart the conversation and re-initialize the agent to trigger the same entry points.

Related issue:

• Cannot easily remove security analyzer from agent mid conversation #668: Not easy to remove security analyzer mid conversation

Related PRs:

• Allow security_analyzer to differ during agent reconciliation #669: Remove reconciliation requirements for security analyzer

(UNSOLVED): Immutability and mid conversation switch

The security_risk risk parameter is dynamically added to Tools depending on whether the LLM security analyzer is active. The tool descriptions are passed to the agent's system prompt, which is how the agent is aware of this requirement. The problem is that the agent history/system prompt is immutable. This leads to complications when toggling the security analyzer. We have three scenarios

Scenario 1:

• Security analyzer exists when starting conversation
• System prompt includes tools with security_risk
• Security analyzer is removed mid conversation
• No risk field is expected, but LLM still believes it needs to pass this field because system prompt was not updated. Passing field will crash the conversation

Scenario 2:

• Security analyzer was not included at start of conversation
• System prompt does not include information on passing security_risk fields
• Security analyzer is added later in the conversation
• security_risk is a required field, but LLM is unaware of this requirement since system prompt did not include this information

Scenario 3:

• Similar to 1, but:
• The security analyzer is removed mid conversation
• The conversation is fully reloaded
• ActionEvents are created in history, and they have the old security_risk field
• Validation fails, and leads to an infinite loop

"arguments": "{\"command\": \"ls -al ~/.openhands\", \"security_risk\": \"LOW\"}

Related PRs:

• Runtime errors with security risk and no configured security analyzer #723: ignore security_risk when analyzer is not configured (solves scenario 1)
• Fix security analyzer #435: don't include security_risk information in system prompt when security analyzer is not active
• Do not add security_risk unless security analyzer is enabled #341: don't include security_risk fields is analyzer is not enabled

Related Issues:

• Slack report o3 infinite loop when attempting to continue a conversation with tool calls with security_risk
• CLI crashes with FunctionCallValidationError: Missing required parameter 'security_risk' for execute_bash OpenHands#11378
• [Bug]: CLI V1 crashes with 'LLM provided a security_risk but no security analyzer is configured' OpenHands#11309
• CLI(V1): DeepSeek LLM fails tool calling due to missing security_risk parameter in V1 SDK OpenHands#11256