Cannot easily remove security analyzer from agent mid conversation

[malhotra5]

This issue is related to OpenHands/OpenHands#11256

We want the ability to remove the security analyzer from the agent because it adds a system prompt which requires the LLM model to return field for risk, something that weaker llm models struggle to do.

There are two ways to implement this

Method 1: update conversation's agent

conversation.agent = conversation.agent.model_copy(
                update={"security_analyzer": None}
)

The problem here is updating the security analyzer as so doesn't update the system prompt causing the error

RuntimeError: LLM provided a security_risk but no security analyzer is configured - THIS SHOULD NOT HAPPEN!

Method 2 Reload conversation from persistent state with a new agent that doesn't contain security analyzer

conversation = Conversation(convo_id='resuming-id', agent=Agent(security_analyzer=None, ...))

The problem here is that reconcilliation fails against the current agent and the one being restored from the persistent copy

ValueError: The Agent provided is different from the one in persisted state.
Diff: security_analyzer: '<missing>' -> {'kind': 'LLMSecurityAnalyzer'}

Current workaround

The current solution is to explicitly save the agent without a security analyzer and load it into the conversation. This is a lot of overhead for sdk clients

agent_without_security_analyzer = conversation.agent.model_copy(
                update={"security_analyzer": None}
)
agent_without_security_analyzer.to_json(settings_file)
conversation = Conversation(agent=Agent.from_json(settings_file))

[malhotra5]

Potential solution would to avoid reconciling the security analyzer against the persistent agent. We should just use the incoming security analyzer and use method 2 described above to update the agent

[xingyaoww]

@OpenHands review this issue, we should allow the security analyzer to be different during resolve from persist in AgentBase -- please implement the changes and start a new PR

[openhands-ai]

I'm on it! xingyaoww can track my progress at all-hands.dev

[openhands-ai]

Summary

I've successfully implemented a fix for issue #668 and created PR #669: #669

Problem Addressed

Previously, users couldn't change the security_analyzer configuration mid-conversation when restarting from persisted state. This was particularly problematic when switching to weaker LLMs that can't handle the security_risk field, requiring complex workarounds to remove the security analyzer.

Solution Implemented

Modified AgentBase.resolve_diff_from_deserialized() to allow the security_analyzer field to differ between runtime and persisted agents. The runtime agent's security analyzer now takes precedence, enabling users to freely add, remove, or change security analyzers when restarting conversations.

Changes Made

Code Changes (minimal and focused):

• openhands/sdk/agent/base.py (5 lines + docstring): Added updates["security_analyzer"] = self.security_analyzer to explicitly use the runtime agent's security analyzer during reconciliation

Test Coverage (comprehensive):

• tests/cross/test_agent_reconciliation.py (4 new tests):
  • Unit test for removing security analyzer
  • Unit test for adding security analyzer
  • Integration test for conversation restart without security analyzer
  • Integration test for conversation restart with added security analyzer

Verification

✅ All 10 tests in test_agent_reconciliation.py pass
✅ Pre-commit hooks pass (type checking, linting, formatting)
✅ No breaking changes - backward compatible
✅ Changes are concise and focused on the specific issue

The PR is currently marked as draft as per repository guidelines.

_{View full conversation}