docs: sync OpenHands LLMs list with Agent SDK VERIFIED_OPENHANDS_MODELS #89
+248
−8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…LS\n\nSource of truth: openhands-sdk/openhands/sdk/llm/utils/verified_models.py\n- Add: claude-haiku-4-5-20251001, gpt-5-codex, claude-opus-4-1-20250805, kimi-k2-0711-preview\n- Remove: devstral-small-2505\n- Align order with VERIFIED_OPENHANDS_MODELS\n\nCo-authored-by: openhands <[email protected]>
|
I've removed myself and asked Xingyao for a look. I don't know how correct the changes are. |
… N/A; add source note\n\nSource: litellm model_prices_and_context_window_backup.json; Verified list remains source-of-truth for models.\n\nCo-authored-by: openhands <[email protected]>
…50514 1M input tokens)\n\nCo-authored-by: openhands <[email protected]>
…ored-by: openhands <[email protected]>
…\n\nCo-authored-by: openhands <[email protected]>
…p LiteLLM source note\n\nCo-authored-by: openhands <[email protected]>
…nCo-authored-by: openhands <[email protected]>
enyst
commented
Nov 6, 2025
|
Yup! I verified some all over the place. In general, the list didn't change much:
|
xingyaoww
reviewed
Nov 6, 2025
…e JSON)\n\n- Skips models not present or intentionally N/A\n- Compares input/cached/output costs per 1M and token limits when available\n\nCo-authored-by: openhands <[email protected]>
…Rs\n\nCo-authored-by: openhands <[email protected]>
…ints in validator\n\nCo-authored-by: openhands <[email protected]>
…oq, cloudflare)\n\nCo-authored-by: openhands <[email protected]>
enyst
commented
Nov 7, 2025
| add_fail(f"input_cost mismatch: mdx={mdx_input_cost} vs litellm={exp_input_cost}") | ||
|
|
||
| # Cached input cost | ||
| if exp_cached_cost is not None or mdx_cached_cost is not None: |
There was a problem hiding this comment.
GPT-5 explanation for why it cares about None on cached input, but not on input or output:
Cached input cost: We treat it as both a price and a capability signal (prompt caching support). So the validator enforces presence parity and numeric accuracy:
- Both None → OK
- LiteLLM None, MDX number → fail (docs claim caching where provider doesn’t report it)
- LiteLLM number, MDX None → fail (docs missing a provider-reported caching price)
- Both numbers → compare within tolerance
Input/output costs: These are fundamental but occasionally missing in LiteLLM for preview/edge cases. To avoid false failures due to incomplete upstream data, we only compare when both sides provide numbers; if either is None, we skip strict enforcement.
There was a problem hiding this comment.
It makes sense to me... WDYT?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR updates the OpenHands LLMs documentation to match the source of truth in the Agent SDK.
Source of truth:
openhands-sdk/openhands/sdk/llm/utils/verified_models.pyVERIFIED_OPENHANDS_MODELSChanges:
claude-haiku-4-5-20251001,gpt-5-codex,claude-opus-4-1-20250805,kimi-k2-0711-previewdevstral-small-2505Why:
Files changed:
openhands/usage/llms/openhands-llms.mdxCo-authored-by: openhands [email protected]
@enyst can click here to continue refining the PR