Test kimi-k2 thinking

[neubig]

https://x.com/Kimi_Moonshot/status/1986449512538513505

@ryanhoangt

• Try Integration Test (7/7)
• Try in V1 CLI
• Try in the OpenHands interface --> Not working yet, we have to wait until we switch the UI to use V1 API: Add support for kimi-k2-thinking OpenHands#11689

@neubig

• Make sure support of the new model is publicized on social media

@juanmichelini

• Evaluate the model
• Add to OpenHands Cloud if results are excellent

@neubig

• Make sure the eval results are publicized on social media

[ryanhoangt]

@neubig I've completed my tasks and updated the statuses in the description above -- LMK if there's anything else that I can help.

[neubig]

thanks, I'll post about it!

[neubig]

@ryanhoangt

how did you get kimi-k2-thinking to work in the CLI?
It seems that it dies due to the current CLI sending the metadata field:

openhands.sdk.conversation.exceptions.ConversationRunError: Conversation run failed for id=63c11c83-c6cb-4a87-b2fc-5d92c366d051: litellm.BadRequestError: MoonshotException - Invalid request: the `metadata` field in the request (expected type string) is illegal, and array is not acceptable

[ryanhoangt]

This is weird, it seems work fine on my side:

What are your LLM settings? I use via the eval LLM proxy:

┌─────────────────────────────────────────────────────────────────────────────| Settings |──────────────────────────────────────────────────────────────────────────────┐
│   Custom Model:        litellm_proxy/moonshot/kimi-k2-thinking                                                                                                        │
│   Base URL:            https://llm-proxy.eval.all-hands.dev                                                                                                           │
│   API Key:             ********                             

[neubig]

I ran

uvx --python 3.12 openhands

and then

│   Custom Model:        moonshot/kimi-k2-thinking                                                                         │
│   Base URL:            https://api.moonshot.ai/v1

[enyst]

Ah, I see, it’s because of the telemetry bug: that we sent extra_body (with metadata) to all providers, when only litellm_proxy knows it

• fixed here: Fix metadata filtering for non-litellm_proxy providers #1134

[neubig]

Yeah, I guess that's it and it wasn't released yet. So I was confused by @ryanhoangt saying it worked, but I guess it's because he did it through the proxy.