[BUG] LiteLLM integration with Strands Agent

[shanicusyee]

Checks

• I have updated to the lastest minor and patch version of Strands
• I have checked the documentation and this is not expected behavior
• I have searched ./issues and there are no duplicates of my issue

Strands Version

1.3.0

Python Version

3.13

Operating System

macos 15.5

Installation Method

pip

Steps to Reproduce

from strands import Agent
from strands.models.litellm import LiteLLMModel

litellm_model = LiteLLMModel(
client_args={
"api_key": "sk-******",
},
# **model_config
model_id="gemini/gemini-1.5-flash",
params={
"max_tokens": 1000,
"temperature": 0.7,
}
)

Create an agent with the model via LiteLLM

agent = Agent(model=litellm_model)

response = agent(
"You're advising a startup with $10M funding, entering the AI productivity tools space. Given current trends, outline a go-to-market plan, suggest a pricing model, and identify key technical differentiators needed to compete with Notion AI and Microsoft Copilot."
)

Expected Behavior

Gemini to reply the request.

Actual Behavior

litellm.exceptions.AuthenticationError: litellm.AuthenticationError: vertex_ai_betaException - b'{\n "error": {\n "code": 400,\n "message": "API key not valid. Please pass a valid API key.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.ErrorInfo",\n "reason": "API_KEY_INVALID",\n "domain": "googleapis.com",\n "metadata": {\n "service": "generativelanguage.googleapis.com"\n }\n },\n {\n "@type": "type.googleapis.com/google.rpc.LocalizedMessage",\n "locale": "en-US",\n "message": "API key not valid. Please pass a valid API key."\n }\n ]\n }\n}\n'

Additional Context

i have tested using litellm without strands agent and it works

Possible Solution

No response

Related Issues

No response

[dbschmigelski]

Hi, can you point to the documentation regarding passing the api key? On https://docs.litellm.ai/docs/providers/gemini#api-keys I see that the environment variable GEMINI_API_KEY should be set.

For OpenAI I see

This library requires an API key for all requests, either through the api_key parameter or the OPENAI_API_KEY environment variable.

https://docs.litellm.ai/docs/providers/openai_compatible

For Gemini, I do not see a similar comment about api_key though?

[shanicusyee]

Hi dbschmigelski,

what do you mean on this on regarding the API key? i am using the AWS solutions FYI. The api key is configured here: https://github.com/aws-solutions-library-samples/guidance-for-multi-provider-generative-ai-gateway-on-aws/blob/main/.env.template

the issue is the same as #661

[dbschmigelski]

In your reproduction example you wrote

litellm_model = LiteLLMModel(
client_args={
"api_key": "sk-******",
},
# **model_config
model_id="gemini/gemini-1.5-flash",
params={
"max_tokens": 1000,
"temperature": 0.7,
}

It seems the

client_args={
"api_key": "sk-******",
},

part is not expected and instead GEMINI_API_KEY should just be set. The sk- seems to be the prefix for openai, not gemini?

---

Apologies if this is just a misunderstanding of LiteLLM. Are you saying that for the proxy approach api_key must be passed via client_args and that is why #661 is the same as this?

[shanicusyee]

sk-*** is the api key for clients to use LiteLLM
internally LiteLLM has a configured gemini/openai/3rdparty FM key
architecture below:
Client LiteLLM Key -> LiteLLM -> LiteLLM check is key is valid -> if valid -> LiteLLM passes the request to model with API key configured in LiteLLM

however, when using StrandsSDK, the params don't get passed properly.

[dbschmigelski]

Gotcha, thanks. We will test this using the proxy approach.

[dbschmigelski]

Hi, apologies for the delay and the misunderstanding. It looks like you are trying to use the LiteLLM Proxy Server.

Following #808, the way to do this is to either

set use_litellm_proxy to true

litellm_model = LiteLLMModel(
        client_args={
            "api_key": "sk-1234",
            "api_base": LITELLM_PROXY_URL  # also fails for "base_url": LITELLM_PROXY_URL,
            "use_litellm_proxy": True
        },
        model_id="amazon.nova-lite-v1:0"
    )

or to prefix the model id

litellm_model = LiteLLMModel(
        client_args={
            "api_key": "sk-1234",
            "api_base": LITELLM_PROXY_URL  # also fails for "base_url": LITELLM_PROXY_URL,
        },
        model_id="litellm_proxy/amazon.nova-lite-v1:0"
    )

What was happening previously was your request was going directly to gemini instead of through your proxy.

Thanks again and sorry for the misunderstanding. We'll know better for next time to disambiguate litellm, operating as a client, from litellm proxy.