fix(models): patch litellm bug to honor passing in use_litellm_proxy as client_args

[dbschmigelski]

Description

A customer complained that when they ran

litellm_model = LiteLLMModel(
        client_args={
            "api_key": "sk-1234",
            "api_base": LITELLM_PROXY_URL  # also fails for "base_url": LITELLM_PROXY_URL
        },
        model_id="amazon.nova-lite-v1:0"
    )

They were seeing httpx.HTTPStatusError: Client error '404 Not Found' for url '{LITELLM_PROXY_URL}/model/amazon.nova-pro-v1%3A0/converse'

The reason for this is that LiteLLM recognizes the model as a Bedrock model, so it selects the BedrockModel provider. Then it uses api_base in a similar way to how in the Strand BedrockModel we have endpoint_url. So LiteLLM, as a client, is making a request to Bedrock but assuming Bedrock is behind some other endpoint.

What the customer expected, is that the request would be route to their own LiteLLM Proxy Server. As stated above, setting the api_base does NOT indicate that the LiteLLM Proxy Server should be used, it indicates that the selected model provider happens to. have an endpoint different than the default one - for example for bedrock the default is https://bedrock-runtime.{region}.amazonaws.com

So the primary fix will be to update https://strandsagents.com/latest/documentation/docs/user-guide/concepts/model-providers/litellm/.

The fix in this PR is to make it easier for customers to use the proxy. There are three methods described in BerriAI/litellm#13454 but according to BerriAI/litellm#13454 passing in as a dynamic param is broken. Meaning without this fix, customers would need to break the abstraction in Strands and directly do os.environ["USE_LITELLM_PROXY"] = "True" or litellm.use_litellm_proxy = True.

This, and the doc fix, will address #661 which until now we did not realize was the same issue.

Related Issues

#661

Documentation PR

strands-agents/docs#233

Type of Change

Bug fix

Testing

Run export LITELLM_PROXY_API_KEY=sk-1234; litellm -m amazon.nova-lite-v1:0

then

from strands import Agent
from strands.models.litellm import LiteLLMModel
    
import litellm
litellm._turn_on_debug()

litellm_model = LiteLLMModel(
    client_args={
        "api_key": "sk-1234",
        "api_base": "http://0.0.0.0:4000"
    },
    model_id="amazon.nova-lite-v1:0"
)

agent = Agent(model=litellm_model)
agent("Tell me a story")

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

• I ran hatch run prepare

Checklist

• I have read the CONTRIBUTING document
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.