-
-
Notifications
You must be signed in to change notification settings - Fork 303
Description
Basic checks
- I searched existing issues - this hasn't been reported
- I can reproduce this consistently
- This is a RubyLLM bug, not my application code
What's broken?
When I run RubyLLM::Models.refresh! I can see the following in the output:
#<RubyLLM::Model::Info:0x0000ffff3c3b9610
@capabilities=["streaming", "function_calling"],
@context_window=200000,
@created_at=nil,
@family=:other,
@id="eu.anthropic.claude-sonnet-4-5-20250929-v1:0",
@knowledge_cutoff=nil,
@max_output_tokens=4096,
@metadata={provider_name: "Anthropic", inference_types: ["INFERENCE_PROFILE"], streaming_supported: true, input_modalities: ["TEXT", "IMAGE"], output_modalities: ["TEXT"]},
@modalities=#<RubyLLM::Model::Modalities:0x0000ffff3c3a2ac8 @input=["text", "image", "pdf"], @output=["text"]>,
@name="Claude Sonnet 4.5",
@pricing=
#<RubyLLM::Model::Pricing:0x0000ffff3c3a2a50
@data=
{text_tokens:
#<RubyLLM::Model::PricingCategory:0x0000ffff3c3a29b0
@batch=#<RubyLLM::Model::PricingTier:0x0000ffff3c3a2910 @values={input_per_million: 0.05, output_per_million: 0.1}>,
@standard=#<RubyLLM::Model::PricingTier:0x0000ffff3c3a2960 @values={input_per_million: 0.1, output_per_million: 0.2}>>}>,
@provider="bedrock">,... but according to https://aws.amazon.com/bedrock/pricing/, the cost of 1M non-batch input tokens with Sonnet 4.5 in the EU is $3, not $0.1. For 1M output tokens it's $15, not $0.2. (Those are the same prices as on https://docs.claude.com/en/docs/about-claude/pricing)
I understand you're using a third party service here but perhaps it would be a good idea to flag the pricing info as unreliable?
I think if it was just a flat file where you could easily see in git that it's stale, maybe that would be a different story, but the refresh! method suggests that it's pulling accurate(ish) data from somewhere which doesn't seem to be the case.
On a side note, the output also shows batch pricing but Bedrock/Sonnet4.5 doesn't support batching and there are no prices listed.
How to reproduce
Ensure AWS credentials are present, then:
RubyLLM::Models.refresh!
RubyLLM::Models.find("eu.anthropic.claude-sonnet-4-5-20250929-v1:0").pricing.text_tokens.standard.to_h
=> {input_per_million: 0.1, output_per_million: 0.2}Expected behavior
RubyLLM::Models.refresh!
RubyLLM::Models.find("eu.anthropic.claude-sonnet-4-5-20250929-v1:0").pricing.text_tokens.standard.to_h
=> {input_per_million: 3, output_per_million: 15}What actually happened
Environment
RubyLLM version a0efaa4 but I've also verified with 1.8.2.