Add OpenRouter as a supported provider

[khasinski]

Addresses #8

This adds support for OpenRouter.

You can use it by configuring the provider with:

RubyLLM.configure do |config|
  config.openrouter_api_key = 'your key'
end

Then calling models by their OpenRouter ids (it might be wise to prefix those with openrouter or something else if there is a possibility there will be more multi-model providers in the future)

chat = RubyLLM.chat(model: "google/gemini-2.0-flash-lite-preview-02-05:free")
chat.ask "What's the best way to learn Ruby?"

OpenRouter doesn't support embedding models and image generation so those are not implemented here.

Getting model details requires multiple calls to OpenRouter API, since they don't expose function calling and structured output support in the responses. However they do allow filtering models by those capabilities. I memoized those calls for subsequent lookups, so the model fetching goes faster.

Code is based on OpenAI provider, since they share most of the API, but I've kept it separate since they might differ in future.

A small note - this adds a lot of data to models.json, it might be wise to generate that file on first use instead of bundling it with the gem or add refreshing to docs just like Ollama PR does (#10).

[crmne]

This is an excellent PR! You've clearly understood RubyLLM's design philosophy and implemented OpenRouter support in a way that fits seamlessly with the rest of the library.

I'll test it out and do a comprehensive code review soon. Your concern about models.json is valid - we need a better long-term solution. Requiring users to refresh models has its own issues (filesystem access, rate limits), so maybe we need something like a CDN that we update via GitHub Actions.

For multi-provider model selection, we might want to introduce a provider: parameter to .chat and a .with_provider(:sym) method to let users explicitly choose, with defaults to the original provider. This would let us cleanly handle cases where multiple providers offer the same model.

[khasinski]

Models.json issue

Should I propose some provider options? If that's the case should I do it in another PR or extend this one? I think Ollama PR already has some progress by creating a concept of enabled/disabled providers and fetching models on demand, but of course we can do that CDN option as well :)

Python's AI ecosystem often downloads config/model files during first run (and some Ruby AI ecosystem follow this pattern, like for example Tokenizers gem), so it wouldn't be unusual to fetch models.json during initialization. We can also leverage github or huggingface for hosting that models.json.

Rate limits and depending on 3rd party APIs

One thing that I've noticed about openrouter is that the models API is actually public and doesn't require any keys which simplifies the implementation a bit. However it's also somewhat undocumented, as the supported capabilities aren't mentioned in the docs, but they are used by the openrouter website.

[khasinski]

Rebased to resolve a conflict in README.md

[crmne]

Two new features to consider in your provider implementation:

1. Model Aliases: Please add entries for your provider in aliases.json:

   "claude-3-5-sonnet": {
     "anthropic": "claude-3-5-sonnet-20241022",
     "your-provider": "your-provider-specific-id"
   }

2. Provider Selection: Users will be able to specify your provider:

   chat = RubyLLM.chat(model: 'claude-3-5-sonnet', provider: 'your-provider')

Docs: https://rubyllm.com/guides/models#using-model-aliases

[crmne]

Added configuration requirements handling in 75f99a1

Each provider now specifies what configuration is required via a simple configuration_requirements method (you will need to implement this in your main provider file) that returns an array of config keys as symbols. The Provider module uses this to:

1. Determine if a provider is properly configured
2. Throw an error if you're trying to use that provider without configuration
3. Include ready-to-paste configuration code in the error message
4. Skip unconfigured providers during model refresh while preserving their models

Example of the new error messages:

RubyLLM::ConfigurationError: anthropic provider is not configured. Add this to your initialization:

RubyLLM.configure do |config|
  config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
end

[khasinski]

@crmne Updated according to your specs

[codecov]

Codecov Report

Attention: Patch coverage is 50.83333% with 59 lines in your changes missing coverage. Please review.

Project coverage is 91.01%. Comparing base (0aff3fb) to head (534d2b6).
Report is 7 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/ruby_llm/providers/openrouter/chat.rb	38.46%	16 Missing ⚠️
lib/ruby_llm/providers/openrouter/tools.rb	39.13%	14 Missing ⚠️
lib/ruby_llm/providers/openrouter/media.rb	40.90%	13 Missing ⚠️
lib/ruby_llm/providers/openrouter.rb	66.66%	9 Missing ⚠️
lib/ruby_llm/providers/openrouter/models.rb	63.63%	4 Missing ⚠️
lib/ruby_llm/providers/openrouter/streaming.rb	70.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #29      +/-   ##
==========================================
- Coverage   92.73%   91.01%   -1.73%     
==========================================
  Files          73       79       +6     
  Lines        2795     2915     +120     
  Branches      406      424      +18     
==========================================
+ Hits         2592     2653      +61     
- Misses        203      262      +59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[khasinski]

I'll add some tests to raise the coverage

[khasinski]

Added specs, rebased on top of main branch.

[khasinski]

Side note - maybe we shouldn't sort models by name but by provider, then name. This might be better for PR diffs

[khasinski]

Rebased on top of main branch and resolved conflicts.

@crmne any chance for a review? I'd like to merge this :)

[crmne]

Thank you for this PR!

I have left a couple of comments about it.

In addition to those, it would be great to add haiku to the chat_content_spec.rb, chat_error_spec.rb, chat_functions_spec.rb, chat_pdf_spec.rb

Looking forward to see the changes!

[crmne]

let's add the openai version here first just to keep the style consistent

[khasinski]

I'm not sure what you mean, can you explain a bit more?

[crmne]

Looks like we don't need the entire capabilities module since their API returns all we need. Check out https://github.com/crmne/ruby_llm/blob/main/lib/ruby_llm/providers/openai/models.rb

[khasinski]

Are you sure about this? If I read this correctly then OpenAI provider in RubyLLM basically just hardcodes list of capabilities here: https://github.com/crmne/ruby_llm/blob/main/lib/ruby_llm/providers/openai/capabilities.rb and does the lookup by id in the file you mentioned. This won't work for a very broad and dynamic list of models that OpenRouter provides, so it's easier to use their API to detect model capabilties.

[crmne]

@khasinski I'm playing around with the OpenAI API of OpenRouter and so far things are going smoothly. Is there any reason to implement their API anymore?

[khasinski]

Other than fetching the models not really. We can probably close this if you're ok with not having the model list in the API or we can trim it down to just fetching the list of models. Your choice :)

[crmne]

Hi @khasinski I decided to go ahead and implement it myself in d698dd1.

Thank you very much for your effort and contribution!