prompt caching SPIKE #109
prompt caching SPIKE #109
Conversation
| url: /guides/tools | ||
| - title: Streaming | ||
| url: /guides/streaming | ||
| - title: Prompt Caching |
There was a problem hiding this comment.
Think Cline got a little overzealous w/ documentation
| def supports_extended_thinking?(model_id) | ||
| model_id.match?(/claude-3-7-sonnet/) | ||
| end | ||
|
|
There was a problem hiding this comment.
I imagine there is an existing philosophy how to use this data I am unaware of
|
|
||
| def ask(message = nil, with: {}, &block) | ||
| add_message role: :user, content: Content.new(message, with) | ||
| # Extract cache_control from the with hash if present |
There was a problem hiding this comment.
I'm unsure if this is how we want to handle things. this essentailly adds global functionality for an anthropic specific concept.
| cache_control: true) | ||
|
|
||
| # Add a PDF document with cache control | ||
| chat.ask("Please analyze this document:", |
There was a problem hiding this comment.
this API seems fine to me. We need to tag the messages with "this is where to cache", so the with option seems like a fine place for that.
|
What's a \prompt caching SPIKE? |
|
It makes things hella cheap https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching Some models do it automatically, some don’t I’m unsure if I’m editing the library code in a sensible way, what do you think? |
|
Ok, cool. Can you please change the title and description of your PR? |
2 participants
No description provided.