Skip to content

Conversation

@Wh1isper
Copy link
Contributor

@Wh1isper Wh1isper commented Nov 15, 2025

closes #3441
closes #3467

@Wh1isper Wh1isper changed the title feat: add anthropic_cache_all in AnthropicModel for auto caching all … feat: add anthropic_cache_all in AnthropicModel Nov 15, 2025
@Wh1isper
Copy link
Contributor Author

One more thing, if enable anthropic_cache_all, we should ignore all CachePoint in message history

@DouweM
Copy link
Collaborator

DouweM commented Nov 18, 2025

One more thing, if enable anthropic_cache_all, we should ignore all CachePoint in message history

@Wh1isper Ignore all, or only respect the final 4 i.e. #3467?

@Wh1isper
Copy link
Contributor Author

@DouweM You're right, we can handle this more generically:

  1. When enable anthropic_cache_all, mark the last message as a cache point
  2. Retain the last four cache points(2-4 for message, 1 for instruction, 1 for tool definition)

@Wh1isper Wh1isper changed the title feat: add anthropic_cache_all in AnthropicModel Add cache all and limit cache point in AnthropicModel Nov 19, 2025
@Wh1isper Wh1isper force-pushed the feat-anthropic-cache-all branch from ec1dcb5 to b4862a0 Compare November 19, 2025 03:32
@Wh1isper Wh1isper marked this pull request as ready for review November 19, 2025 03:33
@Wh1isper Wh1isper requested a review from DouweM November 19, 2025 03:33
@Wh1isper Wh1isper changed the title Add cache all and limit cache point in AnthropicModel Add anthropic_cache_messages and fix cache point in AnthropicModel Nov 20, 2025
@Wh1isper Wh1isper requested a review from DouweM November 20, 2025 03:41
@Wh1isper Wh1isper changed the title Add anthropic_cache_messages and fix cache point in AnthropicModel Add anthropic_cache_messages and CachePoint stripping in AnthropicModel Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic prompt caching fails when there are more than 4 cache points Anthropic prompt caching for tool response

2 participants