Skip to content

Add cache control to Anthropic Bedrock provider #1615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
skull8888888 merged 10 commits into from
Apr 9, 2026
Merged

Add cache control to Anthropic Bedrock provider #1615

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f748b80
Add cache control to Anthropic Bedrock provider
skull8888888 Apr 9, 2026
ff2b26a
refactor: extract build_cache_point helper to deduplicate CachePointB…
skull8888888 Apr 9, 2026
67de219
fix: skip cache points when extended thinking is enabled on Bedrock
skull8888888 Apr 9, 2026
ed9be98
fixes
skull8888888 Apr 9, 2026
2512d58
removed print
skull8888888 Apr 9, 2026
3125eae
Fix cache point placement and total token count consistency
skull8888888 Apr 9, 2026
ad2f450
Add cache point to tool definitions in Bedrock provider
skull8888888 Apr 9, 2026
0e375c5
fix
skull8888888 Apr 9, 2026
c275fc6
fixes and migration
skull8888888 Apr 9, 2026
61a40b7
fix: add cache_control to last tool definition in Bedrock provider
skull8888888 Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions app-server/src/main.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -921,7 +921,7 @@ fn main() -> anyhow::Result<()> {
// == LLM Client ==
let llm_provider_client: Option<Arc<signals::provider::LlmClient>> =
if is_feature_enabled(Feature::Signals) {
log::info!("Initializing LLM client for signals");
log::info!("Initializing LLM client");
match runtime_handle.block_on(signals::provider::LlmClient::new()) {
Ok(client) => Some(Arc::new(client)),
Err(e) => {
Expand Down Expand Up @@ -1195,7 +1195,7 @@ fn main() -> anyhow::Result<()> {
QueueConfig::new(SIGNALS_QUEUE, SIGNALS_EXCHANGE, SIGNALS_ROUTING_KEY),
);
} else {
log::warn!("Gemini client not available - skipping signals workers");
log::warn!("LLM client not available - skipping signals workers");
}

// Spawn notification workers
Expand Down
4 changes: 2 additions & 2 deletions app-server/src/signals/filter.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ use uuid::Uuid;
use crate::cache::keys::SPAN_DROP_RULES_CACHE_KEY;
use crate::cache::{Cache, CacheTrait};
use crate::ch::spans::CHSpan;
use crate::signals::spans::extract_exception_from_events;
use crate::db::spans::SpanType;
use crate::mq::MessageQueue;
use crate::signals::prompts::{FILTER_GENERATION_SYSTEM_PROMPT, FILTER_GENERATION_USER_PROMPT};
Expand All @@ -16,6 +15,7 @@ use crate::signals::provider::models::{
ProviderPart, ProviderRequest, ProviderTool,
};
use crate::signals::provider::{LlmClient, ProviderThinkingConfig, ProviderThinkingLevel};
use crate::signals::spans::extract_exception_from_events;
use crate::signals::utils::{
InternalSpan, emit_internal_span, request_to_span_input, request_to_tools_attr,
};
Expand Down Expand Up @@ -287,7 +287,7 @@ pub async fn generate_and_cache_drop_rules(
tools: Some(build_filter_tool_definitions()),
generation_config: Some(ProviderGenerationConfig {
temperature: Some(1.0),
max_output_tokens: Some(2048),
max_output_tokens: Some(4096),
thinking_config: Some(ProviderThinkingConfig {
include_thoughts: Some(true),
thinking_level: Some(ProviderThinkingLevel::Medium),
Expand Down
Loading