Thinking & Reasoning Models

Reasoning models (a.k.a. “thinking” models) produce private chain-of-thought before their answer: Claude extended thinking (native Anthropic and Bedrock), OpenAI’s o-series, Gemini thinking, and Ollama models like qwen3 and deepseek-r1. Kova gives you one consistent way to enable, read, and account for that reasoning, regardless of provider.

The core rule

Chain-of-thought is surfaced separately and, as visible reasoning, never re-sent to the model as prompt. You get to see it (for display or logging), but it doesn’t pollute the context or cost you tokens on the next turn.

Where it shows up:

• Blocking calls (run) → AgentResponse::thinking: Option<String> (from the underlying ModelResponse::thinking).
• Streaming (run_stream) → AgentEvent::ThinkingDelta { text }.

One exception: signed Anthropic blocks

Anthropic requires the model’s signed thinking blocks to be replayed verbatim when a tool-use turn continues. Kova round-trips those through history as ContentBlock::Thinking (streamed turns included), so as long as you persist new_messages unchanged, tool loops with extended thinking just work. Other providers drop Thinking blocks when building requests, so cross-provider history stays valid.

Enabling reasoning, per provider

Each provider exposes the knob its API uses; the output is unified.

Anthropic (native) — adaptive extended thinking is on by default; tune effort:

use kova_sdk::provider::anthropic::AnthropicProviderConfig;

let config = AnthropicProviderConfig::new("claude-opus-4-8")
    .with_api_key("sk-ant-…")
    .with_effort("high")           // low | medium | high | xhigh | max
    .with_adaptive_thinking(true); // default on

OpenAI o-series — reasoning effort:

use kova_sdk::provider::openai::OpenAiProviderConfig;

let config = OpenAiProviderConfig::new("https://api.openai.com", "o3")
    .with_api_key("sk-…")
    .with_reasoning_effort("high");   // "low" | "medium" | "high"

AWS Bedrock (Claude extended thinking) — pass a token budget through:

use kova_sdk::provider::bedrock::BedrockProviderConfig;

let config = BedrockProviderConfig::new("us-east-1", "anthropic.claude-sonnet-4-20250514-v1:0")
    .with_additional_model_request_fields(serde_json::json!({ "budgetTokens": 5000 }));

Google Gemini — thinking budget:

use kova_sdk::provider::gemini::GeminiProviderConfig;

// -1 = dynamic/unlimited, 0 = off (default), positive = hard cap
let config = GeminiProviderConfig::new("gemini-2.5-flash")
    .with_api_key("AIza…")
    .with_thinking_budget(5000);

Ollama — think mode (needs a thinking-capable model):

use kova_sdk::provider::ollama::{OllamaProviderConfig, OllamaThink};

let config = OllamaProviderConfig::new("qwen3")
    .with_think(OllamaThink::High);   // Enabled | High | Medium | Low

Reading the reasoning

Blocking

let history = vec![user_message("Prove that √2 is irrational.")];
let response = agent.run(&history).await?;

println!("Answer:\n{}", response.text);
if let Some(reasoning) = &response.thinking {
    eprintln!("\n[model reasoning]\n{reasoning}");
}

Streaming — separate the two channels

Render the answer on stdout and the (dimmed) reasoning on stderr so a user can watch the model think:

use futures::StreamExt;

let stream = agent.run_stream(&history);
futures::pin_mut!(stream);
while let Some(event) = stream.next().await {
    match event? {
        AgentEvent::ThinkingDelta { text } => eprint!("\x1b[2m{text}\x1b[0m"), // dim
        AgentEvent::TextDelta { text }     => print!("{text}"),                // answer
        AgentEvent::TurnCompleted { .. }   => {}
        _ => {}
    }
}

Token accounting

Thinking tokens are counted where the provider reports them separately:

• UsageStats::thinking_tokens: Option<u32> — reasoning tokens as a subset of output_tokens (not additive). None means the provider doesn’t report a separate count — surfaced honestly as “unknown” rather than a misleading 0.
• Aggregated across every provider call in a turn: None + None stays None; any reported value flips the running total to Some.

Which providers report them:

Provider	Separate thinking-token count?
OpenAI (o-series)	✅ via reasoning_tokens
Gemini	✅ via thoughtsTokenCount
Anthropic (native)	❌ folded into output_tokens (None)
Bedrock	❌ folded into output_tokens (None)
Ollama	❌ folded into output_tokens (None)

let response = agent.run(&history).await?;
let u = &response.usage;
match u.thinking_tokens {
    Some(t) => println!("{t} of {} output tokens were reasoning", u.output_tokens),
    None    => println!("provider doesn't break out reasoning tokens"),
}

Practical notes

• Latency & cost. Higher reasoning effort / larger budgets mean more tokens and slower responses. Tune reasoning_effort / budgetTokens / thinking_budget to the difficulty of the task.
• Non-reasoning models ignore the knobs. Setting reasoning_effort on a non-o-series model, or thinking_budget on a non-thinking Gemini model, is a no-op — safe to leave in provider-agnostic code.
• History stays clean. Because thinking is never persisted, multi-turn conversations don’t accumulate reasoning text, keeping context small.

Key takeaways

• Reasoning is surfaced via thinking / ThinkingDelta and never stored or re-sent.
• Enable it with the provider’s own knob (reasoning_effort, budgetTokens, thinking_budget, OllamaThink); the output shape is unified.
• UsageStats::thinking_tokens is an Option subset of output tokens — None = provider doesn’t report it (OpenAI/Gemini do, Bedrock/Ollama don’t).