Streaming

Waiting for a whole response before showing anything feels slow. Streaming lets you render tokens the moment the model produces them — the difference between a chatbot that feels alive and one that feels stuck. Kova exposes streaming as a single pull-based event stream over the agentic loop.

The pull-based event stream

run_stream returns a Stream of AgentEvents you consume with a while let loop. No handler to register, no callbacks — you’re in control of the loop:

use futures::StreamExt;

let mut history = vec![user_message("Tell me a short story.")];

let stream = agent.run_stream(&history);
futures::pin_mut!(stream);

while let Some(event) = stream.next().await {
    match event? {
        AgentEvent::TextDelta { text } => print!("{text}"),
        AgentEvent::ThinkingDelta { text } => eprint!("\x1b[2m{text}\x1b[0m"), // dim reasoning
        AgentEvent::ToolCallStarted { name, .. } => eprintln!("\n⚙ calling {name}…"),
        AgentEvent::ToolCallFinished { name, .. } => eprintln!("✓ {name} done"),
        AgentEvent::TurnCompleted { response } => {
            // the turn is done — persist everything it produced
            history.extend(response.new_messages);
        }
        _ => {}
    }
}

AgentEvent comes from kova_sdk::agent and is re-exported in the prelude.

The AgentEvent variants

Event	Meaning
TextDelta { text }	A chunk of the visible answer. Print these as they arrive.
ThinkingDelta { text }	A chunk of chain-of-thought from a reasoning model.
ToolCallStarted { id, name, input }	The agent is about to run a tool (arguments fully accumulated).
ToolCallFinished { id, name, result, is_error }	A tool finished; its result goes back to the model.
TurnCompleted { response }	The whole turn is done; carries the full AgentResponse.

Note that streaming covers the entire loop, not just one model call: you’ll see text, then tool calls, then more text as the agent works through a multi-step task. TurnCompleted fires once at the very end and gives you the same AgentResponse that run would have returned — including new_messages to persist.

Why a pull-based stream: back-pressure is natural (you pull at your own pace), errors surface right in the loop (event?), and there’s no shared-state handler to synchronize.

Streaming and tool calls

You don’t have to do anything special for tools. During a stream, argument deltas for a tool call are accumulated internally until the call is complete, then the tool executes (you’ll see ToolCallStarted / ToolCallFinished), and the loop continues streaming the model’s next step. Parallel tool calls are correlated by the provider’s index, so interleaved or repeated calls don’t get merged or duplicated.

Cancelling a live stream

run_stream_cancellable(messages, token) takes a CancellationToken (a prelude re-export of tokio_util’s) that stops the stream mid-flight — mid-provider-call or mid-tool. When the token fires, the stream yields a KovaError::Cancelled and ends. Handy for a “stop generating” button or a per-turn deadline:

use kova_sdk::prelude::CancellationToken;

let cancel = CancellationToken::new();
let stream = agent.run_stream_cancellable(&history, cancel.clone());
// … elsewhere: cancel.cancel();  // aborts the in-flight turn

See Reliability & Errors.

Robustness built in

• UTF-8 safety. Streams are decoded line-wise, so a multi-byte character split across two network chunks is never corrupted.
• Retries. Establishing the stream is retried on transient failures, same as blocking calls. See Reliability & Errors.
• Usage tracking. The final TurnCompleted carries an AgentResponse whose usage (including any cache_read_tokens / thinking_tokens) is summed across every provider call in the turn, so you can meter consumption.

Streaming vs. blocking

Both run and run_stream execute the exact same loop and produce the same AgentResponse — the only difference is whether you observe it incrementally. Reach for run_stream when you’re rendering to a user in real time (a CLI, TUI, or a server-sent-events endpoint); use plain run for background work where nobody’s watching the tokens.

Key takeaways

• One streaming API: the pull-based run_stream yielding AgentEvents.
• Streaming spans the whole loop — text, tool calls, and more text.
• TurnCompleted marks the end; grab its AgentResponse::new_messages to persist.
• run_stream_cancellable stops a stream mid-flight with a CancellationToken.
• Multi-byte-safe, retried, and usage-reported out of the box.