The Agentic Loop

The agentic loop is the single most important concept in Kova. Everything else — providers, tools, streaming — exists to serve it. Understand this page and the rest of the SDK falls into place.

The problem it solves

A raw language model can only do one thing: take a list of messages and return some text (or a request to call a tool). It can’t fetch a web page, read a file, or query a database. To make it do things, you have to:

1. tell it which tools exist,
2. notice when it asks to use one,
3. actually run that tool,
4. hand the result back,
5. and repeat, because it might want another tool before it can answer.

That back-and-forth is the agentic loop. Writing it by hand is fiddly and easy to get wrong. Agent runs it for you.

The loop, step by step

When you call agent.run(&history), Kova runs this:

run(history)
  │
  └─ loop (up to max_iterations + 1 provider calls):
       ├─ 1. build messages = system_prompt + history
       ├─ 2. provider.chat_completion(messages, tool_defs, config)
       │
       ├─ 3. model wants to use tools?  (StopReason::ToolUse)
       │       ├─ record the assistant's tool-use request
       │       ├─ execute the requested tools concurrently
       │       ├─ append each ToolResult to the history
       │       └─ go back to step 1
       │
       └─ 4. model is done?  (EndTurn / MaxTokens)
               └─ return AgentResponse { text, new_messages, usage, … }

  (if the loop runs past its cap → Err(KovaError::MaxIterations))

The crucial part is the arrow from step 3 back to step 1: after tools run, the model is asked again, now with the tool results in front of it. It may answer, or it may ask for more tools. The loop continues until the model stops asking — or until it hits max_iterations (default 10).

A worked example

You ask: “What files are in my project and how big is the largest one?”

Iteration	Model does	Kova does
1	Asks to call list_dir(".")	Runs it, appends the file list
2	Asks to call read_file("big.log")’s stats	Runs it, appends the size
3	Writes the final sentence (EndTurn)	Returns it to you

Three provider calls, two tool executions, one line of your code (agent.run(&history).await?).

What a turn produces

run returns an AgentResponse:

pub struct AgentResponse {
    pub text: String,                           // the final assistant text
    pub new_messages: Vec<ConversationMessage>, // everything the turn produced
    pub stop_reason: StopReason,                // why the loop ended
    pub usage: UsageStats,                       // tokens summed across all calls
    pub llm_calls: u64,                          // how many provider round-trips
    pub thinking: Option<String>,                // chain-of-thought, if any
}

new_messages is the important one: it holds every message the turn generated — the assistant’s tool-use requests, the tool results, and the final answer. Persist it (history.extend(response.new_messages)) and your next turn has full context.

Concurrency: tools run in parallel

When the model asks for several tools in one step, Kova executes them concurrently, not one after another, bounded by a semaphore (max_concurrent_tools, default 10). Results are collected in order, so a model that requests get_weather("Tokyo") and get_weather("Paris") together gets both back in a single round-trip’s worth of latency.

let agent = AgentBuilder::new()
    .provider(provider)
    .max_concurrent_tools(4)   // cap parallel tool execution at 4
    .build()?;

Bounding the loop

Two knobs keep a misbehaving model from looping forever:

let agent = AgentBuilder::new()
    .provider(provider)
    .max_iterations(10)        // default: max 10 tool-calling rounds
    .build()?;

If the model keeps asking for tools past the cap, run returns Err(KovaError::MaxIterations(n)) rather than spinning indefinitely. Raise the cap for deep multi-step tasks; lower it to fail fast.

The agent is stateless — you own the history

Agent::run(&[ConversationMessage]) is the only entry point to the loop, and it’s stateless. You pass the history in, you get new_messages out, and you decide where to store them. Nothing is read from or written to any hidden store, which is exactly what makes an agent cheap to Arc-share across concurrent requests and trivial to scale horizontally.

let mut history = vec![user_message("What's the weather in Tokyo?")];
let response = agent.run(&history).await?;
history.extend(response.new_messages);   // you decide where this goes

Persistence, sessions, and compaction are the host’s job, not the SDK’s. See Conversations & History for the patterns.

Variants of run

The same loop is available through a few variants, all stateless:

• run_with_config(messages, overrides) — apply per-call InferenceConfig overrides for this turn (unset fields fall back to the agent’s defaults).
• run_structured::<T>(messages, format) — constrain the final answer to a JSON schema and parse it into T. See Structured Output.
• run_stream(messages) — the loop as a live event stream (TextDelta, ToolCallStarted, ToolCallFinished, TurnCompleted). Same control flow, in real time. See Streaming.
• run_cancellable(messages, token) / run_stream_cancellable — take a CancellationToken that aborts the turn mid-provider-call or mid-tool with KovaError::Cancelled, producing no messages. See Reliability & Errors.

Key takeaways

• The loop is: ask the model → run the tools it wants → repeat → return the answer.
• Agent runs it; you supply the provider and (optionally) tools.
• Tools requested together execute concurrently.
• max_iterations bounds the loop; overflow is a clean MaxIterations error.
• The agent is stateless: run takes history in, hands new_messages back, and you own persistence.