A CLI Chatbot

import { Steps } from ‘@astrojs/starlight/components’;

This example is a full, runnable terminal chatbot: it holds a conversation, streams the model’s reply token-by-token, and remembers earlier turns. Because Kova is stateless, you own the history — it’s a plain Vec<ConversationMessage> the program keeps and feeds back each turn. It ties together providers, conversations & history, and streaming.

Set up

1. Create the project:

   cargo new kova-chat && cd kova-chat
   cargo add kova-sdk
   cargo add tokio --features macros,rt-multi-thread,io-std
   cargo add futures

2. Have a model endpoint ready. The code below points at a local OpenAI-compatible server (Ollama, LM Studio, vLLM) needing no key. To use hosted OpenAI or Anthropic, change the provider (noted inline).

The full program

src/main.rs

use std::io::{self, Write};
use std::sync::Arc;

use futures::StreamExt;
use kova_sdk::prelude::*;
use kova_sdk::provider::openai::{OpenAiCompatibleProvider, OpenAiProviderConfig};

fn user_message(text: &str) -> ConversationMessage {
    ConversationMessage { role: Role::User, content: vec![ContentBlock::Text { text: text.into() }] }
}

#[tokio::main]
async fn main() -> Result<(), KovaError> {
    // Nice structured logs; swap for OTLP in production (see the Telemetry guide).
    kova_sdk::telemetry::TelemetryConfig::builder()
        .log_level(tracing::Level::WARN)
        .build()
        .init()
        .ok();

    // --- Provider ----------------------------------------------------------
    // Local server, no API key. For hosted OpenAI:
    //   OpenAiProviderConfig::new("https://api.openai.com", "gpt-4o")
    //       .with_api_key(std::env::var("OPENAI_API_KEY").unwrap())
    let config = OpenAiProviderConfig::new("http://127.0.0.1:1234", "my-model")
        .with_temperature(0.7);
    let provider = Arc::new(OpenAiCompatibleProvider::new(config)?);

    // --- Agent -------------------------------------------------------------
    let agent = AgentBuilder::new()
        .provider(provider)
        .system_prompt("You are a concise, friendly assistant.")
        .build()?;

    // We own the conversation. The system prompt is prepended by the agent, so
    // it never lives in `history`.
    let mut history: Vec<ConversationMessage> = Vec::new();

    println!("Kova chat — type 'exit' to quit.\n");

    loop {
        // --- Read the user's line ------------------------------------------
        print!("you › ");
        io::stdout().flush().ok();

        let mut line = String::new();
        if io::stdin().read_line(&mut line).unwrap_or(0) == 0 {
            break; // EOF (Ctrl-D)
        }
        let input = line.trim();
        if input.is_empty() {
            continue;
        }
        if input.eq_ignore_ascii_case("exit") || input.eq_ignore_ascii_case("quit") {
            break;
        }

        // Append the user's turn to the history we own.
        history.push(user_message(input));

        // --- Stream the reply ----------------------------------------------
        print!("bot › ");
        io::stdout().flush().ok();

        let stream = agent.run_stream(&history);
        futures::pin_mut!(stream);
        while let Some(event) = stream.next().await {
            match event {
                Ok(AgentEvent::TextDelta { text }) => {
                    print!("{text}");
                    io::stdout().flush().ok();
                }
                // The turn is done — fold everything it produced back into history
                // so the next turn has full context.
                Ok(AgentEvent::TurnCompleted { response }) => {
                    history.extend(response.new_messages);
                    println!();
                }
                Err(e) => {
                    eprintln!("\n[error] {e}");
                    // Drop the user message we couldn't answer, so history stays clean.
                    history.pop();
                    break;
                }
                _ => {}
            }
        }
    }

    println!("\nbye 👋");
    Ok(())
}

Run it

cargo run

Kova chat — type 'exit' to quit.

you › what's the capital of France?
bot › Paris.
you › and how many people live there?
bot › About 2.1 million in the city proper, ~11 million in the metro area.
you › exit

bye 👋

The second question (“there”) only makes sense because the first turn is still in history — you kept response.new_messages after each turn, so the model sees the whole thread. That’s the entire persistence story: a Vec you own.

Keeping context bounded

Left alone, history grows forever and eventually blows past the model’s context window. Because you own it, trimming is a few lines — keep the most recent turns:

const MAX_MESSAGES: usize = 40;
if history.len() > MAX_MESSAGES {
    // Keep the tail. If you add tools, cut at a user-message boundary instead,
    // so an assistant tool-use block never loses its following tool result.
    history.drain(0..history.len() - MAX_MESSAGES);
}

To fail fast instead of silently dropping context, add .context_budget(n_tokens) on the builder — an over-long prompt then returns KovaError::ContextBudgetExceeded before the request is sent. See Reliability & Errors.

Extend it

• Give it tools. Add .tool(Arc::new(GetWeather)) and the bot can act — see A Research Agent.
• Show reasoning. Point it at a thinking model and print ThinkingDelta dimmed — see Thinking & Reasoning Models.
• Persist across restarts. history is a Vec of serde-serializable messages — write it to a file or database on exit and load it on start. See Conversations & History.