Telemetry & Metrics

Agents are distributed systems in miniature: model calls, tool executions, retries, streaming turns. Kova gives you two complementary ways to observe them — tracing (structured spans, optionally exported to OpenTelemetry) and a lightweight metrics collector that’s always available.

Tracing

Kova instruments its async operations with tracing spans. Every agent turn carries attributes like llm.input_tokens, llm.output_tokens, llm.stop_reason, and llm.iterations, and each provider and tool call gets its own span.

Without the telemetry feature (zero overhead)

TelemetryConfig::init() works out of the box. Without the feature it installs a plain tracing_subscriber — you get structured logs to stderr and no OpenTelemetry dependencies compiled in:

use kova_sdk::telemetry::TelemetryConfig;

TelemetryConfig::builder()
    .log_level(tracing::Level::DEBUG)
    .build()
    .init()?;

This is the right default for most apps: readable logs, nothing heavy.

With the telemetry feature (OpenTelemetry export)

Enable the feature to export spans to an OTLP collector, Jaeger, or stdout:

kova-sdk = { version = "0.9", features = ["telemetry"] }

use kova_sdk::telemetry::{TelemetryConfig, ExporterConfig, OtlpProtocol};

TelemetryConfig::builder()
    .service_name("my-agent")          // sets the OTEL service.name resource
    .log_level(tracing::Level::INFO)
    .exporter(ExporterConfig::Otlp {
        endpoint: "http://localhost:4317".into(),
        protocol: OtlpProtocol::Grpc,  // or Http
    })
    .sampling_rate(0.5)                // sample 50% of traces
    .build()
    .init()?;

The API is identical whether or not the feature is on — only the backend changes. So you can develop against plain logs and flip on OTLP in production by enabling one feature and adding an exporter.

Why it’s feature-gated

The OpenTelemetry crates are heavy (compile time and binary size), and most embedders don’t need full OTEL. Gating them keeps builds lean by default while leaving the door open for production observability.

Metrics collector

Separate from tracing, MetricsCollector is an always-available, no-feature-flag counter for quick introspection — request counts, token totals, error counts, tool durations. Attach one to the agent and it records automatically:

use std::sync::Arc;
use kova_sdk::telemetry::MetricsCollector;

let metrics = Arc::new(MetricsCollector::new());

let agent = AgentBuilder::new()
    .provider(provider)
    .metrics(metrics.clone())    // agent records LLM latency/tokens/errors + tool durations
    .build()?;

// … run some turns …
agent.run(&history).await?;

// Read the counters any time, from any thread:
println!("LLM requests: {}", metrics.llm_request_count());
println!("total tokens: {}", metrics.total_tokens());
println!("LLM errors:   {}", metrics.error_count());

You can also record into it manually (e.g. from your own tools or code paths):

metrics.record_llm_request(150.0, 100, 50);  // latency_ms, input_tokens, output_tokens
metrics.record_tool_invocation(25.0, true);  // duration_ms, success
metrics.record_llm_error();

Design notes: it uses atomic integers and fixed-bucket histograms (constant memory — it won’t grow unbounded under load). Snapshot getters return a HistogramSnapshot. It intentionally does not feed OTEL metrics — think of it as a lightweight, always-on dashboard, not a full metrics pipeline.

Tracing vs. metrics — which do I use?

	Tracing	MetricsCollector
Feature flag	optional telemetry for export	none — always available
Granularity	per-span, per-operation detail	aggregate counters/histograms
Best for	debugging a specific turn; distributed traces	dashboards, health checks, quick totals
Overhead	logs (light) / OTEL (heavier)	negligible atomics

Use both: metrics for at-a-glance health, tracing when you need to see exactly what one turn did.

A production setup

// One-time at startup
TelemetryConfig::builder()
    .service_name("support-bot")
    .log_level(tracing::Level::INFO)
    .exporter(ExporterConfig::Otlp {
        endpoint: std::env::var("OTLP_ENDPOINT").unwrap(),
        protocol: OtlpProtocol::Grpc,
    })
    .sampling_rate(0.1)
    .build()
    .init()?;

let metrics = Arc::new(MetricsCollector::new());
let agent = AgentBuilder::new()
    .provider(provider)
    .metrics(metrics.clone())
    .build()?;

// Expose metrics.llm_request_count() etc. on a /healthz endpoint.

Key takeaways

• Tracing works without any feature (plain tracing_subscriber); enable telemetry to export to OTLP/Jaeger/stdout with the same API.
• MetricsCollector is always available (no feature flag); attach with .metrics(...) and the agent records requests, tokens, errors, and tool durations automatically.
• Use metrics for aggregate health, tracing for per-turn debugging.