Structured Output

Sometimes you don’t want prose — you want data. A router that must return one of a fixed set of labels, an extractor that must produce a well-formed record, a classifier that must emit a score. Kova’s structured output constrains the model’s final answer to a JSON schema and parses it into your own type in a single call.

run_structured

Agent::run_structured::<T> runs a normal turn (tools and all) but requires the final text to validate against a JSON schema, then deserializes it into T:

use kova_sdk::models::ResponseFormat;

#[derive(serde::Deserialize)]
struct Route {
    route: String,
    confidence: f64,
}

let format = ResponseFormat::named("route", serde_json::json!({
    "type": "object",
    "properties": {
        "route": { "type": "string", "enum": ["billing", "technical", "sales"] },
        "confidence": { "type": "number" }
    },
    "required": ["route", "confidence"],
    "additionalProperties": false
}));

let history = vec![user_message("My invoice is wrong and I was double-charged.")];
let (route, response): (Route, _) = agent.run_structured::<Route>(&history, format).await?;

println!("route = {} ({:.0}% sure)", route.route, route.confidence * 100.0);

It returns a tuple: your parsed T and the full AgentResponse, so you still get new_messages to persist, usage for token accounting, and the raw text.

ResponseFormat

A ResponseFormat is a schema plus an optional name:

// Named — some providers (OpenAI) require a schema name.
let format = ResponseFormat::named("route", schema_value);

// Unnamed — the name defaults to "output".
let format = ResponseFormat::new(schema_value);

schema_value is an ordinary serde_json::Value. You can hand-write it with json! (as above) or generate it from your type with a crate like schemars.

Setting it on the config instead

run_structured is the convenient path, but the constraint is just a field on InferenceConfig::response_format. Set it on the agent to make every turn structured, or per call via run_with_config when you want the raw AgentResponse and will parse the text yourself:

let overrides = InferenceConfig { response_format: Some(format), ..Default::default() };
let response = agent.run_with_config(&history, overrides).await?;
let route: Route = serde_json::from_str(&response.text)?;

How it maps per provider

The constraint is translated to each provider’s native JSON-schema mechanism — it’s not prompt-engineering hidden in the system message:

Provider	Mechanism
OpenAI-compatible	response_format: { type: "json_schema", …, strict: true }
Anthropic (native)	output_config.format
Gemini	responseMimeType: "application/json" + a sanitized responseSchema
Ollama	format
Bedrock	no native support — rejects requests that set response_format

Bedrock has no structured output

The Bedrock provider rejects any request that sets response_format. If you target Bedrock, route structured steps to another provider (native Anthropic reaches the same Claude models), or drop the constraint and validate the text yourself.

Keep schemas simple

Stick to the common subset every provider accepts, or a schema that validates on one provider may be rejected on another:

• type: "object" with properties and required
• enum for closed value sets (great for classifiers/routers)
• additionalProperties: false to forbid stray keys
• primitive types: string, number, integer, boolean, array

Avoid exotic constructs ($ref, oneOf, deep nesting, format assertions) — Gemini’s schema sanitizer strips some of them, and support varies. Simpler schemas also steer the model more reliably.

When to prefer a tool instead

Structured output constrains the final answer. If you instead want the model to hand you arguments mid-conversation — to look something up, then keep talking — that’s a tool, whose parameters_schema already constrains its input. Rule of thumb: tool schema for “call this with these fields”, run_structured for “give me the answer as this type”.

Key takeaways

• run_structured::<T>(messages, format) constrains the final text to a JSON schema and returns (T, AgentResponse).
• ResponseFormat::named(name, schema) / ResponseFormat::new(schema) wrap an ordinary serde_json::Value schema.
• Mapped natively per provider (OpenAI/Anthropic/Gemini/Ollama); Bedrock rejects it — route those steps elsewhere.
• Keep schemas to the common subset (object/enum/required/ additionalProperties: false).