Embeddings

Embeddings turn text into fixed-length vectors whose distance reflects semantic similarity — the foundation of retrieval-augmented generation (RAG), semantic search, clustering, and deduplication. Kova exposes a small EmbeddingProvider trait plus two implementations.

Kova ships no vector store

Embeddings are an input seam, not a full retrieval stack. Chunking, indexing, storage, and nearest-neighbour search belong to the host application (or a dedicated vector database). Kova gives you the vectors; what you build on top is yours.

The EmbeddingProvider trait

#[async_trait]
pub trait EmbeddingProvider: Send + Sync {
    /// Embed `texts`, returning one vector per input, in the same order.
    async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>, KovaError>;

    /// Vector dimensionality when known up front; `None` = discover from the
    /// first call.
    fn dimensions(&self) -> Option<usize> { None }
}

embed batches server-side where the API allows it, and inputs map 1:1 onto output vectors. It’s re-exported in the prelude, so Arc<dyn EmbeddingProvider> lets you swap providers the same way LlmProvider does for chat.

OpenAI-compatible embeddings

Backed by any OpenAI-compatible /v1/embeddings endpoint (feature openai):

use kova_sdk::embedding::openai::OpenAiEmbeddingProvider;

let embedder = OpenAiEmbeddingProvider::new("https://api.openai.com", "text-embedding-3-small")
    .with_api_key(std::env::var("OPENAI_API_KEY")?)
    .with_dimensions(256);   // optional: reduced-dimension vectors (text-embedding-3 models)

let vectors = embedder.embed(&[
    "Rust is a systems programming language.".to_string(),
    "The Eiffel Tower is in Paris.".to_string(),
]).await?;

assert_eq!(vectors.len(), 2);

with_dimensions(n) requests shorter vectors from models that support it (cheaper storage, faster search); omit it for the model’s native size.

Ollama embeddings

For a local (or remote) Ollama server’s /api/embed (feature ollama), no API key required:

use kova_sdk::embedding::ollama::OllamaEmbeddingProvider;

let embedder = OllamaEmbeddingProvider::new("http://localhost:11434", "nomic-embed-text")?;
let vectors = embedder.embed(&["hello world".to_string()]).await?;

A minimal retrieval loop

With vectors in hand, similarity is just a dot product on normalized vectors. Here’s the whole “find the most relevant document” step, host-side:

fn cosine(a: &[f32], b: &[f32]) -> f32 {
    let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
    let na: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
    let nb: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
    if na == 0.0 || nb == 0.0 { 0.0 } else { dot / (na * nb) }
}

// 1. Embed your corpus once and store the vectors (in a DB, a vector store, or memory).
let docs = vec![
    "Kova is an async Rust library for LLM agents.".to_string(),
    "Paris is the capital of France.".to_string(),
];
let doc_vectors = embedder.embed(&docs).await?;

// 2. At query time, embed the question and rank by similarity.
let query = embedder.embed(&["What is Kova?".to_string()]).await?[0].clone();
let best = doc_vectors.iter()
    .enumerate()
    .max_by(|(_, a), (_, b)| cosine(&query, a).total_cmp(&cosine(&query, b)))
    .map(|(i, _)| &docs[i]);

// 3. Feed the retrieved text to the agent as context.
let prompt = format!("Using this context, answer the question.\n\nContext: {}\n\nQ: What is Kova?", best.unwrap());
let response = agent.run(&[user_message(&prompt)]).await?;

For anything beyond a handful of documents, store the vectors in a purpose-built index (pgvector, Qdrant, LanceDB, …) rather than scanning in Rust — but the shape stays the same: embed, store, embed the query, rank, inject the top hits into the prompt.

Feature flags

Provider	Feature	Endpoint
OpenAiEmbeddingProvider	openai	/v1/embeddings
OllamaEmbeddingProvider	ollama	/api/embed

Both features are on by default. They’re the same flags that enable the corresponding chat providers.

Key takeaways

• EmbeddingProvider::embed(&[String]) -> Vec<Vec<f32>> turns text into vectors, batched and order-preserving.
• Two implementations: OpenAiEmbeddingProvider (with_dimensions for reduced vectors) and OllamaEmbeddingProvider (local, no key).
• Kova ships no vector store — chunking, indexing, and search are the host’s job; embeddings are just the input seam for RAG.