perf(embedder): batch Ollama indexing requests #314

Problem Statement

OllamaEmbedder.EmbedBatch currently loops over its inputs and sends one deprecated POST /api/embeddings request per code chunk. This leaves most local GPU capacity idle and pays HTTP/tokenization/scheduling overhead for every chunk, even though Ollama's current POST /api/embed endpoint accepts an input array.

On an RTX PRO 2000 Blackwell with local nomic-embed-text, indexing the same 59-chunk public Go file took 3.73 seconds and 59 legacy requests. A batched prototype took 1.203 seconds and one modern request. The resulting search path, line range, and cosine score were identical.

Proposed Solution

• Send indexing inputs through POST /api/embed in bounded batches of at most 64.
• Set truncate: false so grepai's adaptive re-chunking remains authoritative instead of silently indexing truncated code.
• Validate response count and reject empty vectors.
• Preserve the failing chunk index when Ollama reports a context limit.
• Fall back once, then remain on POST /api/embeddings, when an older server or proxy reports that /api/embed is unavailable.
• Do not mask model-not-found or general server errors as endpoint-compatibility failures.
• Keep single-query Embed behavior unchanged.

Alternatives Considered

• Parallel legacy requests: adds load and ordering/error complexity while retaining per-request overhead.
• Opt Ollama into the existing cross-file BatchEmbedder path: that path does not preserve Ollama's adaptive per-file re-chunking on context errors. Per-file bounded batching is the smaller safe change.
• Require a new Ollama version: unnecessary when a narrow legacy fallback preserves current installations.

Category

Performance / Embedding Providers

Acceptance Criteria

• Multiple chunks use fewer HTTP requests while preserving output order.
• Inputs are never silently truncated.
• Older endpoint behavior, cancellation, context errors, and model/server errors remain distinguishable.
• Unit/race tests and a real Ollama indexing/search scenario pass.

Contribution

I am contributing the implementation.