Home / Blog / Benchmarks

Benchmarks

Finder 1.0.0 speed benchmark: 0.5–0.7 s new questions, 35 ms repeats

On two real repositories, Finder 1.0.0 answered a never-seen question in 484–708 ms on average and a repeated one in about 35 ms, against 650–954 ms and 729–1,135 ms for another local context-engine MCP server. Recall was level (0.85 vs 0.82 and 0.73 vs 0.73), Finder's output was less than half the size, and it used 20–40% less memory. A grep-only baseline read about 24× more tokens for 0.29 recall. Finder was slower on 2 of 12 first asks.

Published October 10, 2026 · 5 min read

Method

Apple M1 Pro, 10 cores, October 9, 2026. Both servers ran locally and were called over their MCP HTTP endpoint, the way a real client calls them. Finder was the build that shipped as 1.0.0, with default settings. The other engine is a local context-engine MCP server of the same kind as Finder: embeddings plus a call graph, running on your machine.

Each question has ground-truth files checked by hand or by grep. Recall is the share of those files returned, precision the share of returned files that were expected, and primary hit means the main file appeared at all. Every question was asked once fresh ("new question"), then three more times ("repeat question").

Flutter app, 4 flow questions

A production Flutter codebase of about 1,400 files. The questions ask for a flow that spans several files, not for a symbol by name.

Rust codebase, 8 questions

The Finder repository itself (Rust + TypeScript).

Treat the CPU row with care: the other engine was re-indexing during this run, which inflates its figure. The Flutter table is the cleaner CPU comparison.

Against an agent with only grep

On October 11 we re-ran the eight Rust questions three ways: Finder 1.0.0, the other engine, and a grep-only baseline that searches and reads files the way an agent without an MCP server does.

Both engines found the main file every time in one call; Finder's answer was about half the size, and the grep-only baseline read roughly 24 times more tokens for less than half the recall. The other engine was more precise in this run (0.28 vs 0.18). Timings from this run are left out: Finder had already answered these questions, so its 41 ms median is the repeat path, not a fair first-ask number.

Cold start

Six fresh questions, each sent right as a session opened. Finder took 0.46–1.09 s, mean 0.61 s; the other engine took 0.53–1.10 s, mean 0.81 s. Finder was faster on 5 of the 6.

Why repeats are so much faster

Finder keeps the result of a question it has already answered on an unchanged index, so a repeat costs about 35 ms. The other engine recomputes each time. Agents do repeat themselves — after a context reset, or when a subagent asks what the parent already asked — but a new question is the fairer comparison, which is why both rows are shown.

The trade-off behind the default

The finder tool no longer makes a second, serial rerank call by default. Turning it back on buys a little recall for a lot of latency:

To opt in, set "finder_query_rerank": true in ~/.finder/settings.json. file-finder and multi-part questions keep reranking either way.

What changed to get here

• A readiness check was reading an empty legacy store, which added an index run and a 500 ms wait to every query. It now reads the current store.
• Questions containing / : ? ( broke the keyword-search parser, so the keyword half of hybrid search came back empty. The query is now sanitized.
• The embedding connection is warmed when a repo worker starts and kept warm while Finder is in use.
• The query embedding is hedged: a second request goes out after 600 ms if the first has not returned.

Read before trusting

• Finder lost twice. On 2 of the 12 first asks the other engine was faster (479 vs 582 ms, and 735 vs 1,041 ms).
• Recall is a tie, not a win. 0.73 vs 0.73 on the Rust repo, 0.85 vs 0.82 on Flutter with only four questions.
• Small sample. Twelve questions on two repositories, one machine, one day.
• Our questions, our repos. We wrote the questions and tuned Finder on these repos, so treat the gap as an upper bound.
• Not comparable with earlier posts. Different build and different default settings.

Frequently asked questions