Home / Blog / Benchmarks

Benchmarks

Finder 0.0.14 vs GitNexus: code search benchmark (October 2026)

On eight hand-checked questions, Finder 0.0.14 found 93% of the right files and the main file every time, in one call of about 2.1k tokens. GitNexus 1.6.9 answered 3× faster (186 ms vs 606 ms median) but found 27%. An agent with only grep read ~146k tokens per question and found 29%.

Published October 5, 2026 · 5 min read

Results

Same eight hand-checked questions as our October 2 run, on the same repository, re-run on October 5 with Finder 0.0.14 and GitNexus 1.6.9. Each tool got one untimed warm-up call per question, then three timed calls.

Recall is the share of ground-truth files that appear in the answer. "Tokens to read" is the response size divided by four; for the no-MCP setup it is the files the agent opened.

Question by question

GitNexus does well when the question uses the same words as the code: P8 names "rerank", which is also in the symbol names, and both tools find everything. It misses when the question describes behavior in other words. P4, P6 and P7 scored 0 for GitNexus because the code says vector, voyage and index-events, not "ranking", "embedding API" or "live progress". Finder embeds the code, so it matches meaning, and then follows the call graph to the callers and tests.

What changed since October 2

• Finder got faster. Median latency fell from 1,313 ms (0.0.11, never-seen questions) to 606 ms here. Part of that is the warm-up call: it fills Finder's embedding cache, so the timed calls skip one remote round trip. Recall and main-file hits did not change (0.93, 8/8).
• GitNexus is unchanged. Same version, same recall (0.27), same main-file hits (4/8), and 186 ms instead of 206 ms.
• Grep is still expensive. The no-MCP agent read about 145,700 tokens per question across ~16 tool calls and found the main file only twice.

When to use which

• Finder for "where is X handled?" and "how does Y work?" questions, where you do not know the file or symbol names yet. One call returns the right files with line ranges.
• GitNexus when you already know the symbol: its graph tools (callers, impact, rename) are fast and precise, and they were not part of this test.
• No MCP only for tiny repos. On this one it cost about 68× more tokens than Finder and still missed most main files.

Caveats, read these

• Home-field advantage. The questions and ground truth are on Finder's own repository, and Finder was tuned on them. Treat the gap as an upper bound and run the script on your own repo.
• Eight questions is a small sample. One question moves recall by about 0.12.
• Warm timings. Every tool got one warm-up call per question. A question Finder has never seen takes longer (about 1.3 s in our October 4 cold run) because it calls a remote embedder and reranker; GitNexus runs fully local.
• Index age. GitNexus was indexed a few commits before the run; none of the changes touched the ground-truth files.

Frequently asked questions