Local-first evidence for coding agents

Evidence before edits.

Open Kioku is a local index your coding agent reads before it edits. It shows its evidence — which files, which streams answered, what is missing — declares an edit boundary in a plan, and verifies the finished diff against that plan. Nothing leaves your machine.

✓ Read-only by default✓ No hosted code index✓ MCP over local stdio
16MCP tools in 4.0.0, one question each, down from 58
4real repositories; retrieval re-scored nightly on hosted runners
0 of 5no-gold tasks presented as answers on the frozen fixture
0.02–0.05sexact class lookup, fresh process; 3.1.0 scale record, one workstation
What to expect

The floor your agent starts from, measured on real repositories.

Retrieval is scored on the production ok context path — the same builder behind ok plan and the MCP build_context_pack tool — on four repositories indexed at a fixed base commit. Each case is a later commit: the query is its subject line, the answer is the source files it changed. Both splits are gated nightly; the table shows holdout.

CorpusHoldout casesR@5R@20MRR
Java, about 10k files1130.5660.7080.516
Go application, ~800 files840.6900.8210.539
TypeScript, ~900 files1670.8260.8800.683
Python library, ~4k files1990.6630.7540.543
R@5 — the share of tasks for which at least one file the commit changed is in the first five results.
R@20 — the same within the first twenty results, roughly the whole context pack.
MRR — the average of 1 / rank of the first correct file; 1.0 means it was always first, 0.5 is what you get if the first correct file were always second, or first half the time and never found the rest.
Read it plainly: on a Java repository of about ten thousand files the right file is in the top five about half the time and in the pack about seven times in ten; on a TypeScript repository of about nine hundred files, in the pack nearly nine in ten and in the top five about four in five. That is the floor the agent starts from; exact lookups and the plan → edit → verify loop sit on top of it. Baselines were frozen on 2026-09-25 from one hosted Linux runner matrix run of commit bd5d06ab, confirmed by a second run of the same commit with identical scores, and are re-derived nightly against the frozen files in benchmarks/commit-derived/ by the scripts in the repository; the job fails when a watched metric drops more than 0.03 below its baseline, or when a routed task family of 34 or more cases, with unchanged membership, drops more than max(0.03, 2/n) below its own. Queries are commit subjects, not issue text, so these numbers are not comparable with published benchmarks. Method, corpora, both splits, and scripts ↗
What 4.0.0 changed

Fewer tools, each one true to its name. An index that says what it skipped.

4.0.0 shipped on 2026-09-11. It changes the index format (run ok index after upgrading) and cuts the MCP surface to sixteen tools. Every figure below has its method in CHANGELOG.md or the linked doc, and names its commit and artifact where one exists; the retrieval figures are local-workstation A/B runs, not the nightly hosted baselines above.

16

16 MCP tools, down from 58

Every advertised tool answers one question no other tool answers. Twenty-three names folded into the sixteen with nothing lost; thirteen moved to the CLI and still ship; one (structural_search) was removed because no structural matching existed, only the name. A retired name answers with where its capability went, not a bare refusal. Six descriptions that said what their names suggested now say what the implementation does.

docs/mcp-tools.md · migration table for every retired name
RE

regex_search now does regex

It advertised regular-expression matching and dispatched to ranked lexical search, so an agent that asked for a pattern got guesses with nothing in the response saying so. It now compiles the pattern once, evaluates it line by line over indexed text, reports how many files it scanned, and says when it stopped early. ok search <pattern> --regex makes the same call.

exact single-line hits at confidence 1.0 · index-only corpus caveat kept
OLD

A stale index refuses; it does not guess

Opening a pre-4.0 index withholds relationship evidence and answers run ok index to rebuild. ok impact, ok plan, ok context and the MCP tools built on them refuse rather than answer from an empty graph, because an empty answer reads as “no such relationship exists”. Search and symbol lookup keep working meanwhile.

ok doctor fails the graph check · repo_status reports graph_rebuild_required
COV

The index reports what it did not index

Per-language coverage in ok index, ok doctor, ok status and repo_status, with every omission attributed to a skip reason. An ingest rule had silently dropped 25 Java source files from one repository, and nothing surfaced it. Source files whose path contains “secret” are now indexed with their parser messages redacted; generated files are indexed, flagged, and ranked last unless the task names them.

docs/ranking.md · docs/security-model.md
RGN

The right file, and now more of the right region

Packs showed the right file but the wrong cut: selected units covered 3–22% of the lines the real commit changed. The top three files now widen to the enclosing symbol and adjacent chunks. Share of changed lines shown within 8k tokens: Java 0.216 → 0.248, Go 0.207 → 0.299, TypeScript 0.155 → 0.335, Python 0.130 → 0.203 (626 paired cases, no case worse, about three times the tokens; the Go corpus reads high because 50 of its cases are near-identical release bumps).

benchmarks/commit-derived/region-widening-ab.json · docs/ranking.md
ID

Task words reach the repository’s own identifiers

ChannelsUtils Tests reaches ChannelUtilsTests; a misspelt part the repository spells nowhere is corrected by one edit. No model, no network, no re-index. Neutral on commit-subject benchmarks by construction; on 259 perturbed queries, R@5 0.656 → 0.699 and MRR +0.036 (95% CI +0.015 to +0.062). Read that as an upper bound: the perturbations are the inverse of the lattice’s own hypothesis, and this measurement has no committed artifact.

docs/ranking.md · identifier_lattice_anchor_boost · no committed artifact
Also in 4.0.0: derived-file edges (a generated file and its origin, a test and the module it is named after) join impact analysis as labeled possibilities, a declared origin carrying its proof and a naming convention marked heuristic; the graph tables store typed columns and a string dictionary instead of a JSON document per row; and every confidence score is re-weighted, so a 3.1.0 pack is not comparable to a 4.0.0 one. Upgrade notes and the full list: CHANGELOG.md ↗
Scale record · 3.1.0 lineage · 2026-08-31

Repository intelligence that survives contact with real code.

The most recent end-to-end scale record validated the 3.1.0 release lineage (source 3959fdf) on a large Java repository — same protocol and same hardware as the 3.0.4 record; the before/after figures compare against main at c96f61a on the identical corpus. 4.0.0 changed the index storage format and has not been re-run on this corpus; these numbers describe 3.1.0.

Cold structural index
19m28s
13,607 indexed files, 247,499 symbols, 248,107 chunks, 402,844 graph nodes, and 1,522,135 relationship edges — a corpus 46% larger than the prior record. A repeat rebuild reproduced identical totals.
0.02–0.05s
exact class lookup, fresh process — down from 13.9s on main at c96f61a, same host as the 3.0.4 record
495,606
local semantic vectors · 0 failures
84,504
tests indexed and tiered by evidence
0
lock failures across four concurrent graph readers
Measured, not promised: timings come from one local workstation and vary with hardware, repository shape, Git history, and enabled evidence. Four concurrent graph reads completed without lock failures; optional compiler-grade SCIP evidence was unavailable for this build and remained explicitly reported as missing. Inspect the JSON evidence ↗ Read the method and caveats ↗
Built to be trusted

The parts we can prove, we prove. The parts we can’t, we publish.

Every claim on this page traces to a frozen corpus and a machine-readable artifact in the repository — including the numbers that are not good yet.

ABS

When the task has no answer

On the frozen 30-case fixture, none of the five no-gold tasks is presented as an answer: the no-gold false-positive rate is 0.0 (the CI ceiling is 0.25), and the advisory routed context-pack strategy in the same run returns none of them above Low. A low-confidence pack still lists candidates — it tells the caller not to trust them rather than returning nothing. Calibrated abstention is built and gated on held-out performance, and off by default until it earns activation.

benchmarks/retrieval-baseline.json · benchmarks/retrieval-thresholds.json
TIER

Proof-tiered claims

A file that merely shares a name can never become a “proven dependency” or a “required test.” Structural claims carry typed proofs; heuristic candidates stay clearly labeled suggestions.

retrieval may speculate · truth may not
GEN

Atomic index generations

Every index builds into its own generation and publishes with one atomic switch — designed so a reader never sees a half-built index; a kill mid-build or a downgrade falls back to the last good generation.

atomic publish · last-known-good recovery
Release-linked evidence

Don’t take the marketing copy on faith. Inspect the proof.

The evidence below identifies the exact build lineage, publishes aggregate results, and keeps caveats beside the claims they qualify. The scale record is from the 3.1.0 lineage; 4.0.0’s own measured changes are listed above with their commits and methods.

Scale record · 3.1.0 lineage · 2026-08-31

The 3.1.0 lineage, tested at scale.

Validated end to end on a large Java workload — and honest about the result. The adjacent artifacts are the source of record; the replay on the left shows output in the shape the CLI prints it.

3959fdf source16,537 tracked files0 semantic failures
Machine-readable evidence

Auditable without exposing the repository.

The JSON records build source, host profile, configuration, workload totals, measurements, quality checks, and limitations — including what still isn’t perfect. Repository identity is withheld, so exact replay is not claimed.

247,499 symbols1,522,135 edges4 parallel readers
Inspect JSON evidence ↗
Method + limitations

Quality means preserving uncertainty.

The repeated structural run reproduced identical totals, the exact class lookup returned the correct class in 0.02s, and missing compiler-grade SCIP evidence stayed visible instead of being treated as present.

0 lock failures0 failed vectors
Read methodology ↗
Evidence boundary: these are single-workstation observations, not latency guarantees. Publishing an anonymous workload protects the test target but prevents exact source-level reproduction.
One evidence pipeline

From repository facts to the smallest useful context.

Open Kioku combines deterministic code intelligence with explainable retrieval signals. Exact repository facts do not get demoted by a fuzzy match.

01
Indexsymbols, chunks, tests, graph
02
Retrievelexical + local semantic context
03
Provereferences, history, runtime, coverage
04
Planfiles, boundaries, validation
05
Verifychanged files against the plan
exact symbolsreferencesBM25local semanticimpact graphtestsgit historyruntimecoveragearchitecturecontracts
EX

Facts outrank guesses

Definitions, references, paths, and exact evidence stay authoritative. Heuristics can help retrieval without rewriting repository truth.

authority ≠ score
CTX

Context with a purpose

Ranked snippets, exact line ranges, impact, nearby tests, and caveats give the next agent step a bounded evidence set.

search → context → plan
ANN

Scales locally

Optional semantic retrieval stays on-device and can use persistent approximate nearest-neighbor search while exact-flat remains the correctness oracle.

local semantic · no hosted index
HIS

History is evidence

Co-change, churn, ownership, reviewer, provenance, renames, and similar-change signals are derived from bounded local Git history.

local history enabled by default
VAL

Tests before edits

Validation candidates are selected from indexed tests and repository evidence before a change begins, not as an afterthought.

plan with validation targets
VFY

Verification closes the loop

Plans define edit boundaries. Verification compares the actual changed files with those boundaries and reports evidence caveats.

plan → edit → verify
The loop, recorded

Plan, edit, verify. One edit outside the boundary fails the check.

A terminal recording against a 39-file fixture: ok setup agent indexes and wires the agent, ok context shows evidence and confidence, ok plan writes the boundary, then one edit inside it and one drive-by outside it. ok verify fails on the drive-by. Rendered from demo.tape; anyone with ok installed can re-record it.

Terminal recording: ok setup agent indexes a repository, ok context shows evidence and confidence, ok plan writes a boundary, an edit outside the boundary makes ok verify fail.
Interactive product tour

A coding workflow with memory of what matters.

Click through the same sequence an agent can use before, during, and after a repository change. The verify step is the attested dogfood run in verification-dogfood.json, produced by ok 2.4.0. The 12-second recording under the tour shows that step on its own.

ok plan
Terminal recording: the allowed files from plan.json, one edit inside that boundary, one edit outside it, then ok verify prints Verification: Fail, reports [out_of_boundary] go/shipping/carrier.go, and exits 1.
Local by design

Your repository is the source of truth — and it stays yours.

The default workflow is designed for local repository intelligence rather than a hosted code-index service.

No hosted index

Core search and evidence are stored under the repository’s local .ok/ data.

Read-only by default

The MCP server provides evidence tools; source edits remain in the normal editor and agent workflow.

Network-denial posture

Command execution and model downloads are policy-gated. Default configuration includes network denial, and denial fails closed. Local semantic search needs no hosted embeddings API.

Share proof, not source

ok prove reports index and validation signals while intentionally omitting source snippets.

Evidence stays honest: runtime traces and coverage reports are optional local inputs; when evidence is missing, Open Kioku reports the caveat instead of pretending certainty.
Start with your own repository

Give your coding agent evidence before the next edit.

Install from the channel you already use, run one setup command, then ask for evidence on the task you actually care about.

$ npm install -g open-kioku
npm
npm install -g open-kioku

Pulls the native binary for macOS, Linux, or Windows through platform packages.

crates.io
cargo install open-kioku-cli

Or cargo binstall open-kioku-cli for the prebuilt binary.

GitHub releasesok-{linux,macos,windows}-*

Binaries with SHA256SUMS, an SBOM, PROVENANCE.json, and build-provenance attestations. Release trust ↗

Claude Code pluginclaude_plugin.json

Plugin manifest and .claude-plugin/ marketplace entry in the repository.

Cursor / Codex plugins.cursor-plugin/ · .codex-plugin/

Agent, rules, and skill for Cursor; plugin manifest for Codex.

MCP directoriesglama.json · smithery.yaml

Server manifests for the Glama and Smithery MCP directories.

Setup guidesok setup agent claude|cursor --repo . --apply

Claude Code · Cursor · Codex · Gemini CLI

If Open Kioku improves your agent workflow, consider starring the repository ↗.