Research
Zorp is built on one claim: an answer is only worth as much as the record behind it. This page is that argument in full, along with every number we have actually measured.
It is the long version on purpose. Nothing here is a summary of something else, and the results section publishes the runs that did not resolve alongside the ones that did.
A confident answer is not a defensible one
An LLM will produce a fluent answer to a hard question in seconds. What it will not do is tell you whether to believe it, what evidence it weighed, or what it found that pointed the other way.
Zorp treats that gap as the actual problem. A question becomes an investigation, an investigation produces an evidence record, and the record is what the answer is accountable to.
Two paths, one question
The Kill Threshold
A number, supplied by a human, that says in advance what would prove this wrong.
One investigation, start to kill
Before zorp gathers anything, it commits the hypothesis, the metric, and the kill threshold to git. The file is human-readable and hash-verified on load, so a run cannot quietly rewrite what it set out to test.
The agent never proposes the threshold. A human does, and only a human can move it. Every attempt is recorded, not just the one that worked. When a run crosses the line, the record says why it was killed.
tracks/kafka-migration/prereg.toml
hypothesis = "Replacing Kafka with NATS cuts p99 publish latency"
metric = "p99_publish_latency_ms"
threshold = { direction = "decrease", min_delta_pct = 20 }
registered = "2026-08-11T09:14:22Z"
sha256 = "e3b0c44298fc1c149afbf4c8996fb924..."Four capabilities, one loop
The loop and its human gates
01
validate
Is this question worth investigating? Scored on two dimensions, redundancy and feasibility, each of which must cite retrieved evidence or score zero.
02
investigate
Gather evidence through staged, pre-registered attempts. One attempt per invocation, so every attempt surfaces at a checkpoint.
03
co-write
Draft the artifact from the evidence record. The model is handed only what the track recorded and is instructed to cite only those figures.
04
deliver
Match the finished artifact to the right format and audience.
A human checkpoint sits between each step. There is no non-interactive mode, because a research checkpoint has no safe default. The human is always the author of record.
The evidence record
Every investigation writes to a six-table store. Metrics are typed key-value pairs, not narrative logs, so a claim in the finished report resolves to a row rather than to a paragraph an agent wrote about itself.
This is what the answer is accountable to. It is also what makes the answer checkable by someone who was not there.
Evidence record schema
- tracks
- One investigation. Its question, its hypothesis, its status.
- preregistrations
- The committed hypothesis, metric, and kill threshold, with the hash that makes them tamper-evident.
- experiments
- Every attempt, including the ones that conflicted and the one that ended the run.
- metrics
- Typed key-value measurements, such as accuracy: 0.87.
- checkpoints
- Each human yes or no, and when it was given.
- validations
- Redundancy and feasibility scores, each with the citation that justified them.
The loop does not know what domain it is in
Nothing in pre-registration, evidence recording, or grounded drafting is specific to software. The loop needs a question and it needs sources. What changes between industries is the sources, not the method.
These are examples of questions the loop is shaped for, not products we ship for each industry.
| Hi-tech | Does replacing this message broker actually cut tail latency, or just move it? |
|---|---|
| Healthcare | Does the published evidence support this care-pathway change for our patient mix? |
| Life sciences | Which of these four targets has prior art strong enough to justify a program? |
| Fintech | Does this fraud signal survive out of sample, or did we fit it to last quarter? |
| Banking | What is the defensible basis for this exposure estimate, line by line? |
| Real estate | Do the comparables actually support this valuation once you control for the obvious? |
| Automotive | Does this supplier change alter the failure profile in a way the warranty data would show? |
| Retail | Did the promotion drive incremental demand or pull it forward? |
| Space | Which of these mission-profile assumptions has never been tested against flight data? |
| Manufacturing | Is this yield improvement real, or is it the measurement changing? |
| Federal | What evidence underpins this assessment, and what pointed the other way? |
| Law | Does this line of authority hold once you read the cases that cite it? |
| Media | Can every claim in this story be traced to a source a reader can check? |
What we've measured, including what didn't resolve
A behavioural-compatibility pilot ran six contracts across 24 tasks, 48 paired runs each, testing whether the harness keeps its behavioural guarantees when the reasoning mode underneath it is substituted. Two contracts came back compatible. Four came back inconclusive, because the reference runtime failed its own eligibility floor.
We are publishing all six. A page about grounded reporting does not get to report selectively. Note carefully what this pilot is: it measures the harness against itself, not against a plain LLM.
Contract negative-flip rate
The shape of the guarantee
- observed non-flip rate
- lower confidence bound
| Contract | CNFR | n | Lower bound | Reference pass rate | Verdict |
|---|---|---|---|---|---|
| diagnose_before_retry | 0.000 | 48 | 0.947 | 48/48 | compatible |
| no_success_before_evidence | 0.021 | 48 | 0.912 | 48/48 | compatible |
| limit_modification_scope | 0.021 | 48 | 0.912 | 7/48 | inconclusive (reference ineligible) |
| inspect_before_modify | 0.042 | 48 | 0.882 | 12/48 | inconclusive (reference ineligible) |
| stop_after_acceptance | 0.083 | 48 | 0.827 | 36/48 | inconclusive (reference ineligible) |
| verify_after_final_change | 0.083 | 48 | 0.827 | 36/48 | inconclusive (reference ineligible) |
Not yet measured
- no groundedness or citation-accuracy evaluation
- no hallucination rate
- no comparison against a plain-LLM baseline
- no stored investigation traces published yet
Where this actually is
Zorp is pre-alpha. The execution harness and the research foundation are built and tested, as are all four capabilities. It is not yet a finished product, and this page will not pretend otherwise. That is why the results section above publishes the runs that did not resolve alongside the ones that did.
- Base execution harness
- Research foundation: tracks, evidence records, pre-registration, checkpoints
- validate
- investigate
- co-write
- deliver
- A published investigation trace, start to finish
- A grounded-vs-baseline evaluation
- A systems paper about zorp itself, submitted to arXiv
Anatomy of the artifact
Most landing pages end with a row of round numbers. This one ends with the codebase itself, measured and broken apart, because a claim about grounded software should be checkable against the software.
Every crate below is a real line count. They sum to the total exactly, and a test fails the build if they ever stop doing so.
Codebase composition by crate
- zorp-agent18,602the harness: tools, sandboxing, the four capabilities
- zorp-track2,623pre-registration, evidence records, checkpoints
- zorp-eval2,095the contract suite these results came from
- zorp-mcp887tool transport, with trust on first use
- zorp460the binary that ties the crates together
- tests298top-level integration tests
- 605
- tests passing
- 97
- commits
- 13
- agent tools
- MIT
- licence
Blogs
All blogs →
Read the evidence record, not the summary
Zorp is MIT licensed and the code is public. It is early and incomplete, and this page has said so throughout. Everything it claims can be checked against the repository.