Skip to content

Live package

pi-legwork

Delegation layer that hands read-only research from Claude Code or Codex to a cheaper agent, returning the answer instead of the search trail

Measuring a delegation tool surfaced eleven defects that code review could not see. Seven were in the measurement, and one of them withdrew the headline ratio the tool was built to prove

11
Defects Surfaced
7 of 11
In The Measurement
Withdrawn
Headline Ratio
Read-only
By Profile

Reading notes

What It Does

A coding agent answering “which files reference the retry helper, and what for” reads about forty files to produce ten rows. The ten rows are worth keeping. The forty reads are not, and they sit in the agent’s context for the rest of the session, crowding out the work you actually care about.

pi-legwork moves that reading to a second, cheaper agent. The delegate works in its own session and writes its full answer to a file. What returns to Claude Code is a short preview and a path.

Claude Codeor Codexcontext stays cleanpi agentcheaper model40 reads, greps, dead endstask10 rows and a file paththe reads never cross back

The Two Gates

A task is worth delegating only if both hold.

Churn much greater than output. Many reads, short answer. The answer returns either way, so only the churn can be saved. This is economics, not capability: a stronger delegate does not change it.

The output carries its own evidence. It can be checked without redoing the work, through paths, line numbers, commit hashes, source URLs, page numbers.

The second gate is the one people miss. “Is this analysis sound” reads twenty files and returns a paragraph. Excellent ratio, terrible delegation, because the paragraph cannot carry what you would need to believe it, so checking means reading the twenty files yourself.

What The Measurement Found

The build rule was fixed before any data existed: ship it only if delegation came in under 0.6 times direct on one task class, with equal answers. One grep-heavy class cleared it, at roughly a quarter of the caller’s tokens.

That ratio is withdrawn rather than corrected. A transcript records one model response once per content block, so summing the rows counts the same usage several times over, by a factor that varies per run. Every arm was inflated by a different amount, which is the worst kind of error: it does not look like noise, it looks like a finding.

Eleven defects surfaced only because something was being counted, and the split between them is the lesson. Four were in the tool:

  • The second-opinion mode silently loaded the project’s context file, so it held half the caller’s assumptions and agreed with them.
  • A token counter inflated its own headline by summing a cumulative field that folds in cache counters.
  • One tool profile had never run at all, because macOS ships bash 3.2 and it aborts on expanding an empty array.
  • Thirty megabytes of session files had accumulated from a tool with one-shot semantics.

Seven were in the measurement, and those were worse, because a broken benchmark reports a number instead of an error:

  • A pass or fail check read through a pipe returned the pipe’s exit status, so every arm would have scored as passing.
  • Cost was attributed by diffing a global counter, which quietly adopted an unrelated session. One arm came back billed for two million tokens across a hundred tool calls, including tools it had never been granted. A contaminated arm does not look wrong, it looks surprising, which is how a wrong number survives.
  • Transcript rows were summed without deduplicating, the error that withdrew the ratio above.
  • A delegation launched in the background scored as work never done, because the non-interactive mode has no later turn to collect it in.
  • Three separate parsing bugs in the answer scorer, counted here as three, all penalised the answer that explained itself and rewarded the terse one.

None were visible by reading the code. The documentation and the artifact disagreed, and only a canary and a token count could tell them apart. The measurement turned out to be harder to get right than the thing it measured.

What Cost Actually Tracks

Turns, not work.

A caller re-reads its entire resident context on every turn, so a turn costs roughly what it costs regardless of what it accomplished. Separating the parent’s transcript from its delegate’s made this concrete: across eight runs the parent’s cost stayed flat to within one percent, while everything that varied happened inside the delegate.

Delegation therefore wins by ending the conversation sooner, not by moving file reads elsewhere. That explains the failure mode that matters most in practice: one round of double-checking the delegate costs more than the delegation saved. It also means any figure here belongs to one caller with one context size, and a leaner setup would produce different numbers from identical work.

Codex Works, But Does Not Save

Given a short block pasted into AGENTS.md, Codex delegates on its own, picks the enforced read-only profile itself, and quotes the gates back before acting. Answers were correct in every run.

It saved nothing. On a greppable survey the ratio came out near 1, because ripgrep answers that question in two tool calls and there is no churn to move. On a read-heavy classification it was nearer still, because the answer is a list of all sixty modules, so the output is nearly as large as the work. Both were counted on the basis withdrawn above, so they are directions rather than measurements.

Both results are the gates working rather than the tool failing, and the repository says so instead of implying it helps everywhere.

Honest Limits

The guard runs inside the delegate’s own process. It is defence in depth, not a sandbox. The default profile carries a shell, so read-only describes the tasks you send rather than a boundary the tool enforces. A separate profile makes it enforced. A shell also reaches past the tool list it was granted, to whatever the machine can reach.

Savings depend on the caller being expensive or quota limited while the delegate is cheap. Context isolation does not, but costs latency.

Against Claude Code’s own subagents, later runs did separate the two, in this tool’s favour. The mechanism is dull. A subagent works on the caller’s own quota, where it is counted. A delegate does the same work on another provider, where this meter cannot see it. Same job, different bill. That is the asymmetry the tool exploits, not a claim that either agent is better, and the direction replicated where the size of the gap did not.

Published as a case study with the code attached, not a maintained product. The harness stays unpublished: its numbers belong to one machine, and shipping a corpus mostly invites people to quote a ratio that was never theirs.