What the harness knows that the model doesn’t

Every agent that survives a multi-hour task is wrapped in something. A task list it keeps returning to. A rule that says don’t stop because a milestone felt like a good place to report. A verifier that refuses to accept “done” without evidence. We call this the harness, and for hard tasks it often matters more than the model inside it.11“Harness” here means everything around the model call: the loop, the tools, the prompts, the files the agent reads and writes.

That raises a question I keep coming back to: how much of the harness is knowledge the model could hold itself?

01#Three kinds of scaffolding

Looking at the harnesses I’ve written over the past year, the pieces fall into roughly three groups, nested around the model like the layers of an onion.

1. Memory: checklists, scratch files, summaries that survive a context reset. These compensate for a finite window.
2. Discipline: rules about when to stop, when to verify, when to ask. These compensate for habits the model learned elsewhere.
3. Judgment: routing a subtask to the right tool, deciding that a result is good enough. These encode taste.

A harness rule is a correction written down once and paid for on every call.

Memory seems like infrastructure; it will probably always live outside the weights. Discipline is the interesting middle. Judgment is where it gets hard.

02#Pricing a rule

One way to see why internalising matters: every rule in the prompt costs tokens on every call, and attention spent reading the rule is attention not spent on the task. If a harness has $k$ rules of average length $\ell$, and an episode makes $T$ calls, the overhead is roughly

For a long run ($T$ in the hundreds) that adds up fast. A rule the model has internalised costs nothing at inference time.

03#What removal might look like

The experiment I want to run: start with a harness that works, delete one rule at a time, and measure how often a long task still finishes correctly, first for the base model and then for a model trained on trajectories from the full harness.

If the trained curve stays flat while the base curve falls, the rules on the flat stretch have moved into the weights.

04#A tiny example

The simplest discipline rule I use looks like this:

def should_continue(turn, tasks, attempts):    # A text-only end of turn is a report, not proof of completion.    if turn.stop_reason == "end_turn" and tasks.open():        return attempts < 3    return False

It works. But it is also a sign that the model, left alone, believes it is finished before it is. If a model internalised this, the rule would become dead code.

05#Which rules move first

This is a placeholder post written to preview the blog layout. Real writing will replace it.