Yaxin’s Blog

The Grader Is the Optimizer

We let an AI agent improve another agent’s harness. Then a stricter grader re-scored the same pages, and every measured gain shrank by a third to 70%. On open-ended tasks, what an optimizer learns depends on the grader, and is often about it.

Inside

Figure 1

Figure 2

Figure 3

Outline

1. 01How the loop works
2. 02Change the grader, and the gain shrinks by a third to 70%
3. 03Run-to-run noise is as large as your keep bar
4. 04Show the optimizer failed rubric items, and it copies them
5. 05Reward hacking happened, just not as cheating
6. 06Almost all the gain is the first edit
7. 07Watch the loop: the dashboard
8. 08If someone tells you their agent improves itself
9. 09What I would tell someone building this loop