4/4 solved on the first iteration with 3 lines of notes — vs 2/4 after burning the full 5-iteration budget, at 10× the wall time.
One model (zai-org/glm-4.7-flash, local LM Studio), one live bench
(ESP32 + BME280 over I2C, MicroPython v1.27.0, USB-UART console), one task
(uart-menu — a level-3 finite-state machine with submodes, scored by a
runner, not by string equality). Two arms, interleaved attempt by attempt so bench
drift hits both arms equally: notes — the task carries a 3-line
expert note block from the human who wired the bench; bare — the same
task without the notes. Each attempt: up to 5 iterations of
«write firmware → stage it onto the live board → read the board's
serial answer → fix». The notes fact is recorded per attempt in
results.jsonl; every attempt's error_kind is clean
(none/run — no infrastructure failures in the series).
| Arm | Solved | Iterations per attempt | Wall time per arm |
|---|---|---|---|
| notes (3 lines) | 4/4 | 1, 1, 1, 1 (70–146 s each) | ~7 min |
| bare | 2/4 | 5, 5, 5, 1 (two honest FAILs: budget exhausted) | ~73 min |
Every notes-attempt solved on its first iteration. The one bare
attempt that solved on iteration 1 is sampling luck, not a signal. Three factual lines
(«the console is the USB REPL, input()/print(), machine.UART
is not wired», «input() returns without the trailing newline»,
«expected markers match literally») replaced the entire feedback loop.
| Task | Level | notes | bare | Reading |
|---|---|---|---|---|
| bme-read (BME280 driver from scratch) | 4 | 0/1, 5 iters | 0/1, 5 iters | wall: the model's ceiling, the notes effect is unmeasurable |
| uart-menu (FSM with submodes) | 3 | 4/4, iters [1,1,1,1] | 2/4, iters [5,5,5,1] | the effect, in full view |
| uart-echo (echo lines) | 1 | 4/4, iters [1,1,1,1] | 4/4, iters [1,1,1,1] | ceiling from above: both arms are perfect |
error_kind=infra, nothing
looked like a model failure. Fixed and re-verified the same day; the series above is the
clean rerun.