The full ~7,000-test suite in about 90 seconds, verified honest.
- Linux free-tests CI lane in .github/workflows/free-tests.yml runs the full free suite on every PR and push to main with zero secrets and least-privilege token
- Diff-based paid-shard selection skips shards untouched by changes and reports them as skipped-by-diff with reasoning
- Map-diff selection for test/helpers/touchfiles-data.ts re-runs only added/changed/retiered keys when selection data is edited
- Selection unions committed, staged/unstaged, and untracked changes; git failures throw with EVALS_ALL=1 fail-closed behavior
- test/helpers/skill-fixture.ts extracts only needed SKILL.md sections for E2E fixtures instead of copying full 1,800-line files
- GSTACK_EVAL_MODEL_JUDGE env override for LLM-judge model with per-kind GSTACK_EVAL_MODEL_ overrides centralized in lib/eval-model.ts
- Paid evals now bill by diff instead of a flat $38 rate
- Free suite test runner now executes up to six concurrent shard processes, reducing runtime from 454 seconds to 90–100 seconds
- Free suite architecture now runs tree-mutating tests and ratchet readers serially after the parallel phase to prevent measurement races
- Agent SDK capture default model changed from Opus to Sonnet
- Four expensive posture tests demoted from gate to periodic
- Paid runners split EVALS_JOBS from EVALS_CONCURRENCY, add --retry 1 on retry-bearing paths, and use one preflight API ping per run instead of ~30
- CI eval Docker image cache keyed on Dockerfile and bun.lock; Bun upgraded to 1.3.13 in the image
- skill-e2e-review split into three matrix shards; actionlint runs digest-pinned prebuilt image; five single-core jobs right-sized
- CI lint and skill-docs no longer double-run on every PR commit; Windows lane caches bun installs and runs the curated suite
- Ctrl-C now actually cancels a run by scheduling parent runner exit and stopping both shard pools from launching new work on SIGINT/SIGTERM
- Deleted 3,372-line dead eval monolith and revived four never-run tests from it
Paid evals now bill by diff, not $38 flat.
bun run test used to take 454 seconds. It now runs as up to six concurrent shard processes and finishes in about 90 to 100 seconds, under a strict output contract: a shard that exits without bun's own terminal summary line is a failure, a wedged shard is killed at a size-scaled deadline and named in the epilogue, and the console shows only what you need (per-shard status, then ✗ file — test name for anything red, full stream in a per-run log, --verbose for the firehose). Twelve test files that ran under no script and no CI are wired in. A 3,372-line dead eval monolith is deleted, with four never-run tests revived out of it. Paid evals select by diff. Edit one skill and the runner executes only the shards your change touches, reports the rest as skipped-by-diff, and prints the reason. Selection sees uncommitted and untracked work, fails closed with a named cause on git errors, and an edit to the selection data itself re-runs only the changed keys instead of forcing the full suite.
Added: Linux free-tests CI lane (.github/workflows/free-tests.yml): the whole free suite on every PR and every push to main, required from day one, zero secrets, least-privilege token, failure logs uploaded as an artifact, wiring pinned by test/free-tests-workflow-wiring.test.ts. Diff-based paid-shard selection: parent-side skipping with a skipped-by-diff taxonomy and a selection banner naming the reason (scripts/test-paid-shards.ts). Map-diff selection for the selection data itself: editing test/helpers/touchfiles-data.ts re-runs only added/changed/retiered keys (old version evaluated via git show + a bun child; adversarial fixtures in test/touchfiles-map-diff.test.ts). Selection unions committed, staged/unstaged, and untracked changes; git failures throw naming EVALS_ALL=1 (fail closed), and non-ASCII filenames select correctly (core.quotePath=false). test/helpers/skill-fixture.ts: E2E fixtures extract the SKILL.md sections a test needs instead of copying 1,800-line files — nine fixture sites cut 58-97%. GSTACK_EVAL_MODEL_JUDGE env override for the LLM-judge model; eval model resolution centralized in lib/eval-model.ts with per-kind GSTACK_EVAL_MODEL_ overrides.
Changed: Free suite architecture: N concurrent shard processes (serial within each); tree-mutating tests and tree-measuring ratchet readers run in one serial shard after the parallel phase, so measurements never race regeneration. Shard curation lists are pinned against the live file census, and wall deadlines scale with shard size. Agent SDK capture default Opus → Sonnet (D1a). The judge default stays Sonnet: a live A/B on the health rubric scored Haiku 2/2/2 against Sonnet's 4/3/4, so the downgrade was pinned back per D1a's regressor clause (receipts in test/helpers/llm-judge.ts). Four expensive posture tests demoted gate → periodic (D2a). Paid runners: EVALS_JOBS (shard process count) split from EVALS_CONCURRENCY (within-shard), --retry 1 on every retry-bearing paid path, one preflight API ping per run instead of ~30, detach timeouts floor-enforced against the live shard census by test/eval-detach-timeout-floor.test.ts. CI: eval Docker image cache keyed on Dockerfile + bun.lock so version bumps stop rebuilding it; Bun 1.3.13 in the image; skill-e2e-review split into three matrix shards; actionlint runs a digest-pinned prebuilt image; five single-core jobs right-sized; lint and skill-docs stop double-running every PR commit; the Windows lane caches bun installs and runs the curated suite instead of a hand list. Skill-routing E2E fixture installs skill heads, not ~18 full SKILL.md files.
Fixed: Ctrl-C actually cancels a run: the signal forwarders now schedule the parent runner's own exit and both shard pools stop launching new work on SIGINT/SIGTERM — previously the parent killed the current child and kept spawning API-burning shards. The intermittent whole-suite wedge: browse/src/browser-manager.ts close() captures the Chromium child before th