caveman v2.3.0

v2.3.0
Added 3
  • Confirmed savings are attributed to a named method — deterministic_remeasure, controlled_holdout, counterfactual_replay, or interrupted_time_series
  • A controlled-holdout harness runs a fix on and off on the same machine to judge changes whose payback lands outside the edited file
  • caveman learn reconcile checks measured spend against a provider usage export and generates a privacy-safe export digest
Changed 3
  • caveman learn now reports the actual cost of the window from provider-counted usage instead of just token movements
  • The gateway buffers non-streaming upstream bodies before replying
  • The caveman ruleset's framing clauses were rewritten and the bundled agent skill registry was regenerated
Fixed 1
  • Serve-mode warnings now land in ~/.caveman/proxy.log instead of vanishing
Removed 1
  • The GitHub Pages landing page is gone

From caveman

Superseded by v2.3.1. This tag's install.sh / install.ps1 are still pinned at v2.2.0, so the v2.3.0 one-liner bootstraps the previous release. Use v2.3.1.

Learn v3: caveman learn now says what the window cost, not just how many tokens moved — and every savings number it reports carries the method that measured it.

Learn v3 — spend and attributed savings
  • The scan reads billing buckets and outcome signals straight from agent transcripts (Claude, Codex, Gemini, opencode) and prices the window from provider-counted usage, so the cost figure is the provider's arithmetic rather than an estimate.
  • Confirmed savings are attributed to a named method — deterministic_remeasure, controlled_holdout, counterfactual_replay, or interrupted_time_series. Rungs are never summed into one blended headline, because a re-counted file and a before/after median are not the same kind of evidence.
  • A controlled-holdout harness runs a fix on and off on the same machine, so a change whose payback lands somewhere other than the file it edited (a distilled skill, say) can still be judged. Under 5 sessions per arm it returns insufficient_data instead of a small win.
  • caveman learn reconcile checks measured spend against a provider usage export, and a privacy-safe export digest makes that export shareable without shipping prompt text.
  • Spend, the savings ledger, and holdout reports surface in the CLI and in the caveman-learn skill. Spend is what the window cost — never what a fix would return, never projected into a month, and never called verified. On a Max/Plus plan it is the API-equivalent value of the tokens, not money spent.
Proxy
  • The gateway buffers non-streaming upstream bodies before replying.
  • Serve-mode warnings land in ~/.caveman/proxy.log instead of vanishing.
Elsewhere
  • The caveman ruleset's framing clauses were rewritten, and the bundled agent skill registry regenerated.
  • The GitHub Pages landing page is gone.
Runtime

Installs pin the bin-v1.1.3 companions and CLI 1.2.4. All 36 binaries and the signed manifest are published at bin-v1.1.3.

Install
curl -fsSL https://raw.githubusercontent.com/JuliusBrussee/caveman/v2.3.0/install.sh | bash

irm https://raw.githubusercontent.com/JuliusBrussee/caveman/v2.3.0/install.ps1 | iex
View original

Upgraded? How did it go?

Discussion