knit-spool 0.1.0
- Record layer with CBOR hello/sub/digest/list/pull/blob/push/event/ok/err messages pinned to spec vectors with forward-compatible tolerance of unknown records and fields
- Handshake with version negotiation, advertised limits, and bearer-token private spools
- Fan-out live event delivery to every other subscriber of a scope with q-correlated replies and idempotent duplicate pushes
- Retention with oldest-by-arrival eviction, count-bounded tombstones, and per-scope digests with unsolicited re-anchors
- Attachments family (ahave/ahas/aget/achunk/aput) with chunk presence bitmaps, first-write-wins conflict detection, and per-scope byte quota
- Abuse control including stateless PoW on SUB and shed-scope PUSH-recreate, per-connection and per-IP rate limits, and global storage watermark shedding
- Persistence using SQLite with WAL and self-healing boot recompute or in-memory storage
- Operational endpoints including GET /healthz, GET /metrics (Prometheus text), graceful shutdown, and periodic status logging
- knit_spool_egress_bytes_total metric for fan-out egress tracking
- Conformance suite CLI that validates any live spool over WebSocket with destructive and token-file options
- Container image with JRE-only runtime stage, health check, and volume support
- Reverse-proxy configurations for Caddy and nginx that keep bearer tokens out of access logs
- Self-contained TLS compose stack for certificate issuance and renewal
- Minimal overlay deployment for 1 GB boxes with resource caps and metered link optimization
- GitHub Actions CI pipeline running check, koverVerify, and conformance suite on pull requests
- GitLab CI pipeline with kaniko image build, Trivy scans, and tag-only release job
- Release workflow publishing multi-arch images to GHCR and Docker Hub with signed build provenance
- Coverage reporting via Kover with merged and per-module reports and ratcheting line/branch floors
- Hot-path hex encoding, digest computation, and store queries optimized to reduce per-record overhead
- WebSocket close path no longer surfaces ping-timeout IOException as an error
- Conformance runner reports non-assertion failures diagnosably and tallies transport faults separately from spec violations
- Container build no longer races Kotlin compile daemon lock file under kaniko and mkdir tolerates volume path creation
From Knit Spool
First implementation of the v1 spool protocol, and the first tagged release. Pre-1.0: every interface below is subject to change, and only the wire protocol's own compatibility rules — which are versioned separately, above — are stable.
Added
- Record layer — CBOR
hello/sub/digest/list/pull/blob/push/event/ok/err, pinned byte-for-byte to the spec's §13 vectors bySpecVectorTest, with forward-compatible tolerance of unknown records and fields. - Handshake — version negotiation, advertised limits, and bearer-token private spools
(
wss://host/spool/v1?k=…). - Fan-out — live
eventdelivery to every other subscriber of a scope,q-correlated replies, and idempotent duplicate pushes. - Retention — oldest-by-arrival eviction, count-bounded tombstones, and per-scope digests with unsolicited re-anchors after eviction or expiry.
- Attachments (§6.5/§7.3) —
ahave/ahas/aget/achunk/aput, chunk presence bitmaps, first-write-wins withconflicton mismatch, truncated (never refused) over-longaget, and a per-scope byte quota.SPOOL_MAX_ATTACH_BYTES=0removes the whole family fromhello. - Abuse control — stateless PoW on SUB and on the shed-scope PUSH-recreate path, with the
per-
(scope, day)cache; per-connection and per-IP rate limits (rate+retryMs, escalating to close 4003); and a global storage watermark that sheds the least-active scope. - Persistence — SQLite (WAL, self-healing boot recompute) or in-memory behind one store contract, plus a periodic sweeper.
- Ops —
GET /healthz,GET /metrics(Prometheus text, token-gated on private spools), graceful shutdown, and a periodic one-line status log under its ownapp.getknit.spool.Statuslogger — gauges absolute, everything else a delta since the previous line (SPOOL_STATUS_MS, 5 min;0disables). knit_spool_egress_bytes_total— fan-out means one push leaves as (subscribers − 1) copies, and on a metered link the transfer allowance binds long before CPU or memory does.- Conformance suite (
:conformance) — a CLI that validates any live spool over WebSocket, TAP on stdout and a MUST tally on stderr. Depends only on:protocol, never on:daemon, so it tests the wire contract rather than this repo's internals.--destructiveenables the quota and rate-limit checks;--token-filekeeps a bearer token out of argv, wherepsand shell history can read it. - Deployment — a container image (JRE-only runtime stage, uid 65532,
/datavolume,/healthzHEALTHCHECK); Caddy and nginx reverse-proxy configurations, both keeping the?k=token out of access logs; a self-contained TLS compose stack that issues and renews certificates; and a tiny overlay for 1 GB boxes that side-loads or pulls the image instead of building it, caps every container, bounds the log driver, and re-sizes the limits for a metered link. - CI — two pipelines over the same gating checks. GitHub Actions
(
.github/workflows/ci.yml) runscheckwith merged coverage,koverVerify, and the conformance suite against the freshly built daemon on every pull request, and publishes the coverage badge on a default-branch push. The maintainer's internal GitLab pipeline (.gitlab-ci.yml) runs the same two test jobs and adds what needs a registry credential: a kaniko image build, advisory Trivy filesystem/image and markdownlint scans, and a tag-only release job. - Releases — a
v*tag runs.github/workflows/release.yml, which is the default source of release images. It re-runscheckand the conformance suite against the tagged tree, then publishes a multi-arch (linux/amd64,linux/arm64) image to GHCR and Docker Hub with a signed build provenance attestation, and opens a draft GitHub Release carrying the distribution archives and theirSHA256SUMS. The image is built fromDockerfile.dist, which layers a natively compiled distribution onto the multi-arch JRE base instead of compiling under emulation. - Community and automation — GitHub issue forms, a pull-request template, label-driven canned
replies, keyword triage, and stale sweeps under
.github/. - Coverage reporting (Kover) — one merged report over all three modules
(
./gradlew koverHtmlReport), plus per-module reports.koverVerifyholds line and branch floors as a ratchet against tests being deleted, and the merged percentage is published as the README's coverage badge. Process entry points and generated serializers are excluded — the former only run out of process, under the conformance self-test, where Kover cannot see them.
Changed
- Hot-path hex encoding, digest computation, and store queries reworked to cut per-record overhead.
Fixed
- The WebSocket close path no longer surfaces a ping-timeout
IOExceptionas an error. - The conformance runner reports non-assertion failures diagnosably, and tallies transport faults apart from spec violations — a spool that drops the connection no longer looks like a spool that answered wrongly.
- The container build no longer races the Kotlin compile daemon's
/tmplock file under kaniko (compilation runs in-process), andmkdir -p /datatolerates kaniko creating theVOLUMEpath during stage setup.
Container image
Published to GHCR and Docker Hub for linux/amd64 and linux/arm64:
docker pull ghcr.io/getknit/knit-spool@sha256:5844ef0a28e175ecafa66e5cea19dfeb34e6db18d416287a8f3bceecf17d1777
docker pull docker.io/getknit/knit-spool@sha256:5844ef0a28e175ecafa66e5cea19dfeb34e6db18d416287a8f3bceecf17d1777
Pin the digest in SPOOL_IMAGE rather than a tag — see deploy/.env.example. The GHCR
image carries a signed build provenance attestation:
gh attestation verify oci://ghcr.io/getknit/knit-spool:0.1.0 --repo getknit/knit-spool
Standalone archives
For running the daemon without Docker. SHA256SUMS covers every archive attached here.
Both need a JDK 21 runtime on the host; unpack and run bin/knit-spool.
Upgrading a live spool? HOSTING.md covers the operational side, and CHANGELOG.md calls
out anything that moves the wire, the on-disk store, or a configuration default.