Knit Spool v0.1.0

v0.1.0

knit-spool 0.1.0

Added 18
  • Record layer with CBOR hello/sub/digest/list/pull/blob/push/event/ok/err messages pinned to spec vectors with forward-compatible tolerance of unknown records and fields
  • Handshake with version negotiation, advertised limits, and bearer-token private spools
  • Fan-out live event delivery to every other subscriber of a scope with q-correlated replies and idempotent duplicate pushes
  • Retention with oldest-by-arrival eviction, count-bounded tombstones, and per-scope digests with unsolicited re-anchors
  • Attachments family (ahave/ahas/aget/achunk/aput) with chunk presence bitmaps, first-write-wins conflict detection, and per-scope byte quota
  • Abuse control including stateless PoW on SUB and shed-scope PUSH-recreate, per-connection and per-IP rate limits, and global storage watermark shedding
Changed 1
  • Hot-path hex encoding, digest computation, and store queries optimized to reduce per-record overhead
Fixed 3
  • WebSocket close path no longer surfaces ping-timeout IOException as an error
  • Conformance runner reports non-assertion failures diagnosably and tallies transport faults separately from spec violations
  • Container build no longer races Kotlin compile daemon lock file under kaniko and mkdir tolerates volume path creation

From Knit Spool

First implementation of the v1 spool protocol, and the first tagged release. Pre-1.0: every interface below is subject to change, and only the wire protocol's own compatibility rules — which are versioned separately, above — are stable.

Added
  • Record layer — CBOR hello/sub/digest/list/pull/blob/push/event/ok/err, pinned byte-for-byte to the spec's §13 vectors by SpecVectorTest, with forward-compatible tolerance of unknown records and fields.
  • Handshake — version negotiation, advertised limits, and bearer-token private spools (wss://host/spool/v1?k=…).
  • Fan-out — live event delivery to every other subscriber of a scope, q-correlated replies, and idempotent duplicate pushes.
  • Retention — oldest-by-arrival eviction, count-bounded tombstones, and per-scope digests with unsolicited re-anchors after eviction or expiry.
  • Attachments (§6.5/§7.3) — ahave/ahas/aget/achunk/aput, chunk presence bitmaps, first-write-wins with conflict on mismatch, truncated (never refused) over-long aget, and a per-scope byte quota. SPOOL_MAX_ATTACH_BYTES=0 removes the whole family from hello.
  • Abuse control — stateless PoW on SUB and on the shed-scope PUSH-recreate path, with the per-(scope, day) cache; per-connection and per-IP rate limits (rate + retryMs, escalating to close 4003); and a global storage watermark that sheds the least-active scope.
  • Persistence — SQLite (WAL, self-healing boot recompute) or in-memory behind one store contract, plus a periodic sweeper.
  • OpsGET /healthz, GET /metrics (Prometheus text, token-gated on private spools), graceful shutdown, and a periodic one-line status log under its own app.getknit.spool.Status logger — gauges absolute, everything else a delta since the previous line (SPOOL_STATUS_MS, 5 min; 0 disables).
  • knit_spool_egress_bytes_total — fan-out means one push leaves as (subscribers − 1) copies, and on a metered link the transfer allowance binds long before CPU or memory does.
  • Conformance suite (:conformance) — a CLI that validates any live spool over WebSocket, TAP on stdout and a MUST tally on stderr. Depends only on :protocol, never on :daemon, so it tests the wire contract rather than this repo's internals. --destructive enables the quota and rate-limit checks; --token-file keeps a bearer token out of argv, where ps and shell history can read it.
  • Deployment — a container image (JRE-only runtime stage, uid 65532, /data volume, /healthz HEALTHCHECK); Caddy and nginx reverse-proxy configurations, both keeping the ?k= token out of access logs; a self-contained TLS compose stack that issues and renews certificates; and a tiny overlay for 1 GB boxes that side-loads or pulls the image instead of building it, caps every container, bounds the log driver, and re-sizes the limits for a metered link.
  • CI — two pipelines over the same gating checks. GitHub Actions (.github/workflows/ci.yml) runs check with merged coverage, koverVerify, and the conformance suite against the freshly built daemon on every pull request, and publishes the coverage badge on a default-branch push. The maintainer's internal GitLab pipeline (.gitlab-ci.yml) runs the same two test jobs and adds what needs a registry credential: a kaniko image build, advisory Trivy filesystem/image and markdownlint scans, and a tag-only release job.
  • Releases — a v* tag runs .github/workflows/release.yml, which is the default source of release images. It re-runs check and the conformance suite against the tagged tree, then publishes a multi-arch (linux/amd64, linux/arm64) image to GHCR and Docker Hub with a signed build provenance attestation, and opens a draft GitHub Release carrying the distribution archives and their SHA256SUMS. The image is built from Dockerfile.dist, which layers a natively compiled distribution onto the multi-arch JRE base instead of compiling under emulation.
  • Community and automation — GitHub issue forms, a pull-request template, label-driven canned replies, keyword triage, and stale sweeps under .github/.
  • Coverage reporting (Kover) — one merged report over all three modules (./gradlew koverHtmlReport), plus per-module reports. koverVerify holds line and branch floors as a ratchet against tests being deleted, and the merged percentage is published as the README's coverage badge. Process entry points and generated serializers are excluded — the former only run out of process, under the conformance self-test, where Kover cannot see them.
Changed
  • Hot-path hex encoding, digest computation, and store queries reworked to cut per-record overhead.
Fixed
  • The WebSocket close path no longer surfaces a ping-timeout IOException as an error.
  • The conformance runner reports non-assertion failures diagnosably, and tallies transport faults apart from spec violations — a spool that drops the connection no longer looks like a spool that answered wrongly.
  • The container build no longer races the Kotlin compile daemon's /tmp lock file under kaniko (compilation runs in-process), and mkdir -p /data tolerates kaniko creating the VOLUME path during stage setup.

Container image

Published to GHCR and Docker Hub for linux/amd64 and linux/arm64:

docker pull ghcr.io/getknit/knit-spool@sha256:5844ef0a28e175ecafa66e5cea19dfeb34e6db18d416287a8f3bceecf17d1777
docker pull docker.io/getknit/knit-spool@sha256:5844ef0a28e175ecafa66e5cea19dfeb34e6db18d416287a8f3bceecf17d1777

Pin the digest in SPOOL_IMAGE rather than a tag — see deploy/.env.example. The GHCR image carries a signed build provenance attestation:

gh attestation verify oci://ghcr.io/getknit/knit-spool:0.1.0 --repo getknit/knit-spool
Standalone archives

For running the daemon without Docker. SHA256SUMS covers every archive attached here. Both need a JDK 21 runtime on the host; unpack and run bin/knit-spool.

Upgrading a live spool? HOSTING.md covers the operational side, and CHANGELOG.md calls out anything that moves the wire, the on-disk store, or a configuration default.

View original

Upgraded? How did it go?

Discussion