Skip to content

Engineering

Testing Standard

The quality target is SQLite-class. Not the famous ratio — the method that produces it. Four independent techniques, each catching what the others miss, all made possible by the sans-io core.

The four pillars

  • Known-answer vectors

    Every cryptographic primitive is checked against its RFC test vectors — Ed25519 (RFC 8032), HMAC (RFC 4231), HKDF (RFC 5869), and so on. The handshake is checked against the official Noise vectors. This is the "does it match the spec, byte for byte" pillar.

  • Property-based tests

    proptest for anything with round-trip or invariant structure: encode∘decode = identity, index_at(start_of(i)) = i, "rotating a secret yields a disjoint address set." The machine searches the input space for a counterexample.

  • Deterministic simulation

    The centrepiece. The whole protocol runs against a simulated network driven by a seeded RNG — loss, reordering, delay, partition — for millions of adversarial scenarios, each reproducible from its seed. This is what FoundationDB and TigerBeetle do, and it is only possible because the core has no real I/O.

  • Fuzzing

    cargo-fuzz on every decoder that touches untrusted bytes. Wire formats face hostile input from the first byte; the fuzzer finds the panic before an attacker does.

Coverage goal

100% branch coverage on the sans-io core. Because nothing below the transport adapter touches a socket, clock, or RNG, every branch is reachable from a deterministic unit test — so full coverage is an achievable standard, not an aspiration. Untested lines accumulate silently, and every one is a place where the suite says "covered" and means nothing.

The conventions that make it possible

From CLAUDE.md, the non-negotiables that keep the code testable:

  • Protocol logic is I/O-free and deterministic — no sockets, clocks, or randomness reached for inside it. They are passed in, so every state transition is testable without a network. See language strategy.
  • Wire formats are versioned from the first byte. A format change that cannot be negotiated is a breaking change.
  • Errors are typed and exhaustive. No stringly-typed failure paths on the wire.
  • Every protocol behaviour gets a test that drives it through the state machine, not through a live connection.
  • Cryptographic primitives come from vetted libraries. We compose them; we do not implement them (for now — see philosophy).

How tests are written here

Test-first, strictly. Tests are written and watched to fail before any implementation exists — the red state proves the test can fail and the API is settled. No logic is added that no test demands.

The house style: one test, one file, one reason

Each test lives in its own file, opening with a //! WHY: comment explaining what breaks if the test is absent. Edge cases first — boundaries, overflow, underflow, the unreachable-with-real-input cases that nothing else will ever exercise — then a generous set of normal cases. A worked example, from Module 2.1:

tests/
├── edge_window_clamps_at_bucket_zero.rs      # the underflow: index 0 − 1 must not wrap
├── edge_window_saturates_near_max_index.rs   # the overflow twin
├── edge_time_before_epoch_is_rejected.rs     # unsigned underflow → typed error
├── floor_rejects_backward_time.rs            # spoofed-clock DoS; floor must not move
├── hourly_buckets_advance_each_hour.rs       # the expected v1 config
└── ... 26 files total

A passing test in an otherwise-red suite is treated as suspicious and checked — it must pass for the reason you think, not by accident.

See the implementation plan for which modules face which pillars, and Module 2.1 for the build in progress.