Checks

Sandbox and coverage

The clean-room check reruns the test suite in a container with no network, from a fresh export of the head tree, and collects coverage where the tooling allows.

How it works

With docker or podman available, verify copies the head state into a container, installs dependencies with the network on, then runs the test suite with the network off (--network=none). Two container runs share one named volume: the prep run mounts the export read-only and installs dependencies with the network on; the test run mounts only the volume and uses --network=none. The volume is removed afterwards.

Supported suites, detected automatically:

Go: go test ./... when go.mod exists (Go version read from go.mod).
Python: pytest or unittest, in a throwaway venv.
JS/TS: the package.json test script, via npm, yarn, or pnpm.

--sandbox none opts out explicitly and records the check as skipped, no finding attached. When auto finds no container runtime or no recognizable suite, the report says so and SBX004 marks that nothing was verified. Suites that need the network fail in the clean room by design.

Sandbox flags on verify

Flag	Default	Meaning
--sandbox	auto	auto, docker, podman, or none
--sandbox-image	suite default	Override the container image
--sandbox-timeout	10m	Kill the run after this long
--compare-base	off	Also run the suite at base; enables SBX003 and SBX005

Sandbox rules (SBX)

Rule	Severity	What it catches
SBX001	critical	The test suite fails when rerun in a clean container
SBX002	high	The clean run skipped tests and the diff added skip markers
SBX003	medium	With --compare-base, fewer tests executed at head than at base
SBX004	info	The clean-room check could not run, so nothing was verified
SBX005	medium	With --compare-base, total coverage dropped by more than 5 points

Full rule text is on the rules page.

SBX001's evidence names the failing tests when the suite output identifies them (go test and pytest); otherwise it shows the test command and the tail of its output.

Coverage collection

The same test run also collects total test coverage, best effort, without ever changing whether the suite passed or failed. If coverage parsing fails for any reason, the SBX001 to SBX004 outcome is untouched; coverage just reports itself unavailable.

Go: always. go test -coverprofile plus go tool cover -func reads the real per-statement total from the profile.
Python: only when pytest-cov is already importable in the prepared environment. This tool never installs it; a project that does not already depend on it gets no Python coverage number.
JS/TS: only when the test script in package.json names jest or vitest. --coverage --coverageReporters=json-summary, read from coverage-summary.json (Istanbul's default location). Other runners (mocha, ava, a custom script) get no coverage flags at all.

With --compare-base, SBX005 fires when total coverage at head is more than 5.0 percentage points below total coverage at base. The report's coverage block always exists, with an available flag and a reason when it is false; see Report & schema for the exact fields.

A custom --sandbox-image needs the matching coverage tool already installed: Go's toolchain ships one; Python needs pytest-cov in the project's own dependencies; Node needs jest or vitest. Otherwise coverage is reported as unavailable, the same as a missing container runtime.

Limits

Coverage is best-effort and never affects the score by itself; only SBX005, and only with --compare-base, does.
Removing dead code lowers the denominator and can raise or lower the percentage either way; SBX005 is a signal to look, not proof of lost tests.
Windows sandbox support is out of scope today; it degrades to a skipped check.