paranoidparanoid: verification of AI agent code changes
paranoid checks a diff for deleted or weakened tests, dependencies that do not exist, swallowed errors, and claims not backed by the change. It prints a scored report and sets an exit code that CI can use.
curl -fsSL https://raw.githubusercontent.com/alainrk/paranoid/main/scripts/install.sh | sh
$ paranoid verify --base main --claims summary.md paranoid v0.1.0 repo: /home/you/project base: 7db5cfbeaacf head: worktree score: 40/100 verdict: fail dependencies (1 finding) DEP001 critical requirements.txt:2 The new dependency requests-toolkit-pro does not exist in PyPI. test-integrity (2 findings) TI001 critical test_division.py A test file that existed at base is deleted at head. TI004 high test_addition.py:10 A skip was added to an existing test. exit status 2
What it looks for
Three failure modes that occur in agent-authored patches and are easy to miss in review.
Gamed tests
A failing test is deleted, skipped, or rewritten into a tautology, so the suite passes without the code being fixed.
Hallucinated dependencies
A new import names a package that does not exist, was published days ago, or is one typo away from a popular name.
Unbacked claims
The summary states work (“all tests pass”, “refactored the retry logic”) that the diff and the test run do not back.
Checks
paranoid verify
compares the repository before and after the agent worked. Every finding
maps to a documented rule with a fixed ID and severity, listed with
examples in the rule reference.
Test integrity
Parses the test diff (Go with the real Go parser) and flags deleted tests, weakened assertions, new skip markers, tautologies, hidden renames, new mocking, new retries, new sleeps, stretched timeouts, and edits to CI and test configuration.
Dependencies
Cross-checks new imports against manifests, then looks up new packages on npm, PyPI, crates.io, and the Go proxy: nonexistent, brand new, nearly unused, or typosquatting a popular name.
Error handling
Looks at every changed source file for errors made to disappear: a checked error newly discarded, an empty catch block, an except block that only passes, a no-op promise handler, and a return or raise turned into a log line.
Clean-room test run
Reruns the test suite in a container with the network disabled, and collects coverage where the tooling allows. Detects results that only held on the agent's machine, and coverage that dropped.
Claims
Extracts concrete claims from the agent’s summary, a session file from Claude Code, Codex, Gemini CLI, Aider, or Cursor, or the commit messages, and checks each one against the diff, the clean-room result, and the diff's overall radius.
API surface
Diffs public symbols between base and head and flags a symbol removed while the claims say a fix or feature landed: a suite that passes because the code it tested was removed.
Safety controls
Scans every changed file for known patterns: TLS verification disabled, an auth check removed, a tracked path hidden by .gitignore, a git hook changed, CI permissions widened, or paranoid's own baseline and hook config edited. Pattern-based, not a security scanner.
Score
Findings subtract from 100 by severity. The verdict and the exit code follow the score. Output as text, Markdown, schema-versioned JSON that is byte-identical across reruns, or SARIF for GitHub code scanning.
Baseline and fix-prompt
Two commands: one for adopting paranoid in an existing repository, one for passing findings back to the agent.
paranoid baseline
On a repository that has never run paranoid, verify reports every finding, not only those from the latest change. Run paranoid baseline once to snapshot existing findings into .paranoid-baseline.json; after that, only new findings fail CI. See Baseline & suppressions →
paranoid fix-prompt
fix-prompt renders the findings of a verify run as a deterministic instruction block to give back to the agent. See fix-prompt →
Exit codes
| exit | verdict | meaning |
|---|---|---|
| 0 | pass | score 80 or higher |
| 1 | warn | score 50 to 79 (exit 2 with --strict) |
| 2 | fail | score below 50, or below --fail-under |
| 3 | error | paranoid itself failed (not a finding) |
- uses: alainrk/paranoid/action@main with: base: ${{ github.base_ref }} fail-under: 50
Design
Local and private
No telemetry, no analytics, no update checks. The only network use is the optional registry lookups, and --offline turns those off too.
Deterministic by default
No LLM is used as a judge. The default pipeline is static analysis. The same input produces the same report, byte for byte in JSON.
Skipped checks
A check that cannot run (no container runtime, no network, no recognizable test suite) is reported as skipped, and the run continues.
Single static binary
No CGO, no runtime dependencies beyond git, and docker or podman only if you want the clean room. Linux, macOS, and Windows.