catalog / devtools

Test Runner

Run tests, report failures usefully, and stay fast enough to run often.

18 capabilities 1 sections reference specification

Build against it

# give your agent the catalog — no account, no install
claude mcp add provespec -- npx -y provespec-mcp

# then, from any agent session
spec_get(slug: "test-runner")                → the capability tree
spec_checklist(slug: "test-runner")          → the questions to answer
spec_grade(slug: "test-runner", grades: [...])  → parity % + the gap list

# or just fetch it
curl https://provespec.com/catalog/test-runner/spec.json

Grade every capability below as yes · partial · no · na, with a file reference as evidence. Whatever you leave ungraded counts as missing — the gap list is the work queue.

Or just paste this to your agent

Use the ProveSpec MCP server. Fetch the "test-runner" spec with
spec_get, then walk spec_checklist and grade this repository against it —
one verdict per capability, each with a file reference as the note.
Call spec_grade with the results and give me the gap list as the plan.

If the server is not registered, add it first:
  claude mcp add provespec -- npx -y provespec-mcp

The specification

AI 18

Discovery

✓ Find tests by glob and naming convention
✓ Filter by name pattern, file or tag

Structure

✓ Suites, cases and nesting
✓ Setup and teardown at each level teardown runs even when a test throws
✓ Skip, only and todo markers

Assertions

✓ Assertion library or a documented pluggable one
✓ Deep equality with a readable diff
✓ Async tests and rejection assertions
✓ Snapshot testing with review and update

Isolation

✓ Per-file isolation and parallel workers
✓ Deterministic ordering option and a randomised mode to catch coupling
✓ Per-test timeout a hang fails rather than blocks

Reporting

✓ Failure shows expected, received and the stack in user code
✓ Machine formats JUnit XML, JSON for CI
✓ Summary with counts and duration; slowest tests

Coverage

✓ Line and branch coverage with thresholds

Watch

✓ Re-run only tests affected by a change

Exit

✓ Non-zero exit on any failure including unhandled rejections

Download spec.json All 71 specs