Overview

perf-theory-tester helps you test performance hypotheses with controlled experiments. It follows the canonical contract in docs/perf-requirements.md and emphasizes a clean baseline, a single change, 2+ validation passes, and reverting to baseline before the next test. Output is a structured report with hypothesis, change, delta, verdict, and evidence.

How This Skill Works

Prepare a clean baseline, apply a single change tied to your hypothesis, run at least two validation passes, then revert to baseline before the next experiment. Each run records the command and changed files; the tool emits a standardized output including hypothesis id, delta metrics, and a verdict (accept, reject, or inconclusive).

When to Use It

• Validating a performance hypothesis with a controlled change
• Measuring impact of a single code or config alteration
• Ensuring reproducibility with 2+ validation passes
• Documenting evidence for performance decisions
• Isolating experiments by reverting to baseline before each test

Quick Start

1. Step 1: Confirm baseline is clean.
2. Step 2: Apply a single change tied to the hypothesis and run 2+ validation passes.
3. Step 3: Revert to baseline before the next experiment.

Best Practices

• Start with a clean baseline before each experiment
• Apply only one change per experiment to avoid confounding effects
• Run 2+ validation passes to reduce flakiness
• Do not run benchmarks in parallel
• Record evidence for each run using the standardized output format

Example Use Cases

• Test latency impact of a cache optimization in a hot path
• Evaluate performance effect of a new indexing strategy
• Measure throughput change after a concurrency tweak
• Assess memory footprint variation after a refactor
• Validate I/O throughput improvements from an async path

Frequently Asked Questions