pytest-dev
Scannednpx machina-cli add skill BjornMelin/dev-skills/pytest-dev --openclawFiles (1)
SKILL.md
3.1 KB
pytest-dev
Produce high-signal, low-flake, fast pytest suites and CI configs, with an explicit focus on measurable wins (runtime, flake rate, coverage quality).
Default workflow
- Classify the tests
- Unit: pure functions, no I/O (preferred)
- Integration: DB/filesystem/multiprocess, slower but valuable
- System/E2E: external services or UI, keep minimal and well-gated
- Identify boundaries
- Time/clock, randomness, network, filesystem, DB, env vars, global state
- Pick the lightest seam
- Prefer fakes/stubs over deep mocks; prefer dependency injection over patching internals
- Make it deterministic
- Control time, seeds, tmp dirs; avoid order dependencies
- Measure before optimizing
- Collection time vs runtime; quantify with
--durations+ a single baseline
- Collection time vs runtime; quantify with
- Harden for CI
- Enforce marker discipline, strict config, timeouts, isolation for parallel
Quick commands
Use python3 by default. If the project uses uv, prefer uv run python.
- Smallest repro:
python3 -m pytest path/to/test_file.py -q - First failure only:
python3 -m pytest -x --maxfail=1 - Find slow tests:
python3 -m pytest --durations=20 --durations-min=0.5 - Emit JUnit for CI:
python3 -m pytest --junitxml=reports/junit.xml - Parallelize on one machine (xdist):
python3 -m pytest -n auto --dist load
Optimization playbook (high ROI)
- Reduce collection scope (
testpaths,norecursedirs, avoid importing heavy modules at import time). - Fix fixture scoping (move expensive setup up-scope; ensure isolation).
- Eliminate sleeps and retries (poll with timeouts; mock time).
- Parallelize safely (xdist; isolate worker resources: tmp/db ports).
- Shard in CI (split test files by historical timings; keep shards balanced).
Use the bundled references
Read these when needed (keep SKILL.md lean):
references/pytest_core.md: fixtures, markers, parametrization, strict mode, TOML config, subtests (pytest 9.x).references/plugins.md: plugin selection + usage patterns.references/performance.md: collection/runtime profiling and speedups.references/ci_github_actions.md: sharding, artifacts, caching, concurrency.
Use the bundled scripts
scripts/junit_slowest.py: report slowest tests/files from JUnit XML.scripts/junit_split.py: split test files into N shards using JUnit timings.scripts/run_pytest_filelist.py: run pytest for a list of test files.
Quality gates
- Tests pass in a clean environment (no hidden dependency on local state).
- No network/time dependency without explicit control.
- Parallel-safe or explicitly marked/serialized.
- CI emits machine-readable artifacts when relevant (JUnit, coverage).
Source
git clone https://github.com/BjornMelin/dev-skills/blob/main/skills/pytest-dev/SKILL.mdView on GitHub Overview
Produces high-signal, low-flake, fast pytest suites and CI configs with a focus on measurable wins in runtime, flakiness, and coverage quality. It guides classification of tests, deterministic setups, and CI hardening to deliver reliable pipelines. Emphasizes modern pytest features, plugins, and performance tricks for scalable test automation.
How This Skill Works
Follows a concrete workflow: classify tests by scope, identify boundaries like time and IO, choose light seams (fakes/stubs) over deep mocks, and enforce determinism. It measures impact with durations profiling and baselines, then tunes collection, fixture scope, and CI settings before enabling safe parallelism with plugins like xdist and coverage tooling.
When to Use It
- When you need pytest best practices guidance and modern features (pytest 9.x) and plugin usage.
- When tuning test performance and CI speed (profiling collection vs runtime, using --durations, establishing baselines).
- When configuring pytest with TOML config, strict mode, and subtests.
- When selecting and using pytest plugins (xdist, cov, asyncio, mock, httpx) to accelerate tests.
- When optimizing CI (GitHub Actions sharding, caching, parallelism) and test sharding strategies.
Quick Start
- Step 1: Run a baseline to identify slow tests: python3 -m pytest --durations=20 --durations-min=0.5
- Step 2: Enable strict config and TOML config for pytest (add minimal pyproject.toml or pytest.ini referencing strict mode and markers)
- Step 3: Run with parallelism and CI shards: python3 -m pytest -n auto --dist load and configure GitHub Actions sharding + caching
Best Practices
- Classify tests by scope (unit, integration, system) and keep a simple baseline.
- Control boundaries: time, randomness, network, filesystem, env vars, and global state.
- Prefer lightweight seams (fakes/stubs) and dependency injection over deep internal patching.
- Make tests deterministic: fix time, seeds, tmp dirs; avoid order dependencies.
- Measure before optimizing: compare collection time vs runtime with durations and baseline; harden CI with marker discipline and timeouts.
Example Use Cases
- Speed up CI by enabling xdist parallelism and balancing shards with historical timings.
- Migrate to strict configuration and TOML-based pytest config for consistent runs.
- Replace heavy module imports in tests with lazy loading or lighter fixtures to speed collection.
- Adopt pytest-cov and JUnit XML output to improve CI reporting and coverage signals.
- Use plugins like asyncio and httpx mocks to reduce real I/O in tests while maintaining fidelity.
Frequently Asked Questions
Add this skill to your agents
