test-and-fix
Scannednpx machina-cli add skill yu-iskw/coding-agent-fabric/test-and-fix --openclawFiles (1)
SKILL.md
2.0 KB
Test and Fix Loop
Purpose
An autonomous loop for the agent to identify, analyze, and fix failing unit tests using Vitest.
Loop Logic
- Identify: Run
pnpm testto identify failing tests. - Analyze: Examine the test output to determine:
- The failing test file and line number.
- The expected vs actual values.
- Refer to ../common-references/troubleshooting.md for common test failure patterns.
- Fix: Apply the minimum necessary change to either the source code (if it's a bug) or the test code (if the test is outdated).
- Verify: Re-run
pnpm test.- If passed: Move to the next failing test or finish if all are resolved.
- If failed: Analyze the new failure (or the same one if the fix was insufficient) and repeat the loop.
Termination Criteria
- All tests pass (as reported by
pnpm test). - Reached max iteration limit (default: 5).
- The error persists after multiple distinct fix attempts, indicating a need for human intervention or a broader architectural change.
Examples
Scenario: Fixing a logic error
pnpm testfails inpackages/common/src/math.test.tsbecauseadd(2, 2)returned5.- Agent analyzes
packages/common/src/math.tsand finds a typoa + b + 1. - Agent fixes the typo to
a + b. pnpm testpasses.
Scenario: Updating a test after a deliberate change
pnpm testfails because a UI component's text changed from "Submit" to "Confirm".- Agent confirms the change was intentional.
pnpm testpasses.
Resources
- pnpm Commands for Node.js: Common commands for testing and managing dependencies.
- Testing Best Practices: Project-specific testing guidelines.
Source
git clone https://github.com/yu-iskw/coding-agent-fabric/blob/main/.claude/skills/test-and-fix/SKILL.mdView on GitHub Overview
The Test and Fix Loop runs unit tests with pnpm, analyzes failures, and applies the minimal necessary fixes to either source code or tests. It uses Vitest to detect failures and repeats the cycle until all tests pass or a max iteration is reached. This approach helps repair broken tests after changes and prevent regressions.
How This Skill Works
It follows Identify, Analyze, Fix, Verify: run pnpm test to identify failures, examine the output to locate the failing file and line, apply a minimal fix to the source or the test, then re-run pnpm test. The loop repeats until success or the default max iterations (5).
When to Use It
- When pnpm test reports failing tests, to quickly locate and fix issues.
- After adding new features, to ensure no regressions were introduced.
- To repair tests that no longer reflect expected behavior (test mismatches).
- When tests reveal a logic error, to fix the root cause with minimal changes.
- Before releasing, to iterate fixes until all tests pass or human intervention is required.
Quick Start
- Step 1: Run pnpm test to identify failing tests.
- Step 2: Inspect the failure output to locate the file and cause.
- Step 3: Implement the minimal fix in source or test, then re-run pnpm test.
Best Practices
- Run pnpm test to identify failures and note the failing file and line.
- Analyze the failure output and refer to troubleshooting patterns.
- Apply the minimum necessary change to either source code or tests.
- Re-run pnpm test to verify progress and decide next steps.
- Limit iterations to a max of 5 unless human intervention is required.
Example Use Cases
- Fix a logic error where add(2, 2) returned 5 by correcting a typo in math.ts.
- Update a test after UI text change from 'Submit' to 'Confirm' to reflect new behavior.
- Address a test mismatch caused by an API rename by updating the test or code.
- Resolve a regression after a feature refactor by adjusting mocks or expectations.
- Repair a failing test in another package by correcting a mock setup.
Frequently Asked Questions
Add this skill to your agents
