How do I know when to stop adding adversarial cases?

Prioritize critical categories for each input and ensure core error paths and boundary conditions are exercised. If tests consistently pass and no panics occur, you’ve covered the high‑risk areas.

Are panics acceptable in adversarial tests?

Panics are generally not the desired outcome. Use them only if a function cannot recover from a given input; otherwise prefer explicit errors or graceful fallbacks.

How should I express the expected outcome?

Document whether the input should return an error, recover gracefully, or terminate with a non-nil panic. Use clear identifiers like ErrNilInput, ErrEmpty, or similar conventions in your test suite.

vibe-adversarial-test-generation

npx machina-cli add skill ash1794/vibe-engineering/adversarial-test-generation --openclaw

Files (1)

SKILL.md

2.7 KB

vibe-adversarial-test-generation

Happy-path tests prove your code works. Adversarial tests prove it doesn't break.

When to Use This Skill

After writing happy-path tests for a feature
Before claiming test coverage is complete
When preparing for a security review
When "it works on my machine" needs to become "it works everywhere"

When NOT to Use This Skill

Before happy-path tests exist (write those first)
For throwaway/prototype code
When the function is trivially simple (e.g., getters/setters)

Adversarial Categories

1. Boundary Values

Zero, one, max, max+1 for all numeric inputs
Empty string, single char, max-length string
Empty array, single element, very large array
Exactly at threshold values

2. Nil/Null/Undefined

nil pointer as receiver
nil arguments to every parameter
nil nested fields in structs
Returning nil where non-nil expected

3. Type Edge Cases

Unicode: emoji, RTL text, zero-width chars, combining chars
Strings: newlines, tabs, null bytes, control characters
Numbers: NaN, Infinity, -0, very large, very small
Dates: leap year, DST transitions, timezone boundaries, epoch

4. Concurrency

Two goroutines calling the same function
Read during write
Close during use
Cancel during operation

5. Resource Exhaustion

Very large inputs (10MB string, 1M element slice)
Disk full simulation
Network timeout simulation
Memory pressure

6. Malformed Input

Invalid JSON/YAML/XML
Truncated input (cut off mid-field)
Wrong types (string where int expected)
Extra fields, missing required fields
SQL injection patterns, XSS payloads (for external inputs)

Steps

Read the function under test -- understand inputs, outputs, side effects
For each input parameter, generate adversarial values from each category
For each adversarial input, determine expected behavior:
- Should it return an error? (most common)
- Should it handle gracefully? (fallback behavior)
- Should it panic? (almost never the right answer)
Write the tests as table-driven test cases
Run them and fix any unexpected panics or wrong error handling

Output Format

Adversarial Tests: [Function Name]

Tests Generated: X Categories Covered: Y/6

#	Category	Input	Expected	Actual
1	Nil input	nil	ErrNilInput	PASS
2	Empty string	""	ErrEmpty	PANIC!

Issues Found

[Function] panics on nil input (should return error)

Source

git clone https://github.com/ash1794/vibe-engineering/blob/master/skills/adversarial-test-generation/SKILL.mdView on GitHub

Overview

vibe-adversarial-test-generation helps you craft edge-case and failure-mode tests after happy-path tests are in place. It covers boundary values, nil inputs, concurrency, resource exhaustion, and malformed data to validate robustness before release.

How This Skill Works

It analyzes the function under test, enumerates adversarial inputs across those categories, and suggests expected behaviors (error, graceful fallback, or rare panics). It then guides you to write table-driven tests that exercise each input and run them to surface unexpected panics or improper error handling.

When to Use It

After writing happy-path tests for a feature, to expose failure modes
Before claiming test coverage is complete, to deepen validation
When preparing for a security review and resilience assessment
When turning “it works on my machine” into “it works everywhere”
Prior to release to ensure edge cases are covered

Quick Start

Step 1: Read the function under test — inputs, outputs, side effects
Step 2: For each parameter, generate adversarial values from listed categories and note expected behavior
Step 3: Implement table-driven tests and iterate until all edge cases pass or fail with the intended behavior

Best Practices

Read the function under test to understand inputs, outputs, and side effects
For each input parameter, generate adversarial values from every category (boundary, nil, type edge, concurrency, etc.)
Define expected behavior for each input (error, graceful handling, or rare panic)
Write tests as table-driven cases that clearly map inputs to expected outcomes
Run tests and fix panics or incorrect error handling uncovered by adversarial cases

Example Use Cases

Validate a JSON config loader against malformed, truncated, and type-mismatch data
Test a REST API input parser with nil fields and boundary-length strings
Stress-test a data processing function with large inputs to trigger resource exhaustion
Concurrently invoke a function to reveal race conditions and synchronization issues
Test database query builders against invalid or extra fields, and wrong types

Frequently Asked Questions

Add this skill to your agents