testing-with-nullables
npx machina-cli add skill msewell/agent-stuff/testing-with-nullables --openclawTesting With Nullables
Core concept
Nullables are production classes with an infrastructure "off switch." Each infrastructure class provides two factories:
create()— production instance with real I/OcreateNull()— same class, same code paths, but I/O suppressed at the third-party boundary
Tests exercise real production code. Only the lowest-level third-party calls are stubbed.
class CommandLine {
private constructor(private stdout: WritableStream, private _args: string[]) {}
static create(): CommandLine {
return new CommandLine(process.stdout, process.argv.slice(2));
}
static createNull(options?: { args?: string[] }): CommandLine {
return new CommandLine(new NullWritableStream(), options?.args ?? []);
}
write(text: string): void { this.stdout.write(text); }
args(): string[] { return this._args; }
}
Workflow: Making a dependency testable with Nullables
- Identify the infrastructure boundary. Find the class that talks to an external system (HTTP, file system, database, stdout).
- Create an Infrastructure Wrapper if one doesn't exist. One wrapper per external system. Translate between external formats and domain types.
- Add an Embedded Stub. Stub the third-party code (not yours) with a minimal implementation covering only the methods your code calls. Keep it in the same file.
- Add
createNull()factory that injects the embedded stub instead of the real third-party dependency. Both factories return the same class. - Add Configurable Responses to
createNull()so tests can control what the dependency returns. Name parameters by behavior (verificationStatus), not implementation (httpResponseBody). - Add Output Tracking via
trackXxx()methods that record writes using an event emitter. This enables state-based assertions on side effects. - Compose upward. Higher-level classes call
createNull()on their dependencies — no new stubs needed (Fake It Once You Make It).
Workflow: Writing tests in the Nullables style
- Write narrow tests. One test file per module/class. Each test has a single reason to fail.
- Make tests sociable. Exercise real dependencies — don't mock them. If
AppusesRot13, tests run the realRot13. - Assert on state, not interactions. Check return values, object state, or tracked output. Never assert on whether methods were called or in what order.
- Use signature shielding. Create test helper functions that wrap
createNull()calls. When the factory signature changes, update one helper instead of every test. - Write narrow integration tests for each infrastructure wrapper against real (but local/test-isolated) external systems. Run these in CI, not on every save.
- Add smoke tests sparingly. One or two end-to-end tests as a safety net. If you need many, your narrow tests have gaps.
Workflow: Migrating from mocks to Nullables
- Pick the most-mocked dependency.
- Make it Nullable (follow the workflow above).
- Replace its mocks with
createNull()in tests — keep other mocks unchanged. - Run tests. They should still pass.
- Repeat for the next most-mocked dependency.
Nullables coexist with mocks. No big-bang rewrite needed.
Key rules
- Stub third-party code, not your code. Your production code runs in full during tests.
create()andcreateNull()return the same class — not a subclass, not a fake.- Constructors do no work (Zero-Impact Instantiation). Defer connections, servers, and I/O to explicit methods.
- Pure logic classes don't need Nullables. Only infrastructure wrappers need
createNull(). - Output Tracking works on both real and Nulled instances. It's useful in production too (monitoring, auditing).
- Design configurable responses from the consumer's perspective, not the implementation's.
Architecture recommendation
Use A-Frame Architecture for maximum testability:
Application / UI
/ \
Logic Infrastructure
\ /
Values (shared)
- Application layer: Coordinates logic and infrastructure via Logic Sandwich (read → process → write). No business logic, no I/O details.
- Logic layer: Pure business rules. Depends only on values. Test with simple state-based assertions.
- Infrastructure layer: Wraps external systems. One wrapper per system. Made testable via Nullables.
Edge cases
- Languages requiring interfaces (Java, C#): Use Thin Wrapper pattern — define a custom interface covering only methods you use, with real and null implementations. See references/nullability-core.md.
- Event-driven applications: Use Behavior Simulation — add
simulateXxx()methods to infrastructure wrappers that trigger the same handler as real events. See references/nullability-advanced.md. - Complex stateful protocols: For deeply stateful interactions (multi-step auth, transaction sequences), interaction-based testing may be more natural. Combine Nullables for most infrastructure with mocks for interaction-heavy boundaries. See references/comparison-and-tradeoffs.md.
- Legacy codebases: Start from outside in (Descend the Ladder) or inside out (Climb the Ladder). See references/legacy-and-adoption.md.
Reference material
Consult these for detailed patterns, code examples, and guidance:
- Philosophy and foundational patterns: references/philosophy-and-foundations.md — core philosophy (sociable, state-based, production code), narrow tests, overlapping sociable tests, smoke tests, zero-impact instantiation, parameterless instantiation, signature shielding
- Architecture patterns: references/architecture.md — A-Frame architecture, logic sandwich, traffic cop, grow evolutionary seeds
- Logic and infrastructure patterns: references/logic-and-infrastructure.md — easily-visible behavior, testable libraries, collaborator-based isolation, infrastructure wrappers, narrow integration tests, paranoic telemetry
- Nullability core patterns: references/nullability-core.md — Nullables, embedded stub, thin wrapper, configurable responses
- Nullability advanced patterns: references/nullability-advanced.md — output tracking, fake it once you make it, behavior simulation
- Legacy migration and adoption: references/legacy-and-adoption.md — descend/climb the ladder, replace mocks incrementally, throwaway stubs, step-by-step adoption guide
- Comparison and tradeoffs: references/comparison-and-tradeoffs.md — Nullables vs. mocks tables, when each approach fits, common objections and responses
Source
git clone https://github.com/msewell/agent-stuff/blob/main/.archive/skills/testing-with-nullables/SKILL.mdView on GitHub Overview
Nullables are production classes with an infrastructure 'off switch.' Each infrastructure class provides two factories: create() for real I/O and createNull() for IO-suppressed operation at the third-party boundary. Tests exercise the real production code while stubbing only the lowest-level third-party calls, enabling fast, refactoring-friendly tests.
How This Skill Works
Identify the infrastructure boundary and wrap it with an Infrastructure Wrapper if needed. Add an Embedded Stub for the third-party code in the same file. Introduce a createNull() factory that injects the embedded stub and suppresses external I/O. Add configurable responses and track outputs using trackXxx() methods, so tests can assert on state and side effects. Higher-level classes should call createNull() on their dependencies, avoiding new stubs in tests.
When to Use It
- When you want to replace mocks with Nullables to test infrastructure-bound code.
- When you need testable infrastructure code by suppressing I/O at external boundaries.
- When you want to assert on state or output rather than on method call order or interactions.
- When migrating an existing mock-based test suite gradually to Nullables.
- When you require embedded stubs, configurable responses, and behavior simulation for external dependencies.
Quick Start
- Step 1: Identify the infrastructure boundary (the external system) and add a wrapper if none exists.
- Step 2: Implement an Embedded Stub and a createNull() factory that injects it; make responses configurable.
- Step 3: Write narrow tests that exercise real production code and assert on state or tracked output.
Best Practices
- Identify the infrastructure boundary first and design a single wrapper per external system.
- Provide a createNull() factory that returns the same class with IO suppressed at the boundary.
- Embed a minimal third-party stub in the same file to cover only what your code calls.
- Add configurable responses to createNull() so tests can drive behavior by scenario.
- Use trackXxx() methods and an event emitter to enable state-based assertions on outputs.
Example Use Cases
- Test a CommandLine class by using CommandLine.createNull() and a NullWritableStream to verify output state without printing to real stdout.
- Wrap an HTTP client: switch to a Null version that returns configurable responses for different status codes and bodies.
- Test a database wrapper by injecting a stubbed data source through createNull() and asserting on returned state rather than query order.
- Validate a filesystem writer by replacing real I/O with an embedded stub and tracking emitted events.
- Migrate a mock-heavy service: progressively replace mocks with Nullables, keeping other mocks intact while tests exercise real code paths.