Get the FREE Ultimate OpenClaw Setup Guide →

testing-with-nullables

npx machina-cli add skill msewell/agent-stuff/testing-with-nullables --openclaw
Files (1)
SKILL.md
7.6 KB

Testing With Nullables

Core concept

Nullables are production classes with an infrastructure "off switch." Each infrastructure class provides two factories:

  • create() — production instance with real I/O
  • createNull() — same class, same code paths, but I/O suppressed at the third-party boundary

Tests exercise real production code. Only the lowest-level third-party calls are stubbed.

class CommandLine {
  private constructor(private stdout: WritableStream, private _args: string[]) {}

  static create(): CommandLine {
    return new CommandLine(process.stdout, process.argv.slice(2));
  }

  static createNull(options?: { args?: string[] }): CommandLine {
    return new CommandLine(new NullWritableStream(), options?.args ?? []);
  }

  write(text: string): void { this.stdout.write(text); }
  args(): string[] { return this._args; }
}

Workflow: Making a dependency testable with Nullables

  1. Identify the infrastructure boundary. Find the class that talks to an external system (HTTP, file system, database, stdout).
  2. Create an Infrastructure Wrapper if one doesn't exist. One wrapper per external system. Translate between external formats and domain types.
  3. Add an Embedded Stub. Stub the third-party code (not yours) with a minimal implementation covering only the methods your code calls. Keep it in the same file.
  4. Add createNull() factory that injects the embedded stub instead of the real third-party dependency. Both factories return the same class.
  5. Add Configurable Responses to createNull() so tests can control what the dependency returns. Name parameters by behavior (verificationStatus), not implementation (httpResponseBody).
  6. Add Output Tracking via trackXxx() methods that record writes using an event emitter. This enables state-based assertions on side effects.
  7. Compose upward. Higher-level classes call createNull() on their dependencies — no new stubs needed (Fake It Once You Make It).

Workflow: Writing tests in the Nullables style

  1. Write narrow tests. One test file per module/class. Each test has a single reason to fail.
  2. Make tests sociable. Exercise real dependencies — don't mock them. If App uses Rot13, tests run the real Rot13.
  3. Assert on state, not interactions. Check return values, object state, or tracked output. Never assert on whether methods were called or in what order.
  4. Use signature shielding. Create test helper functions that wrap createNull() calls. When the factory signature changes, update one helper instead of every test.
  5. Write narrow integration tests for each infrastructure wrapper against real (but local/test-isolated) external systems. Run these in CI, not on every save.
  6. Add smoke tests sparingly. One or two end-to-end tests as a safety net. If you need many, your narrow tests have gaps.

Workflow: Migrating from mocks to Nullables

  1. Pick the most-mocked dependency.
  2. Make it Nullable (follow the workflow above).
  3. Replace its mocks with createNull() in tests — keep other mocks unchanged.
  4. Run tests. They should still pass.
  5. Repeat for the next most-mocked dependency.

Nullables coexist with mocks. No big-bang rewrite needed.

Key rules

  • Stub third-party code, not your code. Your production code runs in full during tests.
  • create() and createNull() return the same class — not a subclass, not a fake.
  • Constructors do no work (Zero-Impact Instantiation). Defer connections, servers, and I/O to explicit methods.
  • Pure logic classes don't need Nullables. Only infrastructure wrappers need createNull().
  • Output Tracking works on both real and Nulled instances. It's useful in production too (monitoring, auditing).
  • Design configurable responses from the consumer's perspective, not the implementation's.

Architecture recommendation

Use A-Frame Architecture for maximum testability:

      Application / UI
       /          \
    Logic    Infrastructure
       \          /
        Values (shared)
  • Application layer: Coordinates logic and infrastructure via Logic Sandwich (read → process → write). No business logic, no I/O details.
  • Logic layer: Pure business rules. Depends only on values. Test with simple state-based assertions.
  • Infrastructure layer: Wraps external systems. One wrapper per system. Made testable via Nullables.

Edge cases

  • Languages requiring interfaces (Java, C#): Use Thin Wrapper pattern — define a custom interface covering only methods you use, with real and null implementations. See references/nullability-core.md.
  • Event-driven applications: Use Behavior Simulation — add simulateXxx() methods to infrastructure wrappers that trigger the same handler as real events. See references/nullability-advanced.md.
  • Complex stateful protocols: For deeply stateful interactions (multi-step auth, transaction sequences), interaction-based testing may be more natural. Combine Nullables for most infrastructure with mocks for interaction-heavy boundaries. See references/comparison-and-tradeoffs.md.
  • Legacy codebases: Start from outside in (Descend the Ladder) or inside out (Climb the Ladder). See references/legacy-and-adoption.md.

Reference material

Consult these for detailed patterns, code examples, and guidance:

  • Philosophy and foundational patterns: references/philosophy-and-foundations.md — core philosophy (sociable, state-based, production code), narrow tests, overlapping sociable tests, smoke tests, zero-impact instantiation, parameterless instantiation, signature shielding
  • Architecture patterns: references/architecture.md — A-Frame architecture, logic sandwich, traffic cop, grow evolutionary seeds
  • Logic and infrastructure patterns: references/logic-and-infrastructure.md — easily-visible behavior, testable libraries, collaborator-based isolation, infrastructure wrappers, narrow integration tests, paranoic telemetry
  • Nullability core patterns: references/nullability-core.md — Nullables, embedded stub, thin wrapper, configurable responses
  • Nullability advanced patterns: references/nullability-advanced.md — output tracking, fake it once you make it, behavior simulation
  • Legacy migration and adoption: references/legacy-and-adoption.md — descend/climb the ladder, replace mocks incrementally, throwaway stubs, step-by-step adoption guide
  • Comparison and tradeoffs: references/comparison-and-tradeoffs.md — Nullables vs. mocks tables, when each approach fits, common objections and responses

Source

git clone https://github.com/msewell/agent-stuff/blob/main/.archive/skills/testing-with-nullables/SKILL.mdView on GitHub

Overview

Nullables are production classes with an infrastructure 'off switch.' Each infrastructure class provides two factories: create() for real I/O and createNull() for IO-suppressed operation at the third-party boundary. Tests exercise the real production code while stubbing only the lowest-level third-party calls, enabling fast, refactoring-friendly tests.

How This Skill Works

Identify the infrastructure boundary and wrap it with an Infrastructure Wrapper if needed. Add an Embedded Stub for the third-party code in the same file. Introduce a createNull() factory that injects the embedded stub and suppresses external I/O. Add configurable responses and track outputs using trackXxx() methods, so tests can assert on state and side effects. Higher-level classes should call createNull() on their dependencies, avoiding new stubs in tests.

When to Use It

  • When you want to replace mocks with Nullables to test infrastructure-bound code.
  • When you need testable infrastructure code by suppressing I/O at external boundaries.
  • When you want to assert on state or output rather than on method call order or interactions.
  • When migrating an existing mock-based test suite gradually to Nullables.
  • When you require embedded stubs, configurable responses, and behavior simulation for external dependencies.

Quick Start

  1. Step 1: Identify the infrastructure boundary (the external system) and add a wrapper if none exists.
  2. Step 2: Implement an Embedded Stub and a createNull() factory that injects it; make responses configurable.
  3. Step 3: Write narrow tests that exercise real production code and assert on state or tracked output.

Best Practices

  • Identify the infrastructure boundary first and design a single wrapper per external system.
  • Provide a createNull() factory that returns the same class with IO suppressed at the boundary.
  • Embed a minimal third-party stub in the same file to cover only what your code calls.
  • Add configurable responses to createNull() so tests can drive behavior by scenario.
  • Use trackXxx() methods and an event emitter to enable state-based assertions on outputs.

Example Use Cases

  • Test a CommandLine class by using CommandLine.createNull() and a NullWritableStream to verify output state without printing to real stdout.
  • Wrap an HTTP client: switch to a Null version that returns configurable responses for different status codes and bodies.
  • Test a database wrapper by injecting a stubbed data source through createNull() and asserting on returned state rather than query order.
  • Validate a filesystem writer by replacing real I/O with an embedded stub and tracking emitted events.
  • Migrate a mock-heavy service: progressively replace mocks with Nullables, keeping other mocks intact while tests exercise real code paths.

Frequently Asked Questions

Add this skill to your agents
Sponsor this space

Reach thousands of developers