What is cli-e2e-test-harness?

A TypeScript-based helper to run a CLI via child_process.spawn, capturing stdout, stderr, and exitCode, with simple assertion utilities.

How do I assert stdout or exit codes?

Use expectOutput(result).toContain(' ') for stdout, or .toExitWith(code) for the exit code; you can also use .toMatchSnapshot() for stdout.

Can I customize the environment for the spawned process?

Yes. Pass an options object to runCLI; you can override env (e.g., NO_COLOR: '1') and other SpawnOptions as needed.

cli-e2e-test-harness

Scanned

npx machina-cli add skill a5c-ai/babysitter/cli-e2e-test-harness --openclaw

Files (1)

SKILL.md

1.3 KB

CLI E2E Test Harness

Set up E2E test harness for CLI applications.

Generated Patterns

import { spawn, SpawnOptions } from 'child_process';

interface CLIResult {
  stdout: string;
  stderr: string;
  exitCode: number | null;
}

export async function runCLI(args: string[], options?: SpawnOptions): Promise<CLIResult> {
  return new Promise((resolve) => {
    const proc = spawn('node', ['./dist/index.js', ...args], {
      env: { ...process.env, NO_COLOR: '1' },
      ...options,
    });

    let stdout = '';
    let stderr = '';

    proc.stdout?.on('data', (data) => { stdout += data; });
    proc.stderr?.on('data', (data) => { stderr += data; });

    proc.on('close', (exitCode) => {
      resolve({ stdout, stderr, exitCode });
    });
  });
}

export function expectOutput(result: CLIResult) {
  return {
    toContain: (text: string) => expect(result.stdout).toContain(text),
    toMatchSnapshot: () => expect(result.stdout).toMatchSnapshot(),
    toExitWith: (code: number) => expect(result.exitCode).toBe(code),
  };
}

Target Processes

cli-unit-integration-testing
mcp-server-testing-suite

Source

git clone https://github.com/a5c-ai/babysitter/blob/main/plugins/babysitter/skills/babysit/process/specializations/cli-mcp-development/skills/cli-e2e-test-harness/SKILL.md

View on GitHub

Overview

This skill provides a reusable E2E test harness for CLI applications by spawning the CLI process and collecting its stdout, stderr, and exit code for assertions. It mirrors the runCLI and expectOutput helpers shown in the example, enabling deterministic CLI validations in tests such as cli-unit-integration-testing and mcp-server-testing-suite.

How This Skill Works

It uses Node's child_process.spawn to run the CLI command (node ./dist/index.js with the provided args) and pipes stdout and stderr into strings. When the process closes, it resolves a CLIResult containing stdout, stderr, and exitCode; you then use the expectOutput helper to assert on the results.

When to Use It

End-to-end validation of a CLI tool built with Node/TypeScript
Testing argument parsing, environment handling, and exit codes
Integrating CLI tests with unit/integration suites like cli-unit-integration-testing
Verifying help text, version output, or other stdout content with snapshots
Testing failure modes and non-zero exit codes

Quick Start

Step 1: Import runCLI and expectOutput from the harness module
Step 2: const result = await runCLI(['start', '--port', '8080'], {});
Step 3: expectOutput(result).toContain('Server started'); expectOutput(result).toExitWith(0);

Best Practices

Use NO_COLOR: '1' in the spawned environment to simplify output as shown in the example
Test both successful and failing invocations to cover exit codes
Leverage toContain and toMatchSnapshot for stable stdout assertions
Pass a stable set of args and environment via runCLI(args, options) for reproducibility
Isolate test runs to avoid side effects (e.g., temporary dirs or mocks)

Example Use Cases

Automated end-to-end tests for cli-unit-integration-testing by spawning the CLI and asserting startup messages
Testing mcp-server-testing-suite commands to ensure correct server CLI behavior
Asserting that --help output contains expected options and descriptions
Snapshotting typical CLI output to detect regressions across releases
Verifying exit codes for both successful runs (0) and error conditions (>0)

Frequently Asked Questions

Add this skill to your agents