What is cli-snapshot-tester?

A utility to snapshot test CLI outputs with an update workflow and diff reporting, ensuring CLI behavior stays stable.

How do I update snapshots?

Set the environment variable UPDATE_SNAPSHOTS when running tests to write new outputs to the snapshot files.

Where are snapshots stored and how are they named?

Snapshots are stored under a __snapshots__ directory with files named {name}.txt corresponding to each test case.

cli-snapshot-tester

Flagged

{"isSafe":false,"isSuspicious":true,"riskLevel":"high","findings":[{"category":"shell_command","severity":"high","description":"The code executes arbitrary shell commands via execSync(cmd, ...). If cmd comes from untrusted input, this enables command injection and could lead to destructive actions or system compromise (e.g., rm -rf, data loss).","evidence":"const output = execSync(cmd, { encoding: 'utf-8' });"},{"category":"prompt_injection","severity":"medium","description":"The cmd argument is a string sourced from outside and is executed directly. In AI harnesses or code-generation contexts, this creates a potential prompt injection or manipulation vector where external inputs influence the harness behavior.","evidence":"export function runCliSnapshot(cmd: string, name: string): void { ... execSync(cmd, { encoding: 'utf-8' }); }"}],"summary":"The skill content includes a function that runs arbitrary shell commands through execSync using an unvalidated input parameter. This poses command injection risks and potential prompt-injection vectors in AI-related contexts. Recommend restricting commands, validating/sanitizing inputs, or using safer execution patterns (e.g., execFile/spawn with explicit args, or avoiding shell execution entirely)."}

npx machina-cli add skill a5c-ai/babysitter/cli-snapshot-tester --openclaw

Files (1)

SKILL.md

906 B

CLI Snapshot Tester

Set up snapshot testing for CLI output.

Generated Patterns

import { execSync } from 'child_process';
import fs from 'fs';
import path from 'path';

export function runCliSnapshot(cmd: string, name: string): void {
  const output = execSync(cmd, { encoding: 'utf-8' });
  const snapshotPath = path.join('__snapshots__', `${name}.txt`);

  if (process.env.UPDATE_SNAPSHOTS) {
    fs.mkdirSync(path.dirname(snapshotPath), { recursive: true });
    fs.writeFileSync(snapshotPath, output);
    return;
  }

  const expected = fs.readFileSync(snapshotPath, 'utf-8');
  expect(output).toBe(expected);
}

Target Processes

cli-unit-integration-testing
cli-documentation-generation

Source

git clone https://github.com/a5c-ai/babysitter/blob/main/plugins/babysitter/skills/babysit/process/specializations/cli-mcp-development/skills/cli-snapshot-tester/SKILL.md

View on GitHub

Overview

CLI Snapshot Tester adds deterministic checks for CLI output by capturing stdout and comparing against stored snapshots. It saves outputs under __snapshots__/{name}.txt, and supports an UPDATE_SNAPSHOTS mode to refresh snapshots. This helps guard against regressions while enabling clear diff reporting.

How This Skill Works

At runtime, runCliSnapshot(cmd, name) executes the command with execSync, captures the UTF-8 output, and builds a path to __snapshots__/{name}.txt. If UPDATE_SNAPSHOTS is set, it creates the directory and writes the new output; otherwise it reads the stored snapshot and asserts that the actual output matches.

When to Use It

Validating stable CLI outputs in unit/integration tests
Generating and validating CLI docs with example outputs
Catching regressions after CLI feature changes
Running in CI to enforce output consistency with diff reporting
Updating snapshots during development when outputs legitimately change

Quick Start

Step 1: Implement runCliSnapshot(cmd, name) in your test suite (uses execSync, fs, path).
Step 2: Call runCliSnapshot('<your-cli-command>', '<name>') for each scenario; set UPDATE_SNAPSHOTS to refresh when needed.
Step 3: Run tests to compare outputs; snapshots live under __snapshots__/{name}.txt and will diff on mismatches.

Best Practices

Use descriptive snapshot names for each CLI command scenario
Keep snapshots focused on stdout; handle stderr separately if needed
Commit snapshots with tests to prevent drift
Run with UPDATE_SNAPSHOTS only in local development or as an explicit opt-in
Ensure CLI commands are deterministic (avoid dynamic timestamps, locale differences)

Example Use Cases

Snapshot test for a tool's --help output to lock help text
Snapshot test for a list/scan command with stable formatting
Using snapshots to document CLI outputs during docs generation
CI-based regression test to fail when error messages change unexpectedly
Cross-version checks to ensure CLI behavior remains consistent

Frequently Asked Questions

Add this skill to your agents