bellwether

Bellwether is the open-source MCP testing tool designed to catch drift in tool schemas used by AI assistants via the Model Context Protocol. It helps you automatically explore and test MCP tool interfaces, compare against baselines, and surface changes that could cause downstream failures in production. The core workflow focuses on drift detection in CI or local development; advanced workflows enable deeper behavioral testing and contract verification. You can initialize a server, run checks to detect schema drift, and optionally save baselines for future comparisons. Bellwether supports both deterministic drift checks and optional exploratory testing to surface edge cases across MCP tools and prompts.

Prerequisites:

• Node.js and npm installed on your system
• Git access to clone or install via npm registry

Step-by-step installation:

1. Install Bellwether globally (recommended for quick start): """ npm install -g @dotsetlabs/bellwether """
2. Verify installation: """ bellwether --version """
3. Initialize a server for drift testing (example using MCP server placeholder): """ bellwether init npx @mcp/your-server """
4. Run a drift check and optionally save a baseline: """ bellwether check bellwether baseline save """
5. Integrate into CI by adding Bellwether commands to your workflow as shown in the documentation examples.

Notes and tips:

• Bellwether is used to validate MCP tool schemas and detect drift between current tool behavior and a saved baseline.
• You can run in different modes: check (drift detection) and explore (LLM-driven behavioral testing), depending on your needs.
• Environment variables for external AI services (OpenAI, Anthropic, Ollama) are only needed if you enable exploration features; they are not required for core drift checks.
• When using remote MCP servers, you can override headers or pass authentication through the bellwether.yaml configuration or CLI overrides as documented.
• Ensure baselines are committed to your CI to enable consistent drift comparisons across PRs.