conformance

This MCP server README describes the conformance test framework for MCP implementations. The server component runs a suite of conformance scenarios against a target MCP server by acting as a client to the server under test. You can run the full set of server scenarios or invoke a single scenario to verify behavior such as initialization, tool invocation, resource handling, and more. The framework can be integrated into CI via the provided GitHub Action, and it also supports listing available scenarios to understand what can be tested. To use it, start the conformance server runner (via npx) and point it at the MCP server URL you want to test. The runner will exercise defined server scenarios and produce structured results including checks, and any server-side interactions captured during testing.

Prerequisites:

• Node.js (recommended: LTS version, e.g., 18.x or 20.x) with npm installed
• Internet access to fetch the conformance package

Install and run the conformance server tests:

# Ensure you have Node.js and npm installed
node -v
npm -v

# Run the conformance server tests directly with npx (no global install required)
npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp

If you prefer to customize the invocation (e.g., different URL or port), adjust the --url flag accordingly:

npx @modelcontextprotocol/conformance server --url http://<host>:<port>/mcp

For help and additional options, you can also run:

npx @modelcontextprotocol/conformance --help

Tips and considerations:

• This framework is actively developing; expect breaking changes and evolving features.
• Use the --expected-failures option to codify known regressions and keep CI stable.
• Server testing results are written under results/server-<scenario>-<timestamp>/ with checks.json documenting pass/fail per conformance check.
• If testing locally, ensure the target MCP server is reachable at the URL specified by --url before starting the conformance runner.
• For CI, you can integrate the provided GitHub Action as shown in the README to run server or client tests as part of your workflow.
• You can list available scenarios with npx @modelcontextprotocol/conformance list, or filter to server scenarios with the appropriate flags.
• The framework uses environment variables like MCP_CONFORMANCE_SCENARIO and MCP_CONFORMANCE_CONTEXT to pass scenario metadata when needed by client scenarios.