mcp-interviewer

MCP Interviewer is a Python CLI tool that helps you test and evaluate your MCP servers by driving them with an interviewer process. It can perform constraint checking, inspect server capabilities, run functional tests, and generate both Markdown reports and raw data JSON. Use it to verify that your MCP server adheres to constraints, to generate structured evaluations of tools and prompts, and to produce a comprehensive interview report for sharing with teams or stakeholders. The tool is designed to run the target MCP server as a child process, isolating it from your host whenever possible (for example, by running the server inside a container).

To use it, install the interviewer and then provide the MCP server command you want to interview. You can run constraint checks, enable functional testing with --test, and optionally perform LLM-based evaluations with --judge-tools and --judge-test. The output includes a Markdown report (mcp-interview.md) and a corresponding JSON data file (mcp-interview.json) that captures all observed metrics and results.

Prerequisites:

• Python 3.11+ (or compatible environment)
• pip

Install MCP Interviewer as a CLI tool:

pip install mcp-interviewer

Usage as a CLI tool (example):

# Command to run your MCP server (replace with your actual server command)
mcp-interviewer <your-mcp-server-command>

If you prefer to bring in the server as a dependency in uv (uvx) or via a container, follow the respective installation flow shown in the README:

• Bring Your Own Models for interacting with the server via a client
• Use uv (via uvx) or containerized commands to run the MCP server inside a container for isolation

Note: The examples in the README illustrate how to run the MCP interviewer against a server command and generate reports. Replace the placeholder <your-mcp-server-command> with the command that starts your MCP server.

Tips and considerations:

• The interviewer executes the provided MCP server command as a child process. Prefer containerized execution (e.g., Docker) to minimize host system impact.
• Common environment variable patterns in examples include NPX_CONTAINER and UVX_CONTAINER to wrap commands for remote or containerized execution.
• When using --test, the interviewer will generate a test plan with your LLM and execute it against your server, collecting statistics about tool usage and behavior.
• Use --reports to tailor the output sections (e.g., SI for Server Info, TS for Tool Statistics, FT for Functional Tests, CV for Constraint Violations).
• Ensure your MCP server command is robust in non-interactive environments since the interviewer relies on child-process execution.
• If you’re integrating as a dependency (uv or pip), consult the README sections Bring Your Own Models and Python usage for more advanced usage patterns.