pytest-aitest is a Python MCP server that integrates with the pytest-aitest plugin to enable AI-assisted testing of MCP interfaces. It lets you run test-driven development workflows for your MCP servers, prompts, tools, and skills by pairing natural language prompts with tool descriptions and prompts that an LLM can execute. The server runs as a Python module and leverages your existing test suite to validate how agents interact with your MCP endpoints, ensuring that tool usage, schemas, and prompts align with your design goals. You can use the included Agent, Provider, and MCPServer abstractions to spin up a test harness, evaluate model interactions, and analyze results with AI-driven feedback on tool descriptions, costs, and prompts. The workflow emphasizes testing interfaces rather than code paths, helping you improve tool interfaces before production use.

To use pytest-aitest, install the MCP server in your environment and run your tests through the MCP harness. The tool provides multi-provider support and AI analysis to suggest optimizations. For example, you can bundle your MCP server into an Agent with a specific LLM model, execute a test prompt, and inspect whether the agent correctly invoked your tool (e.g., selecting the appropriate action like get_balance or submit_order). The AI analysis will highlight potential improvements in tool descriptions, prompts, or cost management, guiding you in refining your MCP interfaces and test prompts.

Prerequisites:

• Python 3.11+ (for pytest-aitest and its plugins)
• pytest installed in your environment
• Access to a Python environment (virtualenv, pyenv, or container)

Option A: Install via the uvx environment manager (recommended for quick setup):

• Ensure uv is installed in your system
• Run: uv add pytest-aitest

  Optional: install extra capabilities, e.g., Copilot support if available

Option B: Install directly with pip (standalone Python environment):

• Create a virtual environment: python -m venv venv source venv/bin/activate # On Windows use venv\Scripts\activate
• Install the package: pip install pytest-aitest
• Run tests via pytest as you normally would, e.g.: pytest

Option C: Install with pipx for isolated CLI access (if you prefer):

• pipx install pytest-aitest
• Run your tests with pytest as usual, ensuring the plugin is discoverable in your environment.

Tips and common considerations:

• Ensure Python 3.11+ is active in your environment to avoid compatibility issues with pytest-aitest.
• When using cloud LLM providers (Azure, OpenAI, etc.), configure credentials and endpoints as you normally would for your provider integration. The AI analysis features will reference model performance, tool usage, and cost estimates.
• If you enable Copilot-backed providers, you may want to use the Copilot-specific installation option (pytest-aitest[copilot]) to ensure all dependencies are available.
• The MCP server is run as a Python module (python -m pytest_aitest), so ensure your PYTHONPATH and module paths are correct in your execution environment.
• For CI setups, cache Python dependencies and reuse virtual environments to speed up repeated runs.