mcp-factcheck

The MCP Fact-Check server provides a suite of tools designed to validate and fact-check MCP-related content against the Model Context Protocol specifications. It includes a semantic search capability over the MCP specifications, an AI-powered validator for multi-claim content, and a quick single-claim verifier. Use it to verify documentation, tutorials, or statements for accuracy, and to migrate content when moving between MCP specification versions. The server exposes four main tools: check_mcp_claim for comprehensive validation of complex content and decomposing multi-claim statements; check_mcp_quick_fact for fast, single-claim verification; search_spec to find relevant sections of MCP specs using semantic similarity; and list_spec_versions to view available specification versions with current status. For ongoing workflows, you can also use the migrate-mcp-content prompt to guide content migration between specification versions while preserving tone and voice. These tools combine embedding-based search, AI validation, and structured prompts to provide corrective suggestions and confidence scores.

Prerequisites: - A supported runtime environment (Go for server, or access via the MCP registry) - Git to clone repositories (if building from source) - Access to an OpenAI API key for embeddings and validation (OPENAI_API_KEY)

Installation steps:

1. Install dependencies and toolchain

   • Install Go (https://golang.org/dl/) for building the server and utilities.
   • Ensure you have a modern shell (bash/zsh) and jq for meta operations.

2. Clone or obtain the server source

   • If you're building from source, clone the MCP Fact-Check server repository: git clone https://github.com/modelcontextprotocol/servers.git cd servers/mcp-factcheck

3. Build the server and utilities (from source)

   • Build the main server binary: go build -o bin/mfc ./cmd/server
   • Build the spec loader utility (if needed): go build -o bin/specloader ./utils/cmd
   • Optionally run tests: go test ./...

4. Run the server locally (example)

   • Start the server with your data directory configured: ./bin/mfc --data-dir /path/to/data/embeddings
   • Or use the provided mcp_config to run in your environment, ensuring the paths and API keys are set.

5. Verify installation

   • Use the included test client or curl to hit endpoints and confirm responses align with expected validation outputs.

Notes and tips:

• OPENAI_API_KEY is required for embedding generation and validation; ensure it is available in the environment where the server runs.
• If you plan to visualize traces with Arize Phoenix, install and run Phoenix as described in the Observability section and configure the mcp-factcheck server block accordingly.
• You can view and manage specification versions through the provided tools (search_spec, list_spec_versions) to ensure you’re validating against the correct draft or released versions.
• For production deployments, consider setting up OTLP telemetry and securing the endpoint used by the server. Use the env variable OPENAI_API_KEY securely (e.g., via a secrets manager).
• If you encounter rate limits when querying specifications, consider providing a GITHUB_TOKEN to improve API rate limits when extracting specs.