mandoline

Mandoline provides an MCP server that exposes a suite of evaluation tools and resources to help AI assistants reflect on, critique, and improve their own performance using Mandoline's evaluation framework.

To use it, integrate the Mandoline MCP server into your assistant as described in the client setup sections. The hosted Mandoline MCP server runs locally at http://localhost:8080 by default when you start it with npm start, and you can configure clients to point at that locally-hosted endpoint during development. The available tools are organized under Health, Metrics, Evaluations, and Resources:

• Health: get_server_health helps verify the MCP server is reachable and healthy.
• Metrics: create_metric, batch_create_metrics, get_metric, get_metrics, update_metric allow you to define and manage evaluation criteria for tasks you care about.
• Evaluations: create_evaluation, batch_create_evaluations, get_evaluation, get_evaluations, update_evaluation enable you to score prompts and responses against your metrics and inspect historical results.
• Resources: llms.txt and mcp provide quick access to Mandoline docs and MCP setup guidance for assistants.

Once configured, your AI assistant can invoke these tools during conversations to evaluate outputs, compare options, and generate structured feedback, improving alignment with your evaluation criteria over time.

Prerequisites

• Node.js 18+ and npm
• A working Mandoline account and API keys if you plan to use Mandoline tools that require authentication

Installation steps

1. Clone the repository and install dependencies

git clone https://github.com/mandoline-ai/mandoline-mcp-server.git
cd mandoline-mcp-server
npm install

2. Build (if the project uses a build step)

npm run build

3. Optional: configure environment for local run

cp .env.example .env.local
# Edit .env.local to customize PORT, LOG_LEVEL, etc.

4. Start the server

npm start

The server will run on http://localhost:8080 by default (unless you override the PORT in your environment).

Using the hosted server If you prefer the hosted Mandoline MCP server, configure clients to point to https://mandoline.ai/mcp as shown in the client setup instructions. Replace any local endpoints accordingly in your assistant integration.

Notes and tips:

• If you run locally, ensure PORT 8080 is open in your environment and not used by another process.
• When updating configuration (e.g., API keys), restart your client tools (Claude Code, Codex, Claude Desktop, Cursor) to pick up changes.
• For local development, ensure you have a valid Mandoline API key if you plan to enable any authenticated features.
• The MCP server supports a variety of tools (health, metrics, evaluations, resources); you can expose or customize endpoints as needed in your deployment.
• Check the official documentation linked in the README for up-to-date usage guidance and compatibility notes.