mcp-read-website-fast

This MCP server converts web pages into clean Markdown by performing local crawling, noise removal, and content extraction using Mozilla Readability, then rendering to Markdown with Turndown (including GitHub Flavored Markdown support). It preserves links for knowledge graphs and can optionally chunk content for downstream processing. The server exposes a single main tool called read_website which fetches a URL and returns the cleaned Markdown. It also provides a status endpoint to inspect cache statistics and a clear-cache endpoint to manage local storage. To use it in your MCP client, add the read-website-fast server with the provided command, then call read_website with a URL to fetch and convert that page. You can adjust depth, concurrency, and output format via the CLI options described in the development docs.

Prerequisites:

• Node.js and npm installed on your machine.
• Access to the MCP environment where you’ll run the server (local dev or hosting environment).

Installation steps:

1. Install the MCP server package via npx (or install locally if you prefer):

npx -y @just-every/mcp-read-website-fast

2. Alternatively, integrate via your MCP client JSON by referencing the npm package:

{
  "mcpServers": {
    "read-website-fast": {
      "command": "npx",
      "args": ["-y", "@just-every/mcp-read-website-fast"]
    }
  }
}

3. If you’re developing locally, you can install dependencies and build as described in the repository:

npm install
npm run build

4. Add the server to your MCP client configuration (e.g., Claude, VS Code, Cursor) using the same command/args shown above.

Notes:

• You may want to configure caching and robots.txt handling through the provided CLI options when launching read_website-fast.
• The server is designed to start quickly with lazy loading and a restart wrapper for reliability.

Tips and caveats:

• Available tool: read_website(url, pages?) to fetch a webpage and convert it to Markdown. You can set pages to control how many pages to crawl from a site (default 1, max 100).
• Cache endpoints:
  • read-website-fast://status — retrieve cache statistics
  • read-website-fast://clear-cache — clear the cache directory
• CLI options include: --pages, --concurrency, --no-robots, --all-origins, -u/--user-agent, --cache-dir, -t/--timeout, -o/--output (markdown/json/both).
• Output defaults to Markdown; use --output json to get metadata, or --output both for both outputs.
• If a site blocks automated requests, try setting a custom user agent with -u, or enable --no-robots if appropriate for your use case.
• This MCP server is designed to minimize token usage by delivering clean Markdown suitable for integration into AI pipelines.