mcp -webscan

This MCP Webscan server provides a set of web content analysis tools that can fetch pages, extract links, crawl sites, check for broken links, find patterns in URLs, and generate sitemaps. The server runs over the standard MCP setup using stdio transport, making it compatible with MCP clients like Claude Desktop. Once started, you can invoke the available tools to process web content and return structured results such as Markdown conversions, link lists, crawl trees, and sitemap data. Typical workflows involve fetching a page and converting it to Markdown for easy reading, then using link-related tools to analyze the page's links or to crawl an entire site to discover additional content. The pattern-matching and sitemap generation features enable deeper analysis and automated discovery for SEO or content auditing tasks.

Prerequisites:

• Node.js >= 18
• npm

Manual installation steps:

1. Clone the repository: git clone <repository-url> cd mcp-server-webscan

2. Install dependencies: npm install

3. Build the project: npm run build

4. Start the server: npm start

If you prefer the Smithery installation flow (as documented in the README):

1. Install via Smithery: npx -y @smithery/cli install mcp-server-webscan --client claude

Note: The server exposes its tools via MCP and can be configured in your MCP client with the appropriate mcpServers configuration.

Tips and common issues:

• Ensure Node.js version meets the minimum requirement (Node.js >= 18).
• The server is designed to run with stdio transport; ensure your MCP client is configured to use stdio.
• You can customize logging by setting the LOG_LEVEL environment variable (e.g., info, debug).
• If you modify the build output path, update the mcpServers.args accordingly in your MCP client configuration.
• Common errors include invalid parameters, network errors, and content parsing errors; the server returns MCP-formatted error messages to aid debugging.
• For production deployments, consider setting NODE_ENV to production and adjusting log verbosity as needed.