web-retrieval

The Web Retrieval MCP is designed to analyze and extract the frontend structure of any given web page. By supplying a URL, you can obtain a detailed report of the page layout, navigation, content areas, forms, images, and link relationships, along with style features such as responsive behavior. This makes it easier to understand how a site is built and how its various components interact, which is useful for design analysis, accessibility reviews, and frontend-backend integration planning.

Once deployed, you can access the MCP through either the Stdio (local development) mode or the SSE mode via Supergateway. Use the analyze_web_structure tool to run a deep structural analysis on a target URL. The tool will return structured data including layout sections (header, footer, sidebar), heading hierarchy (H1-H6), navigation structure, primary content regions, forms and their fields, image metadata, and API/HTTP endpoints discovered on the page. You can also see evidence of CSS frameworks or frontend tooling clues to infer the technology stack behind the page.

Prerequisites:

• Node.js and npm installed on your machine
• Basic familiarity with MCP workflow and CLI usage

Install the project dependencies:

npm install

Build the project (production-ready bundle):

npm run build

Run the MCP server locally (Stdio mode):

npm start

Optionally, run via an SSE gateway (if you have Supergateway installed):

npm install -g supergateway
npm run sse

Configure Claude (MCP client) to point to the server using the Stdio or SSE configuration shown in the README. For Stdio, reference the built index in path/to/web-retrieval-mcp/build/index.js. For SSE, point to http://localhost:3100/sse with an appropriate timeout.

Notes and tips:

• This MCP has a focus on extracting page structure, including layout sections, navigation, content regions, forms, and image resources.
• Timeouts: The service should set reasonable timeouts for page fetching (e.g., 10 seconds) to avoid long waits.
• User-Agent: A standard browser User-Agent helps with compatibility during page fetches.
• Resource limits: Consider setting maximums for the number of links, forms, and images to avoid excessive memory usage on large pages.
• Security: Validate and sanitize all input URLs and avoid following redirects indefinitely.
• Output: The analyze_web_structure tool returns a structured report that can be extended with custom fields if you integrate with MCP SDK.