usescraper

UseScraper MCP Server provides a single tool called 'scrape' that leverages the UseScraper API to extract content from web pages. Once the server is running under MCP, you can invoke the scrape tool through the MCP interface by passing the target URL and optional parameters such as the desired output format (text, html, or markdown), an optional advanced proxy flag to help bypass bot detection, and an extract_object to specify which data to pull from the page. This makes it easy to obtain structured or plain content from pages without leaving the MCP workflow.

To perform a scrape, construct a request to the 'scrape' tool with arguments like: {"name": "scrape", "arguments": {"url": "https://example.com", "format": "markdown"}}. The server will query the page via UseScraper, apply the chosen format, and return the scraped content. If you need more advanced extractions, you can supply extract_object to target specific elements or data on the page, and toggle advanced_proxy if you encounter bot-detection measures.

Prerequisites:

• Node.js (recommended LTS) and npm installed on your system
• git for cloning the repository

Installation steps:

1. Clone the repository git clone https://github.com/your-repo/usescraper-server.git cd usescraper-server

2. Install dependencies npm install

3. Build the server npm run build

4. Run the server (example) npm run start

Optional: If you plan to integrate with Claude Desktop via Smithery, you can install via Smithery as described below:

Smithery installation (optional): npx -y @smithery/cli install usescraper-server --client claude

Tips and notes:

• Ensure you have a UseScraper API key and set it in the environment variable USESCRAPER_API_KEY when configuring the MCP server.
• The server exposes a single tool named 'scrape'. You can adjust the 'format' option to control output (text, html, markdown).
• If you encounter bot detection or IP blocking, consider enabling advanced_proxy in the scrape parameters or using an appropriate proxy configuration supported by UseScraper.
• Debugging MCP servers uses stdio; use the MCP Inspector tool for an easier debugging experience when running locally.
• When deploying, make sure the path in the MCP config points to the built index.js file produced by npm run build.