scrapi

This MCP server exposes ScrAPI integration as an MCP service that lets you scrape web pages using ScrAPI. It provides two main tools: scrape_url_html and scrape_url_markdown. The scrape_url_html tool returns the HTML content of a target URL, which is useful when you need full DOM structure and to preserve complex layouts. The scrape_url_markdown tool returns the page content as Markdown, which is helpful when you want clean, readable text extraction. Both tools accept a required url parameter and an optional browserCommands parameter, which is a JSON-encoded array of browser commands to interact with the page before scraping (e.g., clicking buttons, filling forms, waiting for elements, scrolling). This enables you to bypass bot detection, dynamic loading, or geolocation restrictions by simulating user interactions before extraction.

Prerequisites:

• Docker installed and running (recommended) or Node.js with npm for NPX usage.
• Optional: ScrAPI API key if you need higher concurrency or quota.

Option A: Run via Docker (recommended for quick setup)

1. Ensure Docker is installed and running:

   • macOS: Docker Desktop
   • Windows: Docker Desktop
   • Linux: install docker and start the daemon

2. Run the MCP server using the official image:

docker run -it --rm \
  -e SCRAPI_API_KEY=<YOUR_API_KEY> \
  deventerprisesoftware/scrapi-mcp

3. Provide your ScrAPI API key via environment variable SCRAPI_API_KEY to enable authenticated usage.

Option B: Run via NPX (Node.js environment required)

1. Ensure Node.js and npm are installed.
2. Install and run the MCP server with NPX:

npx -y @deventerprisesoftware/scrapi-mcp

3. When prompted or configured, supply SCRAPI_API_KEY if you have a key; otherwise you can run with limited free usage.

Option C: Build locally from source (if you clone the repo)

1. Ensure Node.js and npm are installed.
2. Install dependencies and start the server as documented in the repo's package.json scripts.
3. Configure environment variables (e.g., SCRAPI_API_KEY) as needed.

Tips and notes:

• If you run behind a firewall or proxy, ensure outbound access to ScrAPI services is allowed.
• The SCRAPI_API_KEY env var is optional for limited usage but required for higher concurrency or quotas.
• For the Docker setup, you can customize the environment variable mapping or add other ScrAPI-related config as needed.
• The browserCommands input should be a JSON array string, with each item representing a command such as click, input, scroll, wait, waitfor, or javascript as documented by ScrAPI.
• If you encounter rate limits, consider upgrading your ScrAPI plan or introducing throttling in your client.