jcrawl4ai
Java implementation of MCP Server for Crawl4ai
claude mcp add --transport stdio rainyheart-jcrawl4ai-mcp-server java -jar /path/to/your/jar/file/jcawl4ai-mcp-server-1.0.0.jar \ --env CAWL4AI_BASE_URL="Base URL of the Crawl4ai server (e.g., http://host:port)" \ --env CAWL4AI_API_TOKEN="API token for the Crawl4ai server"
How to use
This JCrawl4ai MCP server is a Spring Boot-based MCP service that interfaces with the Crawl4ai API to perform web crawling tasks. It exposes two main MCP APIs: crawl and task. The crawl API accepts a list of URLs, a crawl strategy, a maximum depth, and an output format to start a crawl job via Crawl4ai. The task API retrieves the crawl results by a given task ID. To operate this server, configure the Crawl4ai base URL and API token in application.properties (or via environment variables if supported by your deployment). Once running, you can trigger crawls and fetch results through the MCP orchestration layer or any MCP client compatible with this server.
How to install
Prerequisites:
- Java JDK 8+ installed
- Maven installed (for building the project)
Installation steps:
- Clone the repository or download the source code.
- Navigate to the project directory.
- Build the project using Maven:
mvn clean install
- Run the produced jar (the exact version may vary based on your build):
java -jar target/jcawl4ai-mcp-server-1.0.0.jar
- Ensure that the application.properties (or environment variables) are configured with:
- cawl4ai.base-url: Base URL of the Crawl4ai server
- cawl4ai.api-token: API token for authentication
Optional: Expose the server via containerization or a process manager as part of your deployment.
Additional notes
- The MCP server expects the Crawl4ai API to be reachable from its runtime environment.
- Set cawl4ai.base-url and cawl4ai.api-token (or corresponding environment variables) to prevent authentication errors.
- The MCP configuration example uses the server name jcawl4ai-mcp-server and enables automatic approval for crawl and task operations.
- Logs are reported to the standard MCP log path; monitor ./target/mcp-stdio-server.log for runtime information.
- If you change the jar name or version, update the args accordingly to point to the correct jar file.
- Ensure network permissions allow outbound requests to the Crawl4ai base URL.
