mcp -requests

This MCP server provides HTTP request capabilities for LLMs to fetch and process web content. It exposes three main tools: fetch, fetch_to_file, and http_request. The fetch tool retrieves web content and can convert it to Markdown, while also offering basic or strict cleaning to remove non-display elements. The fetch_to_file tool saves fetched content to disk without feeding it into the LLM context, useful for large payloads or preserving content for later processing. The http_request tool is a generic HTTP client that supports all standard methods (GET, POST, PUT, PATCH, DELETE), custom headers, and both text and JSON bodies; it returns the full HTTP response (status, headers, and body) for downstream processing in your LLM workflows. The server also supports custom User-Agent strings, random User-Agent generation, and an optional MCP Root workspace capability for file operations via the --use-root flag.

To use these capabilities, run the MCP server with the Python module and invoke the subcommands via the mcp_server_requests entry point. For example, the fetch subcommand can be exercised as: python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]. The http_request tool can be accessed through the unified http_request interface, where you specify url, method, headers, and data or json payloads. If you need to save fetched content to a file, use fetch_to_file with a target path; when --use-root is enabled, relative paths are resolved against the workspace root. Use --user-agent or --random-user-agent to customize the User-Agent, and --use-root with --allow-external-file-access to enable operations outside the workspace when supported by your MCP Client.

For workflows requiring root-based file access, ensure clients support the root capability and consider enabling --use-root and related options accordingly. When in doubt, start with default settings and gradually enable features like random User-Agent or root mode, testing each step with a simple URL to validate responses.

Prerequisites:

• Python 3.8+ and pip
• Git

1. Clone the repository:

git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests

2. Create and activate a Python virtual environment (recommended):

python3 -m venv venv
source venv/bin/activate  # on Windows use: venv\Scripts\activate

3. Install the package from the repository (repository root must contain setup.py or pyproject.toml as per project):

pip install .

4. Run the MCP server (examples):

# Start the server using the mcp_server_requests module
python -m mcp_server_requests

# Or using the MCP configuration style (if you prefer a config-driven approach)
# Ensure the server is discoverable by your MCP runner

Prerequisites notes:

• Ensure network access is allowed for outbound HTTP requests when using fetch or http_request.
• If you plan to use root-based file access, your MCP Client must support the root capability and you may need to pass --use-root and --allow-external-file-access accordingly.
• The fetch tool supports different return_content options (raw, basic_clean, strict_clean, markdown).

Tips and common considerations:

• Use --user-agent or --random-user-agent to control the User-Agent. If you enable --force-user-agent, the command-line-provided UA takes precedence over any LLM-provided UA.
• When using fetch_to_file with --use-root, you can provide relative paths (relative to workspace root) or absolute paths depending on the flag. Outside-workspace access is only allowed if --allow-external-file-access is enabled and supported by the client.
• The http_request tool returns a full HTTP response (status line, headers, and body). When sending JSON payloads, use the json parameter and the Content-Type will be set automatically to application/json.
• The older per-method tools (http_get, http_post, etc.) have been replaced by the unified http_request interface. If you have existing scripts, migrate them to http_request by setting the method field accordingly.
• Ensure your environment has network access and any required certificates for HTTPS requests when using fetch or http_request.