webfetch

WebFetch.MCP is a self-hosted MCP server that provides live web access for local AI workflows. It uses SearxNG as a backend search and Mozilla Readability-like content extraction to deliver clean, structured text suitable for LLM consumption. With this server, you can perform configurable web searches across 70+ engines, fetch full article content from URLs, and fine-tune results by source, data type, language, and freshness. The MCP is designed to work with LM Studio and can be plugged into your local toolset without API keys, quotas, or external tracking. After starting the server, you can connect it to LM Studio (or other MCP clients) by adding an entry for webfetch that points to the local Node.js process, enabling “web_search” and “web_fetch” tools for your experiments and research tasks.

Prerequisites:

• Node.js 18+ installed on your system
• Git installed
• (Optional) Docker and Docker Compose if you prefer a containerized setup

1. Clone the repository

   • git clone https://github.com/manull/webfetch-mcp.git
   • cd webfetch-mcp

2. Install dependencies

   • npm install

3. Run the server

   • node server.mjs
   • Or use a process manager (recommended for production):
     • pm2 start server.mjs --name webfetch

4. Connect to LM Studio or another MCP client

   • In LM Studio, add an MCP server entry with the following configuration: { "mcpServers": { "webfetch": { "command": "node", "args": ["/full/path/to/webfetch-mcp/server.mjs"], "env": { "SEARXNG_BASE": "http://localhost:8080", "DEBUG": "false" } } } }

5. Validate

   • Test a quick search: curl "http://localhost:8080/search?format=json&q=test&count=5"
   • Test a fetch: curl "http://localhost:8080/fetch?url=https://example.com/article"

Tips and considerations:

• SEARXNG_BASE should point to your running SearxNG instance. If you’re running SearxNG in a Docker setup, ensure the container is accessible at the specified URL.
• The server relies on Mozilla Readability for clean extraction; some pages may require alternative handling for highly dynamic sites.
• Rate limiting is designed for research use; monitor usage to avoid overwhelming your local search engine or the target sites.
• If you run into CORS or network issues, verify that localhost ports (e.g., 8080 for SearxNG) are accessible from the server process.
• You can customize available engines and language/region preferences through the configuration exposed by the MCP server and the LM Studio settings.