linkedin-web-scrapper

This MCP server exposes LinkedIn scraping capabilities as MCP tools that can be used by AI assistants. It uses Playwright to drive a browser, perform LinkedIn people searches with keyword and filter parameters, and extract profile information from the results. The server persists login cookies to avoid repeated authentication and supports adaptive selectors to cope with UI changes. Tools are accessed via the MCP protocol over stdio, returning structured JSON with profile data and filter outcomes.

To interact with the server, connect your MCP client to the stdio transport provided by the server. Use the available MCP tools described in the README, most notably the search-linkedin-people tool, which accepts keywords, location filters, and network filters to return a list of profiles along with basic details such as name, profile URL, and headline. You can also tune the browser options (headless, slowMo) to suit development or debugging workflows. Remember to maintain compliance with LinkedIn’s terms of service while using automated scraping tools.

Prerequisites:

• Node.js (recommended LTS) and npm installed on your system
• Git installed
• Access to a terminal/command prompt

Installation steps:

1. Clone the repository and navigate to it:

• git clone https://github.com/Phicks-debug/linkedin-web-scrapper.git
• cd linkedin-web-scrapper-mcp-server

2. Install dependencies:

• npm install

3. Install Playwright browsers (Chromium/Firefox/WebKit):

• npx playwright install

4. Configure your LinkedIn credentials:

• cp config.example.json config.json
• Edit config.json to include your LinkedIn email/password and browser settings as shown in the example: { "linkedin": { "email": "your-linkedin-email@email.com", "password": "your-linkedin-password" }, "browser": { "headless": false, "slowMo": 1000, "cookiesPath": "./cookies.json" } }

5. Build the server (TypeScript to JavaScript):

• npm run build

6. Start the MCP server (example):

• node dist/index.js

Optional for development with auto-rebuild:

• npm run watch

Tips and considerations:

• Session persistence: The server saves LinkedIn session cookies to cookies.json to minimize login prompts; ensure the path in config matches where cookies.json is stored.
• Location and network filters: The search-linkedin-people tool supports geo URN codes or location strings for location filtering and uses network codes (F for 1st, S for 2nd, O for 3rd+). If a location is omitted, defaults to a generic search location.
• Adaptive selectors: LinkedIn’s UI changes can break selectors; the server includes adaptive strategies to handle minor changes without code updates.
• Compliance: Use scraping tools responsibly and adhere to LinkedIn’s terms of service. Implement rate limiting and respect automation policies.
• Debugging: Run with headless: false and slower animations (slowMo) to observe actions during development. Use the MCP Inspector tool to validate tool inputs/outputs.
• Environment variables: If deploying in different environments, you can override credentials and browser settings via environment variables or the provided config.json. Ensure the paths for cookies and dist are correctly configured in your deployment context.