automation

Automation MCP exposes a comprehensive set of desktop automation capabilities for macOS, enabling AI models and assistants to control the mouse, type with the keyboard, interact with UI elements, and analyze the screen. It includes tools for mouse operations (click, move, scroll, drag), keyboard input (typing and key combinations), screen capture and analysis (screenshots, color sampling, region highlighting, wait-for-image), and window management (focus, move, resize, minimize). The server is designed to integrate with Claude Desktop or other MCP clients via standard MCP transport, allowing you to issue commands and receive results from the underlying macOS automation layer. Once connected, you can perform end-to-end automation tasks such as filling forms, navigating applications, and validating UI states through image-based or coordinate-based interactions.

To use the MCP server, you typically run it with Bun (as shown in the quick start) and then connect your MCP client to the automation server. The provided toolset covers the common automation scenarios: moving the cursor to precise coordinates, clicking or typing with modifiers, capturing screenshots for verification, querying window information, and waiting for specific images to appear on screen. The server enforces macOS permissions (Accessibility and Screen Recording) to enable full functionality, so you should grant these permissions during the initial setup. You can also integrate this MCP server with Claude Desktop or other automation workflows through MCP configuration snippets that map a logical server name to the Bun command and target script.

Prerequisites:

• macOS with the ability to grant Accessibility and Screen Recording permissions
• Bun runtime installed (curl -fsSL https://bun.sh/install | bash)
• Git (optional, for cloning the repository)

Step-by-step installation:

1. Clone the repository (if you haven’t already):

git clone https://github.com/ashwwwin/automation-mcp.git
cd automation-mcp

2. Install dependencies using Bun:

bun install

3. Start the server (choose the stdio or HTTP transport as needed):

# HTTP transport (recommended for web apps)
bun run index.ts

# Or start with stdio transport (for command line tools)
bun run index.ts --stdio

4. (Optional) If you plan to integrate with Claude Desktop or another MCP client, prepare your MCP config to point to this server (see mcp_config). Ensure macOS permissions are granted when prompted.

Notes:

• The server expects macOS Accessibility and Screen Recording permissions for full functionality. If features don’t work, recheck permissions in System Settings.
• If you move the project, adjust the path in the MCP config to point to the correct index.ts location.
• Use the --stdio flag when you want a CLI-oriented transport; use the default HTTP transport for browser/web integrations.

Tips and common issues:

• Permissions: Ensure Accessibility and Screen Recording are granted. If macOS blocks actions, re-open System Settings to grant permissions and restart the server.
• Transport choice: Use HTTP transport for web apps and --stdio for CLI tools. Depending on your environment, one may perform better or be easier to debug.
• On macOS, some automations may require additional permissions or security prompts; run the app at least once to trigger the prompts and complete the grant flow.
• If changing the project path, update the MCP configuration accordingly to avoid command-not-found errors.
• For image-based waits (waitForImage), provide reliable templates and handle slight UI variations to reduce false negatives.
• When debugging, inspect the logs for permission-related errors or missing coordinate data, and verify the target window/app states before issuing actions.