mcp -macos-use

The mcp-server-macos-use is a Swift-based MCP server that exposes tooling to control macOS applications via the MacosUseSDK. It listens for MCP commands over standard input/output (stdio) and provides tools to open or activate applications, interact with their UI via simulated clicks and keystrokes, and traverse the accessibility tree for inspection. Through the available tools—macos-use_open_application_and_traverse, macos-use_click_and_traverse, macos-use_type_and_traverse, macos-use_press_key_and_traverse, and macos-use_refresh_traversal—you can automate UI tasks, gather UI state, and integrate with MCP clients like Claude Desktop. Each tool accepts parameters to target a specific application (by name, PID, bundle ID, or file path) and perform actions that are followed by a traversal of the accessibility hierarchy, enabling structured results for clients to consume.

To use the server, ensure it is running and connect your MCP client to its stdio interface. Invoke one of the available tools via the CallTool MCP method, supplying the required parameters (and optional action modifiers) as described in the README. The server will perform the requested action through MacosUseSDK and return the traversal results, enabling you to inspect UI structure, element properties, and current state after the action.

Prerequisites:

• macOS machine with Swift toolchain installed (swift build).
• Access to MacosUseSDK (as a Swift package) referenced by the project.

Installation steps:

1. Clone the repository containing the mcp-server-macos-use source.
2. Install dependencies (as appropriate for the project, typically via Swift Package Manager).
3. Build the server in debug or release mode:

swift build -c debug

4. Run the server binary that is produced:

./.build/debug/mcp-server-macos-use

5. Configure your MCP client (e.g., Claude Desktop) to connect to the server by specifying the path to the executable in the mcpServers section, for example:

{
  "mcpServers": {
    "mcp-server-macos-use": {
      "command": "/path/to/your/project/mcp-server-macos-use/.build/debug/mcp-server-macos-use"
    }
  }
}

Ensure MacosUseSDK is properly linked and available to the built binary at runtime.

Tips and considerations:

• The server relies on the MacosUseSDK for controlling macOS UI elements; ensure the SDK is correctly linked and accessible to the binary.
• The tools perform an accessibility tree traversal after actions by default; you can modify behavior with optional ActionOptions such as traverseBefore, traverseAfter, and onlyVisibleElements if supported by your build.
• When using macos-use_click_and_traverse, provide precise coordinates relative to the target window; for macos-use_type_and_traverse and macos-use_press_key_and_traverse, PID targeting is required.
• If you encounter permissions issues, ensure the running process has accessibility permissions granted in macOS System Settings for Automation.
• Use macos-use_refresh_traversal to fetch current UI state without taking any action, which is useful for polling UI state in automation loops.