overlay-companion

Overlay Companion MCP connects your AI assistant to a desktop environment to perform actions via overlays, screen captures, and input automation. Once deployed, you can manage connections to target desktops (local or VM), and configure AI integrations (such as Cherry AI or Claude Desktop) to issue commands like drawing overlays, taking screenshots, clicking or typing on the target desktop, and manipulating the clipboard. The MCP server exposes a web interface and a programmatic API that your AI can reach at the configured MCP URL to coordinate actions between the AI and the desktop surface. Typical workflows include creating visual cues on screen, capturing the current display for analysis, and executing automated interactions across one or multiple monitors through a host-target setup.

Prerequisites:

• Docker or Podman installed on the host machine
• Optional: docker-compose or podman-compose installed
• Internet access to pull images

Step 1: Prepare the host

• Install Docker/Podman if not already installed
• Ensure you have permission to run containers

Step 2: Acquire the Overlay Companion MCP setup

• Clone or download the repository for overlay-companion-mcp

Step 3: Install via compose-based approach (recommended)

• From the project directory, copy the compose and environment templates and customize ports

# 1) Copy the compose file and .env template
cp infra/kasmvnc-compose.yml ./docker-compose.yml
cp infra/.env.example ./.env

# 2) Edit .env with your port preferences and settings
#   CONTAINER_PORT=8080
#   WEB_PORT=8082
#   MCP_PORT=3001
#   KASMVNC_PORT=6080
#   KASMVNC_ADMIN_PORT=3000

# 3) Start the stack (use Podman or Docker as appropriate)
podman-compose up -d
# or
# docker compose up -d

Optional: One-line installer (scripted)

curl -fsSL https://raw.githubusercontent.com/RyansOpenSourceRice/overlay-companion-mcp/main/host-setup-kasmvnc.sh | bash

What this installs:

• Web interface for managing connections (accessible at http://localhost:8080)
• MCP server for AI integration (connects to Cherry AI, Claude, etc.)
• KasmVNC client (for connecting to remote desktops)
• All running in containers (no mess on your system)

Default port: 8080 (script will auto-detect conflicts and offer alternatives)

Step 4: Optional advanced port customization

# Download and run with custom port
wget https://raw.githubusercontent.com/RyansOpenSourceRice/overlay-companion-mcp/main/host-setup.sh
chmod +x host-setup.sh
./host-setup.sh 8081

# Or set environment variable
OVERLAY_COMPANION_PORT=8081 ./host-setup.sh

Notes and tips:

• The MCP server commonly runs inside the same container set as the overlay-companion-mcp image; the web UI is separate (overlay-companion-web) and the KasmVNC remote desktop components run as their own containers.
• The MCP health endpoint is typically available at http://localhost:3000/health; use this to verify the MCP server is running even if the web UI is up on port 8080.
• If you change ports, ensure the AI integration settings point to the MCP URL (e.g., http://localhost:3000) and not the web UI URL (e.g., http://localhost:8080).
• For the AI to control a VM or remote desktop, you must also install the target-side setup (KasmVNC server) on the VM/host you want to control.
• Common issues usually involve container startup order, port conflicts, or missing environment variables in the .env file (MCP_PORT, CONTAINER_PORT, KASMVNC ports).