MAI-UI is a GUI agent foundation model family designed to operate across devices and the cloud, enabling an agent to interact with a user and leverage MCP tools to accomplish tasks. The server exposes a set of capabilities that let you issue natural language instructions, which MAI-UI interprets to perform GUI-grounded reasoning, navigation, and cross-application actions via MCP tooling. Typical usage involves invoking the MCP call surface (for example mcp_call) to access tools or plugins (such as map navigation, shopping, calendar, or other app integrations) and then guiding the agent through a task sequence with human-in-the-loop prompts when needed. You can observe demos where the agent asks clarifying questions, plans multi-step workflows, and delegates subtasks to appropriate tools, coordinating device-local computation with cloud-backed models when data or capabilities exceed device limits.

Prerequisites:

• Python 3.8+ installed on your system
• Access to the internet to fetch dependencies
• Optional: a compatible GPU for accelerated inference

1. Create and activate a virtual environment (recommended): python -m venv venv source venv/bin/activate # macOS/Linux venv\Scripts\activate # Windows

2. Install dependencies (adjust if a requirements file exists in the repo): pip install -r requirements.txt

   or install directly if no requirements file:

   pip install numpy torch transformers pillow

3. Install MAI-UI package (if published to PyPI or from source):

   From PyPI

   pip install mai-ui

   From source (if applicable)

   git clone https://github.com/tongyi-mai/MAI-UI.git cd MAI-UI pip install -e .

4. Configure the server:

   • Create a config file (config.yaml or config.json) if required by the MAI-UI server.
   • Set environment variables as needed (see mcp_config env section).

5. Run the MCP server (example): python -m mai_ui.server

6. Verify the server is running by sending a test MCP request to the endpoint configured by the server (often http://localhost:8000 or a defined port).

Tips and common issues:

• Ensure Python dependencies compatible with your CUDA/cuDNN setup if you use GPU acceleration.
• If the server fails to start due to missing models, verify the paths in MAI_UI_CONFIG and MAI_UI_MODEL_PATH environment variables.
• For MCP tooling, you may need to expose network access to MCP tool endpoints (e.g., map services, calendar, or shopping APIs) and provide necessary API keys or tokens.
• Monitor logs for prompts where the agent requests clarification; providing precise task instructions improves task success rates.
• If you encounter permission errors, run the server with appropriate user permissions or adjust file permissions on model weights and config files.