AI Gateway acts as an enterprise-grade gateway that manages AI models, tools, and agents through the MCP protocol. It enables secure access control, routing, and observability for MCP-based tools and function calls, allowing you to plug in tools and orchestrate agent workflows with policy-driven gateways. The MCP endpoints exposed by this server let clients discover and invoke tools or model backends, while supporting OAuth credential management and secure token exchange. To use it, deploy the MCP server in your environment, configure the necessary credentials and policies, and then register tools or agents that speak the MCP protocol. You can then route requests to different model backends or tools based on policies, versions, or routing rules, and monitor usage through built-in logging and token metrics.

Prerequisites:

• Python 3.12+ installed on the host system
• Git installed to clone the repository
• Virtual environment support (recommended) or use system Python

Steps:

1. Clone the repository git clone https://github.com/Azure-Samples/AI-Gateway.git cd AI-Gateway

2. Create and activate a virtual environment (optional but recommended) python -m venv venv source venv/bin/activate # macOS/Linux venv\Scripts\activate # Windows

3. Install dependencies pip install -r requirements.txt

4. Configure environment variables (see mcp_config in this document for placeholders)

   Example (bash)

   export AZURE_SUBSCRIPTION=sub_id export AZURE_TENANT_ID=tenant_id export AZURE_CLIENT_ID=client_id export AZURE_CLIENT_SECRET=client_secret

5. Run the MCP server python -m ai_gateway.mcp_server

6. Optional: run via a container (If a Docker image is provided by the project, you can adapt the following) docker run -it --env-file .env ai-gateway-image:latest

Prerequisites note: Ensure network egress and required Azure permissions are configured for the MCP server to function correctly.

Tips and common considerations:

• If you plan to use OAuth or other credential flows, ensure the corresponding Azure AD app registrations and permissions are properly configured.
• Check logs and token metrics to troubleshoot MCP routing issues or authentication problems.
• When adding tools or agents via MCP, ensure they expose standard MCP endpoints and honor the security policies (scopes, credentials, and rate limits).
• If you update dependencies, re-run the installation steps to refresh Python packages.
• For production deployments, consider using a container orchestration platform and enable observability via built-in logging and tracing features.