This Kafka MCP Server exposes tools to publish and consume messages to and from Apache Kafka topics through a standardized MCP interface. It acts as a bridge between LLM/Agentic workflows and Kafka, allowing you to publish data to a topic, or consume messages for processing and response generation. The server is designed to be run via Python and can be integrated with clients that support MCP tooling in your environment. You can also create, delete, and manage topics directly from the integrated command set, enabling end-to-end interaction with your Kafka cluster from within your AI workflows.

Prerequisites:

• Python 3.8+ installed on your system
• Apache Kafka cluster accessible from the host running the server
• Access to install Python dependencies (pip)

Installation steps:

1. Clone the repository: git clone <repository-url> cd <repository-directory>

2. Create and activate a virtual environment: python -m venv venv source venv/bin/activate # On Windows, use: venv\Scripts\activate

3. Install dependencies: pip install -r requirements.txt

   If a requirements file is not present, install core packages manually:

   pip install aiokafka python-dotenv pydantic-settings mcp-server

4. Prepare configuration:

   • Create a .env file in the project root with Kafka settings (see README for examples).

5. Run the server: python main.py --transport stdio

Note: If you cannot use a requirements.txt, ensure the following packages are installed: aiokafka, python-dotenv, pydantic-settings, mcp-server.

Environment variables:

• KAFKA_BOOTSTRAP_SERVERS: set to your Kafka bootstrap server(s)
• TOPIC_NAME: topic to publish/consume
• IS_TOPIC_READ_FROM_BEGINNING: set to True or False to control consumption starting offset
• DEFAULT_GROUP_ID_FOR_CONSUMER: consumer group id used when consuming

Common issues:

• Ensure Kafka is reachable from the host running the MCP server (network ACLs, firewall, Docker networking if applicable).
• If consuming shows no messages, verify TOPIC_NAME and IS_TOPIC_READ_FROM_BEGINNING settings and that the group ID is not hard-stuck in a stale consumer group.
• When creating topics via Create-Topic, confirm the broker capabilities and permissions for topic creation.

Tip: The server exposes tools for publishing and consuming (kafka-publish, kafka-consume), as well as topic management (Create-Topic, Delete-Topic, List-Topics, Topic-Configuration, Topic-Metadata). These can be invoked through the MCP interface or Claude Desktop integration as shown in the README.