ai4j

ai4j is a Java SDK that unifies access to multiple large language model platforms (OpenAI, Zhipu, Ollama, DeepSeek, Moonshot, Hunyuan, Lingyi, Baichuan, etc.) through a consistent input/output interface and operational features such as function calling and RAG with vector stores. It supports MCP (Model Context Protocol) via STDIO, SSE, and Streamable HTTP, enabling you to run this server alongside MCP clients and gateways. The server can be embedded in Spring Boot or plain Java applications, and it provides a single set of services (chat, embedding, audio, realtime) across multiple providers, making integration easier and more maintainable. You can leverage built-in support for tool calls, streaming outputs, and dynamic MCP data sources, along with an internal gateway for MCP communications.

Prerequisites:

• JDK 8 or newer (ai4j explicitly supports Java 8 compatibility)
• Maven or Gradle for building Java projects
• Optional: a running PostgreSQL/MySQL or a vector store like Pinecone if you plan to use RAG features

1. Install the library in your project

• Maven:

<dependency>
  <groupId>io.github.lnyo-cly</groupId>
  <artifactId>ai4j</artifactId>
  <version>${project.version}</version>
</dependency>

• Maven (Spring boot starter):

<dependency>
  <groupId>io.github.lnyo-cly</groupId>
  <artifactId>ai4j-spring-boot-starter</artifactId>
  <version>${project.version}</version>
</dependency>

• Gradle:

implementation 'io.github.lnyo-cly:ai4j:${project.version}'

• Gradle (Spring boot starter):

implementation 'io.github.lnyo-cly:ai4j-spring-boot-starter:${project.version}'

2. Configure application (example for Spring Boot)

• application.yml:

ai:
  openai:
    api-key: "YOUR_OPENAI_API_KEY"
  zhipu:
    api-key: "YOUR_ZHIPU_API_KEY"
  okhttp:
    proxy-port: 10809
    proxy-url: "127.0.0.1"

3. Run the server

• If using a standalone Java app:

java -jar path/to/ai4j-server.jar

• If embedding in Spring Boot:

mvn spring-boot:run

4. Interact with MCP clients

• Use MCP-compliant clients to connect via STDIO, SSE, or Streamable HTTP to the running server and access the unified AI services (Chat, Embedding, Audio, Realtime).
• Ensure environment variables for API keys and any gateway or proxy settings are configured as needed.

• ai4j supports multiple backends (OpenAI, Zhipu, Ollama, DeepSeek, Moonshot, Hunyuan, Lingyi, Baichuan, etc.).
• Ensure you provide appropriate API keys or access tokens for each provider you enable.
• If you enable vector stores for RAG (e.g., Pinecone), configure the endpoint and credentials via environment variables or application properties.
• The MCP gateway and dynamic MCP data sources are built-in; you can customize dispatchers and connect pools via SPI.
• For streaming outputs, enable and handle SseListener or equivalent streaming callbacks provided by the client.
• Java 8 compatibility is supported, but some platforms in the ecosystem may have newer requirements; consider using a compatible JDK for production.
• Common issues: network/proxy misconfigurations, API key scope, and mismatched model names across platforms. Check logs for streaming token usage when using stream_options.