chatlab

Chatlab provides a Gradio-based UI that can operate with Ollama locally or via the Together.ai API to perform LLM inference. The setup allows you to run a local Ollama model (e.g., llama3.2:3b) or connect to Together to access larger models through an API key. The application workflow is managed via LLama-Stack for environment management, and Gradio serves as the frontend interface for interacting with the model. Once running, you can input prompts, customize model selection via environment variables, and choose between local Ollama inference or API-based inference depending on your configuration. The guidance in the repository focuses on ensuring the inference environment is properly prepared and that necessary API keys are supplied if you opt for Together-based inference. The interface will be accessible in your browser after starting the Gradio server with the provided command.

Prerequisites:

• A compatible OS with Python and shell access
• Ollama (optional for local inference)
• uv package manager
• Git

Installation steps:

1. Install Ollama (optional for local inference):

   • Visit https://ollama.com/ and follow the installation instructions for your OS.
   • Start the service: ollama serve
   • Pull a model (example): ollama pull llama3.2:3b

2. Install LLama-Stack tooling (uv):

   • Install the uv package manager according to its documentation.
   • Create and activate a virtual environment (venv) as needed.

3. Project setup:

   • Clone the repository: git clone https://github.com/ricardoborges/chatlab.git cd chatlab
   • Create/activate a Python virtual environment and install dependencies: uv venv uv pip install -r myproject.toml

4. Run the application (Gradio):

   • If you use Together.ai, obtain an API key from Together and place it in your environment or .env file as TOGETHER_API_KEY.
   • Create a .env file with required keys, for example: TAVILY_SEARCH_API_KEY= TOGETHER_API_KEY=
   • Start the Gradio interface: gradio main.py
   • Open the URL provided by Gradio in your web browser.

Prerequisites recap:

• Git
• Python environment with uv available
• Ollama (optional for local inference) and model(s) if using Ollama
• Together API key if using Together.ai

Tips and notes:

• If you prefer local inference with Ollama, ensure the Ollama service is running and the model is pulled before starting the Gradio interface.
• If using Together.ai, you must provide a valid TOGETHER_API_KEY; otherwise, you can set DEFAULT_STACK to Ollama to run locally.
• TM values in the .env file are placeholders; replace them with your actual keys or leave them empty if not using those services.
• The chatlab workflow relies on LLama-Stack for environment management; ensure you have uv installed and accessible in your shell.
• Troubleshooting: verify that the Ollama service is up, the virtual environment is activated, and dependencies are installed correctly. If Gradio fails to start, check that the port is not in use and that Python can locate main.py.