marmot

Marmot is an open-source data catalog that lets you discover, organize, and collaborate around data assets across your organization. It provides a fast full-text search experience, metadata-rich asset descriptions, and interactive lineage visualizations. The system exposes a REST API and supports tooling integrations via CLI, Terraform, and Pulumi, making it easy to catalog a wide range of assets—from tables and topics to APIs, dashboards, and pipelines. Use Marmot to search for assets, inspect their metadata, trace data lineage, assign owners, and document business context all in one place.

To get started, deploy Marmot using the available binaries or Docker image. Once running, you can use the REST API or the built-in UI to index assets, run searches with metadata filters, and explore lineage graphs. The CLI and infrastructure-as-code integrations allow you to automate asset registration and governance across your data stack. Ensure your PostgreSQL backend is reachable (Marmot is PostgreSQL-backed for metadata storage) and configure the appropriate API port for access.

Key capabilities include: full-text search with boolean and metadata filters, interactive lineage graphs showing upstream and downstream dependencies, a metadata-first approach for asset details, and collaboration features like ownership assignment and business context notes. You can also export/import metadata and integrate Marmot with Terraform or Pulumi to manage catalog state alongside your data infrastructure.

Prerequisites

• Docker or a compatible container runtime
• A PostgreSQL database for Marmot metadata storage (or use a managed PostgreSQL instance)
• Optional: a domain or load balancer for exposing the Marmot API/UI

Install with Docker

1. Pull and run Marmot (with a PostgreSQL connection string)

# Example: run Marmot with Docker (adjust DATABASE_URL to your environment)
docker run -d --name marmot \
  -p 8080:8080 \
  -e DATABASE_URL=postgres://USER:PASS@HOST:PORT/DBNAME \
  marmotdata/marmot:latest

2. Verify container is healthy and the API is reachable at http://<host>:8080

3. If you prefer a local binary, download the Marmot binary from the releases page, place it in your PATH, and run (example):

# Assuming a binary named marmot
marmot --database-url=postgres://USER:PASS@HOST:PORT/DBNAME --port=8080

4. Configure persistence and networking as needed (e.g., reverse proxy, TLS termination, firewall rules).

Install with Kubernetes (optional)

• Use the Marmot Kubernetes manifests or Helm chart provided in the project to deploy Marmot and its PostgreSQL backing store. Ensure proper config maps/ secrets for DATABASE_URL and ports, then apply with kubectl or helm install.

Tips and common considerations:

• Ensure your PostgreSQL database is sized for Marmot metadata workload; monitor DB performance as the catalog grows.
• Expose Marmot behind a reverse proxy (e.g., Nginx, Traefik) for TLS termination and stable DNS.
• The DATABASE_URL must be accessible from the Marmot container; consider network policies in Kubernetes or Docker network configuration.
• When upgrading Marmot, back up the metadata database and review any breaking changes in the release notes.
• Use the Terraform or Pulumi integrations to register assets programmatically and keep catalog state in sync with your cloud/data infra.
• For large catalogs, tune Marmot’s search indexes and consider enabling pagination on API queries to avoid large payloads.
• Integrations: Marmot supports CLI, REST API, and IaC tools—use them to automate asset ingestion and governance workflows.