model-registry-governance
npx machina-cli add skill BagelHole/DevOps-Security-Agent-Skills/model-registry-governance --openclawFiles (1)
SKILL.md
2.4 KB
Model Registry Governance
Create a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence.
Core Principles
- Traceability: every production model maps to source code, data snapshot, and evaluation results.
- Reproducibility: builds are deterministic with pinned dependencies.
- Policy-driven promotion: no manual bypass for critical safety checks.
- Lifecycle hygiene: stale, vulnerable, or unowned models are retired automatically.
Required Metadata Schema
Track at minimum:
- Model name, semantic version, checksum, and storage URI
- Base model lineage and fine-tune method
- Training/eval datasets and time windows
- License, allowed use cases, prohibited use cases
- Security risk rating and mitigation controls
- Owner, backup owner, and escalation contact
Approval Workflow
- Registration request created from CI.
- Security checks (artifact scan, dependency scan, provenance).
- Evaluation package uploaded (quality, toxicity, jailbreak, bias, latency, cost).
- Required approvals: platform + product + security (as policy dictates).
- Promotion to stage/prod based on signed decision record.
Lifecycle States
draft: internal experimentation.candidate: passed baseline tests.approved: authorized for production rollout.deprecated: replacement announced, new usage blocked.retired: no serving allowed, archived for audit.
Governance Policies
- Reject artifacts without SBOM/provenance.
- Block promotion if known critical CVEs remain unresolved.
- Require refreshed evals after prompt/template changes.
- Expire approvals after a configurable period (for example 90 days).
Audit Readiness
Maintain immutable records of:
- Who approved and when
- Which policy checks executed
- Which exceptions were granted
- What model/version served each customer request window
Related Skills
- sbom-supply-chain - Provenance and signing
- policy-as-code - Enforce governance with policy engines
- llm-fine-tuning - Version adapters and training outputs
Source
git clone https://github.com/BagelHole/DevOps-Security-Agent-Skills/blob/main/devops/ai/model-registry-governance/SKILL.mdView on GitHub Overview
Establishes a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence. It enforces traceability from production models to source code, data snapshots, and evaluation results, and supports reproducible builds with pinned dependencies. It also enforces policy-driven promotion and automatic lifecycle hygiene to retire stale or insecure models.
How This Skill Works
CI creates registration requests and attaches required metadata. Security checks scan provenance, artifacts, and dependencies, while an evaluation package covers quality, toxicity, bias, latency, and cost. Promotion to stage or prod depends on policy-driven approvals and a signed decision record.
When to Use It
- When you need a trustworthy system of record for model artifacts, prompts, adapters, and evaluation evidence.
- When you require policy-driven promotion with multi-party approvals rather than manual bypass.
- When enforcing SBOM/provenance, vulnerability checks, and controlled promotion before production.
- When managing lifecycle states and automatic retirement for stale, unowned, or risky models.
- When preparing for audits or regulatory reviews that demand immutable approval logs and provenance data.
Quick Start
- Step 1: Define required metadata schema and lifecycle states (model name, semantic version, checksum, storage URI, base lineage, datasets and time windows, license, use cases, risk rating, owner contacts).
- Step 2: Wire CI/CD to create registration requests and run security/provenance checks, then upload evaluation packages.
- Step 3: Configure policy-driven approvals and a signed decision record to promote artifacts to stage and prod, with immutable audit logging.
Best Practices
- Enforce SBOM and provenance for every artifact and ensure traceability.
- Keep reproducible builds with pinned dependencies and base model lineage.
- Implement policy-as-code driven approvals with formal decision records.
- Automate lifecycle transitions (draft -> candidate -> approved -> deprecated -> retired).
- Maintain immutable audit trails of approvals, checks, and exceptions.
Example Use Cases
- Fintech enterprise standardizes a model registry with policy-driven promotions and audit-ready records.
- Healthcare AI platform stores provenance, licensing, risk ratings, and evaluation evidence to meet regulatory requirements.
- Cloud AI service enforces SBOM/provenance checks before promoting models across stages via policy engines.
- E-commerce recommendations lifecycle rotates models and blocks deprecated versions to limit risk.
- Research-to-prod pipelines emit signed decision records guiding production deployments and audits.
Frequently Asked Questions
Add this skill to your agents
