verifying-by-consensus
Scannednpx machina-cli add skill tobyhede/turboshovel/verifying-by-consensus --openclawVerifying by Consensus
Overview
Dispatch N agents to independently review the same subject. Collate findings:
- Common (N/N): All agents found → act immediately
- Exclusive (<N/N): Some agents found → cross-check validates
Agent Count Heuristics
| Scope | Default N | Rationale |
|---|---|---|
| Single file change | 2 | Focused review, two perspectives sufficient |
| Multi-file feature | 2-3 | More surface area benefits from diversity |
| Architecture change | 3 | Different perspectives valuable |
| Security-sensitive | 3+ | Higher stakes warrant more eyes |
Override via args: --count 3 or --agents "Explore,Plan,code-agent"
Agent Selection
- Explicit args: If user provides
--agents, use those - Available plugins: Check for specialized agents (cipherpowers code-review-agent, etc.)
- Built-in agents: Use Claude's Explore, Plan agents with review prompts
- Fallback: N instances of same agent with different perspective prompts
Process
Announce: "I'm using the verifying-by-consensus skill to verify [subject]."
Phase 1: Dispatch
- Determine N (default 2, or from args)
- Select agents (from args, plugins, or built-ins)
- Start workflow:
tsv run runbooks/verify.runbook.md - Dispatch agents with StepId prefix in description:
Step(description="1.1 - Review [subject]", prompt="...", subagent_type="...") Step(description="1.2 - Review [subject]", prompt="...", subagent_type="...")
Hooks automate step binding:
| Manual command | Hook trigger | When |
|---|---|---|
tsv run --step 1.1 | PostToolUse (Step) | StepId detected in description |
tsv run --agent {id} | SubagentStart | Agent spawns |
Subagent protocol:
- Write findings to
.work/{date}-verify-{agentId}.md - End response with
STATUS: PASSorSTATUS: FAIL
Phase 2: Collate
After all agents complete, dispatch collation:
- Read all N review files
- Compare findings across agents
- Categorize by consensus:
- Common (N/N): All agents found this issue
- Exclusive: Subcategorize by ratio (e.g., 2/3, 1/3)
- Write collation to
.work/{date}-verify-collated.md
Present immediately:
Collation complete.
## Common (N/N)
[Issues all agents found - can implement now]
## Exclusive
### (N-1)/N
[Issues most agents found]
### 1/N
[Issues one agent found]
Cross-check starting for exclusive findings...
Phase 3: Cross-Check
Dispatch cross-check agent to validate ALL exclusive findings:
- For each exclusive issue, verify against ground truth
- Mark as: VALIDATED | INVALIDATED | UNCERTAIN
- Write to
.work/{date}-verify-crosscheck.md
Present when complete:
Cross-check complete.
VALIDATED: X issues (should address)
INVALIDATED: X issues (can skip)
UNCERTAIN: X issues (user decides)
Phase 4: Complete
tsv complete
Output Files
All files saved to .work/:
{date}-verify-{agentId}.md- Individual reviews{date}-verify-collated.md- Collation report{date}-verify-crosscheck.md- Cross-check results
Templates
Review template: ${CLAUDE_PLUGIN_ROOT}templates/verify-review.md
Collation template: ${CLAUDE_PLUGIN_ROOT}templates/verify-collation.md
Source
git clone https://github.com/tobyhede/turboshovel/blob/main/plugin/skills/verifying-by-consensus/SKILL.mdView on GitHub Overview
Verifying by Consensus dispatches N independent review agents to assess the same subject. Findings are collated by consensus ratio, distinguishing common results from exclusive findings and then cross-checked for validation. This structured workflow helps reduce risk on high-stakes changes by providing robust, verifiable vetting.
How This Skill Works
The skill determines N based on scope (e.g., 2 for single-file changes, 3+ for architecture or security-sensitive tasks). Each agent writes its findings to .work/{date}-verify-{agentId}.md; the system then collates results into a .work/{date}-verify-collated.md and flags exclusive findings for cross-check, which are validated against ground truth before completion.
When to Use It
- Reviewing a single-file change to ensure multiple perspectives agree
- Assessing security-sensitive changes that require higher scrutiny
- Architectural refactors where different viewpoints are valuable
- Multi-file feature work to cover broader surface areas
- Situations where you need explicit cross-check validation of exclusive findings
Quick Start
- Step 1: Determine N from the scope (default 2) or override with --count / --agents
- Step 2: Dispatch agents and run the verify workflow (tsv run runbooks/verify.runbook.md) with StepId prefixes
- Step 3: After agents finish, read .work outputs, generate the collation, run cross-checks, and complete the process
Best Practices
- Choose N according to scope: 2 for simple changes, 3+ for complex or high-stakes work
- Leverage explicit agent prompts and diverse perspectives (built-ins or plugins) to maximize coverage
- Annotate and store each agent’s findings in .work with clear StepId prefixes
- Prioritize common (N/N) findings for rapid action, then rigorously cross-check exclusive items
- Document cross-check results as VALIDATED, INVALIDATED, or UNCERTAIN and act accordingly
Example Use Cases
- Three agents review a security patch; common findings allow immediate deployment while exclusive findings are cross-checked for validation.
- A multi-file feature goes through 2-3 agents to ensure broader surface area coverage, followed by collating results.
- An architecture refactor is evaluated by three agents to surface compatibility concerns and potential regressions.
- A performance optimization is independently reviewed by multiple agents, with exclusive findings validated before merge.
- A critical bug fix is verified by consensus, then cross-checked to confirm ground-truth alignment and release readiness.
