What is the required report schema?

The tool produces a report.json following schema_version 1.1.0 with fields like metadata, executive_summary, summary_statistics, and findings per the skill document.

How are findings ordered in the report?

Unresolved findings are listed first, ordered by severity (CRITICAL to LOW), followed by RESOLVED findings.

Which tooling is assumed?

GitHub CLI (gh pr), Python scripts (validate_report.py, generate_review_report.py), and standard utilities described in the skill documentation.

check-pr-comments

Scanned

npx machina-cli add skill lklimek/claudius/check-pr-comments --openclaw

Files (1)

SKILL.md

4.2 KB

Check PR Comments Workflow

When asked to check/triage/verify existing PR review comments, follow this workflow.

1. Fetch All Comments

Fetch inline review comments, PR-level comments, and review summaries. See the github skill (PR Review Comments section) for the wrapper scripts.

2. Checkout and Pull the PR Branch

gh pr checkout <number>
git pull

3. Verify Each Comment Against Current Code

For every inline comment, read the file at the referenced location and verify whether the identified issue is actually fixed -- not just whether the code changed. Specifically:

Read the current code at the location the comment references
Understand what the comment is asking for
Determine if the current code satisfies the request (semantically, not just syntactically)
For comments with multiple sub-items, verify each one independently
A comment is only "resolved" if all of its sub-items are addressed

4. Build Structured Report JSON

Produce a report.json file following the unified report schema (schemas/review-report.schema.json v1.1.0).

Report structure

{
  "schema_version": "1.1.0",
  "metadata": {
    "project": "<owner>/<repo>",
    "date": "YYYY-MM-DD",
    "branch": "<pr-branch>",
    "commit": "<HEAD short SHA>",
    "scope": "PR #<number> comment verification",
    "reviewers": ["<unique reviewer usernames>"],
    "report_type": "comment_check",
    "pr_number": <number>
  },
  "executive_summary": {
    "overall_assessment": "X of Y review comments resolved",
    "verdict_action": "N comments require attention"
  },
  "summary_statistics": {
    "total_findings": <total>,
    "severity_counts": { "CRITICAL": 0, "HIGH": 0, "MEDIUM": 0, "LOW": 0, "INFO": 0 },
    "verdict_counts": { "RESOLVED": <n>, "UNRESOLVED": <n> }
  },
  "findings": [
    {
      "title": "PR Comment Verification",
      "category": "pr_comments",
      "findings": [ ... ]
    }
  ]
}

Finding format

Each review comment becomes one finding:

{
  "id": "CMT-001",
  "severity": "INFO for RESOLVED, assessed severity for UNRESOLVED",
  "title": "Short description of what the comment requests",
  "location": "path/to/file.rs:42-56",
  "description": "What the comment asked for (multi-line OK)",
  "recommendation": "What was done (RESOLVED) or what to do (UNRESOLVED)",
  "reviewer": "github-username",
  "comment_id": 12345678,
  "comment_url": "https://github.com/<owner>/<repo>/pull/<number>/files#r<commentId>",
  "thread_id": "GraphQL-node-ID-for-thread-resolution",
  "verdict": "RESOLVED or UNRESOLVED"
}

Resolved comments: severity: "INFO", verdict: "RESOLVED". recommendation describes what was done.
Unresolved comments: assessed severity (CRITICAL > HIGH > MEDIUM > LOW), verdict: "UNRESOLVED". recommendation describes what still needs to be done.
Severity levels: see severity skill.
thread_id: from gh-list-review-threads.sh output. Needed for thread resolution in step 7.

Numbering

Assign sequential IDs: CMT-001, CMT-002, etc. Order: unresolved first (by severity descending), then resolved.

5. Validate Report

python3 scripts/validate_report.py report.json

If validation fails, fix the JSON and re-validate. Do NOT proceed with invalid data.

6. Render and Present

python3 scripts/generate_review_report.py report.json --format md

Present the rendered markdown report to the user. Optionally generate HTML (--format html) for richer display.

The user can also invoke triage-findings report.json for interactive browser-based triage of unresolved comments.

7. Resolve Addressed Threads

Always ask the user for confirmation before resolving any threads.

After the report is presented and the user approves, resolve addressed review threads. See the github skill (PR Review Comments > Resolving review threads section) for the wrapper scripts.

Only resolve threads where verification confirms the issue is fixed. Never resolve threads that are only partially addressed.

Source

git clone https://github.com/lklimek/claudius/blob/main/skills/check-pr-comments/SKILL.mdView on GitHub

Overview

This skill automates validating that PR review comments are actually addressed in code by inspecting the referenced locations and their sub-items. It then outputs a triage-compatible report (report.json) that mirrors the unified schema used for PR reviews.

How This Skill Works

Workflow: fetch all inline, PR-level, and review comments, checkout the PR branch, and for each comment read the target code to verify the requested changes are semantically satisfied. For multi-item comments, every sub-item is checked and the overall comment is marked RESOLVED only if all sub-items are addressed. Finally, it builds report.json following schema 1.1.0 and can validate and render the results.

When to Use It

When you need to confirm that every reviewer comment was properly addressed before merging
When a comment has multiple sub-items and granular verification is required
When producing a triage-ready report that summarizes PR comment statuses
When you want to standardize PR-verification workflows with the unified schema
When triaging outdated or conflicting feedback in a PR review

Quick Start

Step 1: Fetch PR comments and check out the PR branch (e.g., gh pr checkout <number>)
Step 2: For each comment, read the referenced file and verify each sub-item is satisfied semantically
Step 3: Generate report.json, run python3 scripts/validate_report.py report.json and render with python3 scripts/generate_review_report.py report.json --format md

Best Practices

Fetch all comment types (inline, PR-level, and summaries) before starting verification
Verify every sub-item within multi-item comments independently
Assess semantic fixes, not just whether code changed
Follow the report schema exactly and emit report.json
Validate the generated report with the provided Python scripts before rendering

Example Use Cases

A reviewer asks to handle a null-check across a module and a test; verify both and mark RESOLVED or UNRESOLVED
A UI-related comment requests DOM updates; confirm the rendered component aligns with the fix
A robustness issue is described; ensure the error handling paths meet the intent
A performance concern is raised; validate that the change preserves behavior and efficiency
An outdated comment is detected; triage and decide next steps

Frequently Asked Questions

Add this skill to your agents