What triggers this skill?

Triggers include file test bug, report test failure, create bug for test, integration test failed, test failure issue, and junit failure.

What sources does it use to build the issue?

It parses tests/reports/junit.xml for failures and reads agent-metadata.md from the corresponding test-run directory.

Where is the issue created and what labels are used?

The issue is created in the microsoft/github-copilot-for-azure repo with labels bug and integration-test, using a detailed structured body.

file-test-bug

Scanned

npx machina-cli add skill microsoft/GitHub-Copilot-for-Azure/file-test-bug --openclaw

Files (1)

SKILL.md

2.5 KB

File Test Bug

Creates a GitHub issue in microsoft/github-copilot-for-azure for integration test failures.

Input

Skill name (required): e.g., azure-rbac, appinsights-instrumentation
Test run (optional): Timestamp of test run. Defaults to most recent with logs for the skill.

Steps

Ask user for skill name if not provided
Parse tests/reports/junit.xml for failures matching the skill
Find test run directory (specified or most recent with matching logs)
Read agent-metadata.md from tests/reports/test-run-<timestamp>/<skillname>-<testname>/
For each failure, read the actual line of code from the test file using the location (file:line) from junit.xml
REQUIRED - Write diagnosis BEFORE creating issue:
- Analyze the agent-metadata.md to understand what the agent did
- Compare agent behavior to what the test expected (from the assertion)
- Identify the root cause (skill issue, test issue, or model behavior)
- Write 2-3 sentences per failed test explaining WHY it failed
- Suggest potential fixes (update skill, update test, or update fixtures)
Create issue via github-mcp-server-create_issue:

owner: microsoft
repo: github-copilot-for-azure
title: Integration test failure in <skill-name>
labels: ["bug", "integration-test"]
body: |
  ## Failed Tests
  - <test-name>: <error message>
  
  ## Diagnosis
  
  ### Root Cause
  <1-2 sentences explaining WHY the test failed based on agent-metadata.md analysis>
  
  ### Analysis per Test
  - **<test-name>**: <what agent did vs what test expected>
  
  ### Suggested Fix
  <one of: update skill, update test assertions, provide test fixtures>

  ## Details
  ### <test-name>
  **Error:** <failure from junit.xml>
  **Location:** <file:line>
  ```typescript
  <actual line of code from the test file at the specified line number>

<details> <summary>agent-metadata.md</summary>

</details> ```

Important

Include the complete, unmodified contents of each agent-metadata.md file in the issue body. Do NOT summarize or truncate the logs. Wrap each log in a <details> block with the test name as the summary.

Source

git clone https://github.com/microsoft/GitHub-Copilot-for-Azure/blob/main/.github/skills/file-test-bug/SKILL.mdView on GitHub

Overview

File Test Bug automatically creates a GitHub issue in microsoft/github-copilot-for-azure when integration tests fail locally. It analyzes junit.xml for skill-specific failures and uses agent-metadata.md to root-cause the issue, producing a structured, actionable bug report.

How This Skill Works

The skill prompts for the target skill, scans tests/reports/junit.xml for failures that match the skill, and selects the most relevant test run. It then reads the corresponding agent-metadata.md to understand what happened, writes a diagnostic narrative for each failed test, and files the issue via github-mcp-server-create_issue with a detailed, labeled body.

When to Use It

Local integration test failures for a specific skill
JUnit failures across multiple tests within a single skill
You know which test run to use, or want the most recent run with logs
Agent behavior differs from test expectations and needs documented diagnosis
You need a detailed issue including per-test analysis, code snippets, and agent logs

Quick Start

Step 1: Provide the skill name or confirm it is already provided.
Step 2: Run local integration tests to generate junit.xml and agent-metadata.md.
Step 3: The tool analyzes failures and creates a GitHub issue via github-mcp-server-create_issue with a structured body.

Best Practices

Prompt for the skill name if not provided
Parse junit.xml for failures that match the skill
Choose the correct test run directory either specified or most recent with logs
Read and include agent-metadata.md for each failure to inform the diagnosis
Write 2-3 sentence per-failure diagnosis before creating the GitHub issue, and use the standard issue format with labels bug and integration-test

Example Use Cases

A local test run for a skill fails with an assertion error and the tool creates a bug issue with diagnosis.
A skill has two failing tests; the tool aggregates both into a single issue with per-test analysis.
No explicit test run is provided; the tool uses the most recent run with logs to generate the issue.
Agent behavior diverges from test expectations and the issue includes a per-test diagnostic section.
Test fixtures or expectations are misaligned; the issue suggests updating the skill or test fixtures.

Frequently Asked Questions

Add this skill to your agents