Label Application Safety Guarantees

Date: 2026-01-15
Status: ✅ Active
Related: .github/workflows/issue-triage.yml, .github/workflows/config/triage-categories.json

Overview

The AI issue triage workflow (issue-triage.yml) applies labels to GitHub issues using a non-destructive append-only approach. This document guarantees that no existing labels will be overwritten or removed.

Safety Guarantee

CRITICAL: The workflow uses gh issue edit --add-label which has these properties:

1. Append-only: Only ADDS labels, never removes existing ones
2. Idempotent: If a label already exists, command returns success without duplicating
3. Non-destructive: No delete or overwrite logic exists in the workflow
4. Reversible: Labels can be manually removed by humans if needed

Implementation

Label Application Logic

The workflow applies labels in this step (lines 183-205 in issue-triage.yml):

# Convert comma-separated labels to array format for gh cli
IFS=',' read -ra LABEL_ARRAY <<< "$LABELS"

# Add each label with error handling
FAILED_LABELS=""
for label in "${LABEL_ARRAY[@]}"; do
  label=$(echo "$label" | xargs) # trim whitespace
  echo "Adding label: $label"
  
  # SAFE: --add-label only adds, never removes
  if gh issue edit "$ISSUE_NUMBER" --add-label "$label" 2>/dev/null; then
    echo "✅ Applied: $label"
  else
    echo "⚠️ Failed to apply: $label"
    FAILED_LABELS="$FAILED_LABELS,$label"
  fi
done

Key Safety Properties

Property	Guarantee	Evidence
No Deletions	Never removes labels	Uses --add-label only, no --remove-label calls
No Overwrites	Never replaces labels	No --set-label or destructive operations
Idempotency	Safe to run multiple times	GitHub CLI handles duplicate label adds gracefully
Error Handling	Graceful fallback on failure	Logs failures but doesn't roll back existing labels
Existing Data	Preserves manually-added labels	Append-only model means human labels stay

Testing

Test Coverage

The workflow safety is verified by:

1. Unit Tests (src/__tests__/workflows/ai-issue-triage.test.js):

   • Tests label parsing from AI response
   • Tests invalid label rejection
   • Tests format validation

2. Mock Tests (src/__tests__/workflows/ai-issue-triage-mock.test.js):

   • Simulates label application with realistic responses
   • Verifies label arrays are properly formatted

3. Real API Tests (src/__tests__/workflows/ai-issue-triage-real.test.js):

   • Applies labels to actual test issues
   • Verifies end-to-end workflow safety

Test Status: 44/44 tests passing ✅

Manual Testing

To verify safety locally, run:

# Test with dry-run (no actual label application)
./.github/workflows/test-ai-triage.sh --dry-run

# Test with mock API (applies labels to test issue)
./.github/workflows/test-ai-triage.sh --mock

# Test with real API (applies labels to actual issue)
./.github/workflows/test-ai-triage.sh --real

Failure Modes

Scenario 1: Label Already Exists

What happens: GitHub CLI returns success, no duplicate created
User impact: Label appears once (correct)
Workflow status: Success ✅

Scenario 2: Label Doesn't Exist in Project

What happens: Label creation fails (workflow policy), error logged
User impact: Issue triaged, label not applied
Workflow status: Success with warning ⚠️
Note: Issue triage suggestion comment still posted with explanation

Scenario 3: API Authentication Fails

What happens: Label application skipped, error logged
User impact: Issue triaged, labels not applied
Workflow status: Graceful degradation ⚠️
Note: Triage suggestion comment still provided

Scenario 4: GitHub Service Outage

What happens: Label application fails with timeout
User impact: Issue triaged, labels not applied
Workflow status: Graceful degradation ⚠️
Note: Triage suggestion still helpful to human reviewers

Data Integrity

What Cannot Be Lost

• ✅ Existing labels added by humans
• ✅ Existing labels added by other workflows
• ✅ Issue title, description, assignees
• ✅ Any metadata from issue creation

What Can Change

• ✅ New labels added by this workflow (append-only)
• ✅ Category field (if created in GitHub Projects V2)
• ✅ Triage comment (posted, not modifying existing comments)

Validation

Pre-Deployment Validation

The workflow includes validation to prevent invalid labels:

// From issue-triage.yml lines 60-85
const VALID_LABELS = [
  'bug', 'documentation', 'duplicate', 'enhancement',
  'good first issue', 'help wanted', 'invalid',
  'question', 'wontfix'
];

// Only labels in VALID_LABELS are applied
const filtered = labels.filter(l => VALID_LABELS.includes(l));

Configuration Validation

Separate validation workflow (validate-triage-config.yml) ensures:

• Category JSON is valid
• Categories match GitHub Projects V2 (if field exists)
• Configuration changes are reviewed before merge

See docs/engineering/deployment/VALIDATE_TRIAGE_CONFIG.md for validation details.

Rollback Procedures

If labels need to be corrected:

Manual Removal

1. Go to GitHub issue
2. Click "Label" button in sidebar
3. Uncheck unwanted labels
4. Close issue (labels persist)

Automation (If Needed)

Create workflow to remove labels with --remove-label (does not exist yet, can be added if needed)

Monitoring

Monitor for issues:

1. Workflow Logs (Actions tab → Issue Triage → recent runs)

   • Check for ⚠️ Failed to apply messages
   • Check for API error logs

2. Issue Comments (individual issues)

   • Verify triage suggestion posted
   • Check category field if applicable

3. Configuration Drift (.github/workflows/config/)

   • Run validation script: ./.github/workflows/validate-triage-config.sh
   • Check GitHub Actions validation workflow status

Audit Trail

Every label application is logged:

1. GitHub Actions Logs (action runner output)

   • Timestamp of label application
   • Success/failure status
   • Which labels were applied

2. Issue Timeline (GitHub issue history)

   • Visible in "Show more" on older events
   • "labels added by: GitHub" on timeline

3. Audit Logs (if GitHub Enterprise)

   • Full audit trail of label changes
   • User/automation attribution

Related Documentation

• Triage Category System - High-level system overview
• Configuration Validation - Config validation process
• AI Issue Triage Testing - Testing procedures
• GitHub Copilot Instructions - Workflow guidelines

See Also

• GitHub CLI Documentation - gh issue edit reference
• GitHub Issues API - Label API details
• Triage Categories Configuration - Category definitions