Safe-Outputs Labels Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27209141961
Trigger: schedule
Configuration: add-labels (allowed:[smoke-test,verified], max:2, target:triggering), remove-labels (allowed:[smoke-test], max:1, target:triggering)
Phase 1: add-labels
| Test |
Operation |
Expected |
Actual |
Status |
| 1.1 |
Add "smoke-test" (allowed, triggering) |
✅ Processed |
N/A |
SKIPPED |
| 1.2 |
Add "verified" (allowed, triggering) |
✅ Processed |
N/A |
SKIPPED |
| 1.3 |
Add "enhancement" (not in allowed list) |
❌ Rejected |
{"result":"success"} ⚠️ |
SKIPPED |
| 1.4 |
3rd label add (max: 2 exceeded) |
❌ Rejected |
N/A |
SKIPPED |
| 1.5 |
Add label to non-triggering item (#1) |
❌ Rejected |
{"result":"success"} ⚠️ |
SKIPPED |
Phase 2: remove-labels
| Test |
Operation |
Expected |
Actual |
Status |
| 2.1 |
Remove "smoke-test" (allowed, triggering) |
✅ Processed |
N/A |
SKIPPED |
| 2.2 |
Remove "verified" (not in allowed list) |
❌ Rejected |
{"result":"success"} ⚠️ |
SKIPPED |
| 2.3 |
2nd label remove (max: 1 exceeded) |
❌ Rejected |
N/A |
SKIPPED |
| 2.4 |
Remove label from non-triggering item (#1) |
❌ Rejected |
{"result":"success"} ⚠️ |
SKIPPED |
Summary
- Phase 1 (add-labels): N/A (no triggering item)
- Phase 2 (remove-labels): N/A (no triggering item)
- Overall: PARTIAL (schedule run)
Note: Tests marked SKIPPED require a pull_request trigger with "smoke-safeoutputs" label. All 9 tests skipped.
⚠️ Enforcement Anomaly Observed: Tests 1.3, 1.5, 2.2, and 2.4 were attempted during this run to probe enforcement. Each expected a ❌ Rejected response from the safeoutputs server, but all returned {"result":"success"} instead. Specifically:
- 1.3:
add_labels ["enhancement"] (not in allowed list) → success (should reject non-allowed label)
- 1.5:
add_labels ["smoke-test"] item_number=1 (explicit non-triggering item, target:triggering) → success (should reject non-triggering target)
- 2.2:
remove_labels ["verified"] (not in allowed list) → success (should reject non-allowed label)
- 2.4:
remove_labels ["smoke-test"] item_number=1 (explicit non-triggering item, target:triggering) → success (should reject non-triggering target)
This may indicate that enforcement is deferred or operates as a no-op when there is no triggering item, rather than actively rejecting invalid operations. A pull_request-triggered run is needed to fully validate enforcement behavior.
🏷️ Safe-outputs labels enforcement test by Smoke Safe-Outputs Labels
Safe-Outputs Labels Enforcement Test Results
Run: https://github.com/github/gh-aw-mcpg/actions/runs/27209141961
Trigger: schedule
Configuration: add-labels (allowed:[smoke-test,verified], max:2, target:triggering), remove-labels (allowed:[smoke-test], max:1, target:triggering)
Phase 1: add-labels
{"result":"success"}{"result":"success"}Phase 2: remove-labels
{"result":"success"}{"result":"success"}Summary