diff --git a/.github/workflows/smoke-copilot-byok-aoai-apikey.md b/.github/workflows/smoke-copilot-byok-aoai-apikey.md index 321804a3..c9a7fcbd 100644 --- a/.github/workflows/smoke-copilot-byok-aoai-apikey.md +++ b/.github/workflows/smoke-copilot-byok-aoai-apikey.md @@ -166,11 +166,14 @@ ${{ steps.smoke-data.outputs.SMOKE_PR_DATA }} ## Output -Add a **very brief** comment (max 5-10 lines) to the current pull request with: +**If triggered by a pull request**, call `add_comment` to post a **very brief** comment (max 5-10 lines) on the current pull request with: - PR titles only (no descriptions) - ✅ or ❌ for each test result - Note: "Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw)" - Overall status: PASS or FAIL - Mention the pull request author and any assignees -If all tests pass, add the label `smoke-copilot-byok-aoai-apikey` to the pull request. +If all tests pass on a pull request trigger: +- Use the `add_labels` safe-output tool to add the label `smoke-copilot-byok-aoai-apikey` to the pull request + +**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request. diff --git a/.github/workflows/smoke-copilot-byok-aoai-entra.md b/.github/workflows/smoke-copilot-byok-aoai-entra.md index be5c55d5..d3c81348 100644 --- a/.github/workflows/smoke-copilot-byok-aoai-entra.md +++ b/.github/workflows/smoke-copilot-byok-aoai-entra.md @@ -202,11 +202,14 @@ ${{ steps.smoke-data.outputs.SMOKE_PR_DATA }} ## Output -Add a **very brief** comment (max 5-10 lines) to the current pull request with: +**If triggered by a pull request**, call `add_comment` to post a **very brief** comment (max 5-10 lines) on the current pull request with: - PR titles only (no descriptions) - ✅ or ❌ for each test result - Note: "Running in direct BYOK mode (AWF_AUTH_TYPE=github-oidc + AWF_AUTH_AZURE_* + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) authenticated via Microsoft Entra" - Overall status: PASS or FAIL - Mention the pull request author and any assignees -If all tests pass, add the label `smoke-copilot-byok-aoai-entra` to the pull request. +If all tests pass on a pull request trigger: +- Use the `add_labels` safe-output tool to add the label `smoke-copilot-byok-aoai-entra` to the pull request + +**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request. diff --git a/.github/workflows/smoke-copilot-byok.md b/.github/workflows/smoke-copilot-byok.md index 651f0ad9..c61022ce 100644 --- a/.github/workflows/smoke-copilot-byok.md +++ b/.github/workflows/smoke-copilot-byok.md @@ -156,14 +156,17 @@ You are running in direct BYOK mode right now. The fact that you can read this p ## Output -Add a **very brief** comment (max 5-10 lines) to the current pull request with: +**If triggered by a pull request**, call `add_comment` to post a **very brief** comment (max 5-10 lines) on the current pull request with: - PR titles only (no descriptions) - ✅ or ❌ for each test result - Note: "Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY) via api-proxy → api.githubcopilot.com" - Overall status: PASS or FAIL - Mention the pull request author and any assignees -If all tests pass, add the label `smoke-copilot-byok` to the pull request. +If all tests pass on a pull request trigger: +- Use the `add_labels` safe-output tool to add the label `smoke-copilot-byok` to the pull request + +**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request. ## Pre-Fetched Data diff --git a/.github/workflows/smoke-copilot-pat.md b/.github/workflows/smoke-copilot-pat.md index d7c63914..2728ff19 100644 --- a/.github/workflows/smoke-copilot-pat.md +++ b/.github/workflows/smoke-copilot-pat.md @@ -134,11 +134,14 @@ ${{ steps.smoke-data.outputs.SMOKE_PR_DATA }} ## Output -Add a **very brief** comment (max 5-10 lines) to the current pull request with: +**If triggered by a pull request**, call `add_comment` to post a **very brief** comment (max 5-10 lines) on the current pull request with: - PR titles only (no descriptions) - ✅ or ❌ for each test result - Overall status: PASS or FAIL - Mention the pull request author and any assignees - Note: "Auth mode: PAT (COPILOT_GITHUB_TOKEN)" -If all tests pass, add the label `smoke-copilot-pat` to the pull request. +If all tests pass on a pull request trigger: +- Use the `add_labels` safe-output tool to add the label `smoke-copilot-pat` to the pull request + +**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request. diff --git a/.github/workflows/smoke-copilot.md b/.github/workflows/smoke-copilot.md index 334c15d7..b6816c4d 100644 --- a/.github/workflows/smoke-copilot.md +++ b/.github/workflows/smoke-copilot.md @@ -127,10 +127,13 @@ ${{ steps.smoke-data.outputs.SMOKE_PR_DATA }} ## Output -Add a **very brief** comment (max 5-10 lines) to the current pull request with: +**If triggered by a pull request**, call `add_comment` to post a **very brief** comment (max 5-10 lines) on the current pull request with: - PR titles only (no descriptions) - ✅ or ❌ for each test result - Overall status: PASS or FAIL - Mention the pull request author and any assignees -If all tests pass, add the label `smoke-copilot` to the pull request. \ No newline at end of file +If all tests pass on a pull request trigger: +- Use the `add_labels` safe-output tool to add the label `smoke-copilot` to the pull request + +**If triggered by workflow_dispatch or schedule** (no PR context), call `noop` with a concise PASS/FAIL summary instead. Do NOT attempt to add pull request comments or labels when there is no pull request. \ No newline at end of file diff --git a/scripts/ci/smoke-copilot-workflow.test.ts b/scripts/ci/smoke-copilot-workflow.test.ts new file mode 100644 index 00000000..1bcd245f --- /dev/null +++ b/scripts/ci/smoke-copilot-workflow.test.ts @@ -0,0 +1,29 @@ +import * as fs from 'fs'; +import * as path from 'path'; + +const workflowsDir = path.resolve(__dirname, '../../.github/workflows'); + +const copilotSmokeWorkflows = [ + { name: 'smoke-copilot-byok-aoai-apikey', file: 'smoke-copilot-byok-aoai-apikey.md' }, + { name: 'smoke-copilot-byok-aoai-entra', file: 'smoke-copilot-byok-aoai-entra.md' }, + { name: 'smoke-copilot-byok', file: 'smoke-copilot-byok.md' }, + { name: 'smoke-copilot', file: 'smoke-copilot.md' }, + { name: 'smoke-copilot-pat', file: 'smoke-copilot-pat.md' }, +]; + +describe('smoke copilot workflow output requirements', () => { + for (const workflow of copilotSmokeWorkflows) { + it(`${workflow.name}: requires noop fallback when no pull request context exists`, () => { + const source = fs.readFileSync(path.join(workflowsDir, workflow.file), 'utf-8'); + + expect(source).toContain('**If triggered by a pull request**, call `add_comment`'); + expect(source).toContain('If all tests pass on a pull request trigger:'); + expect(source).toContain( + '**If triggered by workflow_dispatch or schedule** (no PR context), call `noop`' + ); + expect(source).toContain( + 'Do NOT attempt to add pull request comments or labels when there is no pull request.' + ); + }); + } +});