Skip to content

Fix OpenCode fallback approval for green model-output failures #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
seonghobae merged 1 commit into from
Jun 30, 2026
Merged

Fix OpenCode fallback approval for green model-output failures #170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 13 additions & 24 deletions .github/workflows/opencode-review.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4674,18 +4674,11 @@ jobs:
build_central_review_process_fallback_approval_body "$fallback_changed_files_file" "$fallback_approval_body_file"
create_pull_review "APPROVE" "$(cat "$fallback_approval_body_file")"
else
body="$(printf '%s\n' \
"all configured OpenCode model attempts failed to produce a usable current-head control block." \
"" \
"- Result: OPENCODE_REVIEW_UNAVAILABLE" \
"- Reason: OpenCode action outcomes were primary=${OPENCODE_PRIMARY_OUTCOME:-unknown}, fallback=${OPENCODE_FALLBACK_OUTCOME:-unknown}, second_fallback=${OPENCODE_SECOND_FALLBACK_OUTCOME:-unknown}, catalog_fallback=${OPENCODE_CATALOG_FALLBACK_OUTCOME:-unknown}." \
"- Required next evidence: rerun OpenCode with a model/tooling attempt that emits a valid source-backed control block for this head." \
"- Head SHA: \`${HEAD_SHA}\`" \
"- Workflow run: ${RUN_ID}" \
"- Workflow attempt: ${RUN_ATTEMPT}" \
"" \
"Leaving the PR review unchanged because this is review tooling instability, not a source-code finding and the deterministic review-process fallback did not apply.")"
stop_approval_without_review "OPENCODE_REVIEW_UNAVAILABLE" "$body"
approve_after_model_failure_when_current_head_gates_pass \
"$pending_checks_file" \
"$failed_checks_file" \
"$unresolved_human_threads_file" \
"$human_thread_review_body_file"
fi
fi
echo "::endgroup::"
Expand Down Expand Up @@ -4995,18 +4988,14 @@ jobs:
if request_changes_for_merge_conflict_if_present; then
:
else
body="$(printf '%s\n' \
"OpenCode gate result was not publishable for the current head." \
"" \
"- Result: OPENCODE_REVIEW_UNAVAILABLE" \
"- Reason: OpenCode gate result ${gate_result:-empty} was not publishable for head ${HEAD_SHA}." \
"- Required next evidence: rerun OpenCode with a valid source-backed control block, or obtain a source-backed failed-check diagnosis." \
"- Head SHA: \`${HEAD_SHA}\`" \
"- Workflow run: ${RUN_ID}" \
"- Workflow attempt: ${RUN_ATTEMPT}" \
"" \
"Leaving the PR review unchanged because this is review tooling instability, not a source-code finding.")"
stop_approval_without_review "OPENCODE_REVIEW_UNAVAILABLE" "$body"
pending_checks_file="$(mktemp)"
unresolved_human_threads_file="$(mktemp)"
human_thread_review_body_file="$(mktemp)"
approve_after_model_failure_when_current_head_gates_pass \
"$pending_checks_file" \
"$failed_checks_file" \
"$unresolved_human_threads_file" \
"$human_thread_review_body_file"
fi
fi
;;
Expand Down
10 changes: 5 additions & 5 deletions scripts/ci/test_strix_quick_gate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,9 +584,9 @@ assert_opencode_review_uses_codegraph_and_gpt5_fallback() {
assert_file_contains "$workflow_file" 'wait_for_peer_github_checks "$pending_checks_file"' "opencode central review fallback waits for peer checks before approval"
assert_file_contains "$workflow_file" 'collect_unresolved_human_review_threads "$unresolved_human_threads_file"' "opencode central review fallback re-queries human threads before approval"
assert_file_contains "$workflow_file" "deterministic approval fallback verified this is a no-diff or central review-process-only change" "opencode approval publishes transparent no-diff or central review fallback approvals"
assert_file_contains "$workflow_file" "deterministic review-process fallback did not apply" "opencode approval failure text names fallback ineligibility after retry exhaustion"
assert_file_contains "$workflow_file" "OpenCode model attempts did not emit a usable current-head control block, so the approval gate used deterministic current-head evidence instead of model prose." "opencode approval publishes guarded deterministic approval after model-output failures and green current-head gates"
assert_file_contains "$workflow_file" "all configured OpenCode model attempts failed to produce a usable current-head control block" "opencode model-output failures fail the check without publishing a review"
assert_file_contains "$workflow_file" "Leaving the PR review unchanged because this is review tooling instability, not a source-code finding." "opencode model-failure path avoids PR review noise outside central fallback scope"
assert_file_contains "$workflow_file" "Deterministic fallback approval was used only after model-output instability and did not bypass coverage, failed-check, mergeability, or human-review gates." "opencode model-failure path documents the guarded approval criteria"
assert_file_contains "$workflow_file" 'OPENCODE_MODEL_ATTEMPTS: "3"' "opencode primary and deepseek review paths retry model execution"
assert_file_contains "$workflow_file" 'OPENCODE_MODEL_ATTEMPTS: "1"' "opencode catalog fallback probes each model once so step timeout cannot cancel the approval fallback path"
assert_file_contains "$workflow_file" 'OPENCODE_RUN_TIMEOUT_SECONDS: "45"' "opencode catalog fallback is bounded tightly enough to reach deterministic review-process fallback before step timeout"
Expand Down Expand Up @@ -821,9 +821,9 @@ assert_opencode_review_uses_codegraph_and_gpt5_fallback() {
assert_file_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" '^CMD \["/app/scripts/docker_entrypoint\.sh"\]' "opencode failed-check fallback maps missing Docker entrypoint reports to the Dockerfile CMD line"
assert_file_contains "$workflow_file" "Unrelated speculative findings are invalid when failed-check evidence is present." "opencode review prompt forbids unrelated failed-check findings"
assert_file_contains "$workflow_file" "run_failed_check_diagnosis" "opencode approval gate reruns OpenCode diagnosis when checks fail after the initial review"
assert_file_contains "$workflow_file" "OpenCode action outcomes were primary=" "opencode approval gate records invalid model outcome details"
assert_file_contains "$workflow_file" "all configured OpenCode model attempts failed to produce a usable current-head control block" "opencode approval gate reports invalid model output as a review-governance blocker"
assert_file_contains "$workflow_file" "Leaving the PR review unchanged because this is review tooling instability, not a source-code finding." "opencode model-failure path fails the check instead of inventing a source-code finding"
assert_file_contains "$workflow_file" "OpenCode model attempts did not emit a usable current-head control block, so the approval gate used deterministic current-head evidence instead of model prose." "opencode approval gate records deterministic model-failure recovery"
assert_file_contains "$workflow_file" "Deterministic fallback approval was used only after model-output instability and did not bypass coverage, failed-check, mergeability, or human-review gates." "opencode approval gate documents guarded model-failure recovery"
assert_file_contains "$workflow_file" "approve_after_model_failure_when_current_head_gates_pass" "opencode model-failure path reuses green current-head gates instead of inventing a source-code finding"
assert_file_contains "$workflow_file" "request_changes_for_merge_conflict_if_present" "opencode approval gate checks mergeability before approving model or fallback output"
assert_file_contains "$workflow_file" "Merge Conflict Guidance" "opencode approval gate emits explicit conflict guidance when mergeability is dirty"
assert_file_contains "$workflow_file" "Change Flow DAG" "opencode review overview labels Mermaid as changed-file flow analysis"
Expand Down