Skip to content

#462 Add advisability dimension to /assess-ticket #498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
williamthorsen merged 2 commits into from
May 2, 2026
+76 −26
Merged

#462 Add advisability dimension to /assess-ticket #498

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4c0713c
agents|feat: Add advisability dimension to /assess-ticket
williamthorsen May 2, 2026
6c686d0
agents|feat: Link solution-soundness facet to design priorities
williamthorsen May 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/agents/content/skills/_data/ticket-evaluation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ A ticket is a request, report, or suggestion — not a contract. The author may

- **Is the problem real?** Reproduce or verify the underlying observation, not just the framing.
- **Is the scope right?** Look for related instances of the same defect class, neighboring code paths, or assumptions that don't hold beyond the ticket's frame.
- **Is the proposed solution sound?** A ticket may include a recommended fix that solves the symptom but not the cause, or that fixes the cause less well than an alternative.
- **Is the proposed solution sound?** A ticket may include a recommended fix that solves the symptom but not the cause, or that fixes the cause less well than an alternative. Apply the [design priorities](./design-priorities.md) lens — a convenient-but-wrong fix is unsound, even if it matches surrounding code.
- **Is the title accurate?** "Flaky test in X" may be a hygiene anti-pattern that exists in three places. Reframe titles when the underlying truth is broader or different.

When evaluation surfaces a divergence from the ticket as written, raise the observation to the user before designing — but lead with the substantive finding, not the ticket. The ticket's job was to start the conversation; once it has started, the codebase reality and the user's judgment govern, not the original wording.
52 changes: 40 additions & 12 deletions packages/agents/content/skills/assess-ticket/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
---
name: assess-ticket
description: Assess a ticket against the current codebase for drift, relevance, progress, and complexity, and prompt for follow-up actions
description: Assess a ticket against the current codebase for drift, relevance, progress, advisability, and complexity, and prompt for follow-up actions
user-invocable: true
---

# Assess ticket

Assess a ticket against the current codebase across four dimensions: drift, relevance, progress, and complexity. Produces a structured assessment with constrained verdicts and supporting evidence.
Assess a ticket against the current codebase across five dimensions: drift, relevance, progress, advisability, and complexity. Produces a structured assessment with constrained verdicts and supporting evidence.

**Announce at start:** "Using assess-ticket to assess {ticket reference} (mode: {mode})."

## Arguments

- **Ticket source** (optional): issue URL, shorthand reference (`#99`, `issue 99`), file path, or plain text. When omitted, auto-resolved from the environment (see [ticket source resolution](../_data/ticket-source-resolution.md#auto-resolve)).
- **Mode** (optional): `drift`, `relevance`, `progress`, `complexity`, or `all` (default: `all`)
- **Mode** (optional): `drift`, `relevance`, `progress`, `advisability`, `complexity`, or `all` (default: `all`)

## Process

Expand All @@ -23,7 +23,7 @@ Resolve the ticket source using the [ticket source resolution](../_data/ticket-s

### 2. Investigate

Run the investigation for the requested mode (or all modes in order when mode is `all`). When mode is `all`, complexity is investigated last so it benefits from context gathered during drift, relevance, and progress analysis. **Skip complexity when progress is `complete`** — complexity is forward-looking and has no value for finished work.
Run the investigation for the requested mode (or all modes in order when mode is `all`). When mode is `all`, advisability is investigated after progress (synthesizing the prior dimensions' context), and complexity is investigated last so it benefits from context gathered during drift, relevance, progress, and advisability analysis. **Skip both advisability and complexity when progress is `complete`** — neither has value for finished work.

#### Drift

Expand Down Expand Up @@ -79,6 +79,22 @@ Determine whether the described work has been implemented. The output format ada
- 🟠 `partial` — some of the described work has been done but significant portions remain
- 🔴 `none` — none of the described work is present in the codebase

#### Advisability

Determine whether the ticket should be implemented as written. Synthesize the four facets defined in [ticket evaluation](../_data/ticket-evaluation.md) (problem reality, scope correctness, solution soundness, title accuracy) against the codebase and ticket text.

1. Apply each facet in turn — does the underlying observation hold? Is scope right at the appropriate class? Does the proposed solution treat the cause? Does the title accurately describe the work?
2. Synthesize a verdict from the facet results.
3. Emit one prose evidence bullet per concern surfaced. Bullets do not prefix facet names. Omit bullets entirely when the verdict is `advisable`.

Bias toward `advisable` — for a recommendation dimension, false-positive concerns are noisier than false-negative passes. Default to `advisable` unless the codebase yields specific evidence of a facet concern.

**Verdicts:**

- 🟢 `advisable` — recommend implementing as written; all four facets pass scrutiny
- 🟠 `questionable` — recommend with concerns; one or more facets surface issues warranting human review
- 🔴 `inadvisable` — recommend against implementing as written; rework needed before proceeding

#### Complexity

Classify how complex the described work is relative to the current codebase. Reference the [complexity classification](../_data/complexity-classification.md) rubric for level definitions.
Expand All @@ -97,7 +113,7 @@ Classify how complex the described work is relative to the current codebase. Ref

### 3. Output

Format the assessment using the structure below. When a single mode is requested, output only that dimension's section (with the header and provenance line). When mode is `all`, output all dimensions in order — omitting complexity when progress is `complete`.
Format the assessment using the structure below. When a single mode is requested, output only that dimension's section (with the header and provenance line). When mode is `all`, output all dimensions in order — omitting both advisability and complexity when progress is `complete`.

Obtain the base SHA via `git rev-parse --short HEAD`.

Expand All @@ -124,6 +140,11 @@ Assessed at {YYYYMMDD-HHMMSSZ} against {short SHA}
- ✅ {Criterion met}
- ❌ {Criterion not met}

🧭 **Advisability:** {emoji} `{verdict}`

- {Evidence bullet}
- {Evidence bullet}

🧩 **Complexity:** {emoji} `{label}`

- {Evidence bullet}
Expand Down Expand Up @@ -152,6 +173,11 @@ Assessed at {YYYYMMDD-HHMMSSZ} against {short SHA}
- {Evidence bullet}
- {Evidence bullet}

🧭 **Advisability:** {emoji} `{verdict}`

- {Evidence bullet}
- {Evidence bullet}

🧩 **Complexity:** {emoji} `{label}`

- {Evidence bullet}
Expand All @@ -164,7 +190,7 @@ After presenting the assessment output, evaluate whether any verdicts are non-ba

### Emoji mapping

Drift, relevance, and progress use a **concern scale** — green means no concern, red means high concern:
Drift, relevance, progress, and advisability use a **concern scale** — green means no concern, red means high concern:

| Verdict position | Emoji |
| ---------------- | ----- |
Expand All @@ -183,16 +209,18 @@ Complexity uses a **size scale** — emojis represent effort and scope, not conc

### Verdict reference

| Dimension | ⚪ | 🟢 | 🟠 | 🔴 |
| -------------- | --------- | ------------ | ----------- | --------------- |
| **Drift** | — | `none` | `partial` | `severe` |
| **Relevance** | — | `relevant` | `uncertain` | `superseded` |
| **Progress** | — | `complete` | `partial` | `none` |
| **Complexity** | `trivial` | `mechanical` | `involved` | `architectural` |
| Dimension | ⚪ | 🟢 | 🟠 | 🔴 |
| ---------------- | --------- | ------------ | -------------- | --------------- |
| **Drift** | — | `none` | `partial` | `severe` |
| **Relevance** | — | `relevant` | `uncertain` | `superseded` |
| **Progress** | — | `complete` | `partial` | `none` |
| **Advisability** | — | `advisable` | `questionable` | `inadvisable` |
| **Complexity** | `trivial` | `mechanical` | `involved` | `architectural` |

## Key principles

- **Evidence over opinion** — every verdict must be supported by specific evidence (file paths, commit SHAs, code references)
- **Prefer caution on relevance** — use `uncertain` when signals are ambiguous rather than committing to `superseded`
- **Bias `advisable` absent evidence** — Advisability fires a next-steps prompt on every non-baseline verdict; default to `advisable` unless the codebase yields specific evidence of a facet concern
- **Assessment first, action on request** — lead with the assessment; offer follow-up actions but do not execute without user selection
- **Scale to ticket complexity** — a simple ticket gets a brief assessment; a complex ticket with many acceptance criteria gets a thorough one
48 changes: 35 additions & 13 deletions packages/agents/content/skills/assess-ticket/next-steps-after-assessment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@ Prompt for follow-up actions when any assessment verdict is non-baseline. The pr

A verdict is **baseline** when it indicates no concern or action needed:

| Dimension | Baseline verdict |
| --------- | ---------------- |
| Drift | `none` |
| Relevance | `relevant` |
| Progress | `none` |
| Dimension | Baseline verdict |
| ------------ | ---------------- |
| Drift | `none` |
| Relevance | `relevant` |
| Progress | `none` |
| Advisability | `advisable` |

Complexity verdicts are purely informational and never trigger a prompt.

Expand All @@ -30,14 +31,23 @@ Each non-baseline verdict maps to one or more follow-up actions. Actions describ

These collapse into a single "Update ticket" action regardless of how many dimensions trigger it.

### ♻️ Revise actions

| Verdict | Action |
| ---------------------------- | ---------------------------------------------- |
| Advisability: `questionable` | Revise ticket to address advisability concerns |

This action shares the ♻️ icon with "Update ticket to match current codebase state" but is a distinct action — it covers content or scope rework prompted by an advisability concern, not factual sync with the codebase. When both fire, both appear as separate options.

### 🏁 Close actions

| Verdict | Action |
| ----------------------- | ------------------------------------------------------ |
| Drift: `severe` | Close as outdated |
| Relevance: `superseded` | Close with comment explaining what superseded it |
| Progress: `partial` | Close and create a new ticket scoped to remaining work |
| Progress: `complete` | Close with comment summarizing completed work |
| Verdict | Action |
| --------------------------- | ------------------------------------------------------ |
| Drift: `severe` | Close as outdated |
| Relevance: `superseded` | Close with comment explaining what superseded it |
| Progress: `partial` | Close and create a new ticket scoped to remaining work |
| Progress: `complete` | Close with comment summarizing completed work |
| Advisability: `inadvisable` | Close as inadvisable |

When multiple close actions apply, present each as a separate option — they differ in intent and outcome.

Expand All @@ -51,8 +61,8 @@ When multiple close actions apply, present each as a separate option — they di

When multiple dimensions produce non-baseline verdicts, their actions are combined into a single numbered list:

1. **Collect** actions grouped by type: ♻️ update → 🏁 close → 💬 comment.
2. **Deduplicate** — the update action appears at most once regardless of how many verdicts trigger it.
1. **Collect** actions grouped by type: ♻️ update → ♻️ revise → 🏁 close → 💬 comment.
2. **Deduplicate** — each distinct action appears at most once. ♻️ "Update ticket to match current codebase state" and ♻️ "Revise ticket to address advisability concerns" are distinct actions and remain as separate options when both fire.
3. Number sequentially.

## Interaction protocol
Expand Down Expand Up @@ -81,3 +91,15 @@ Next steps:
```

Action 1 combines drift and progress update triggers into a single option. Action 2 comes from progress. Relevance is baseline and contributes nothing.

## Example: distinct ♻️ flavors

Assessment produces drift: `partial` and advisability: `questionable` (other dimensions baseline):

```
Next steps:
1. ♻️ Update ticket to match current codebase state
2. ♻️ Revise ticket to address advisability concerns
```

Both actions share the ♻️ icon but represent distinct intents — factual sync with the codebase versus content/scope rework — so they appear as separate numbered options rather than collapsing under the dedupe rule.
Loading