Skip to content

Fix parser audit finding filters - #82

Merged
TraderSamwise merged 2 commits into
masterfrom
test/chat-parser-candidate-mining
Jun 6, 2026
Merged

Fix parser audit finding filters#82
TraderSamwise merged 2 commits into
masterfrom
test/chat-parser-candidate-mining

Conversation

@TraderSamwise

@TraderSamwise TraderSamwise commented Jun 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • tighten parser audit status-leak matching to line-shaped terminal/status output
  • add repeatable --flag filtering for parser audit mining
  • cover prose/code false positives and filtered finding counts

Verification

  • vitest run src/agent-output-parser-audit.test.ts
  • tsc --noEmit
  • eslint src/ --max-warnings=0
  • yarn build
  • node scripts/audit-agent-output-parser.mjs --history /Users/sam/cs/aimux/.aimux/history --context /Users/sam/cs/aimux/.aimux/context --flag status-leak-response --max 10
  • vitest run all src tests except src/daemon.test.ts (daemon test is blocked locally by live aimux-dev service on fixed port)

Summary by CodeRabbit

  • New Features

    • Added a CLI option to specify one or more flags to include when running the parser audit, with validation of provided values.
  • Improvements

    • Broadened status-leak detection to use multiple recognized patterns for more reliable detection.
    • Audit output now filters findings and aggregate counts to only the explicitly requested flags when provided.
  • Tests

    • Added tests covering flag filtering and the updated status-leak pattern variants.

@vercel

vercel Bot commented Jun 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
app Ready Ready Preview, Comment Jun 6, 2026 12:33pm

@coderabbitai

coderabbitai Bot commented Jun 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ade73ba9-7ac8-4255-917c-35af6f011995

📥 Commits

Reviewing files that changed from the base of the PR and between 01f9ded and 18a8e46.

📒 Files selected for processing (2)
  • src/agent-output-parser-audit.test.ts
  • src/agent-output-parser-audit.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/agent-output-parser-audit.test.ts
  • src/agent-output-parser-audit.ts

📝 Walkthrough

Walkthrough

This PR adds flag-based filtering to the audit agent output parser. The ParserAuditOptions interface gains an optional flags parameter to restrict which audit findings are retained. Status-leak detection is expanded from a single regex to multiple patterns. The CLI script now supports --flag <value> arguments that are validated and passed through to the audit function.

Changes

Audit Flag Filtering

Layer / File(s) Summary
Status-leak detection patterns
src/agent-output-parser-audit.ts, src/agent-output-parser-audit.test.ts
ParserAuditOptions adds optional flags field. Status-leak detection refactors from single STATUS_LEAK_RESPONSE_PATTERN regex to array-based STATUS_LEAK_RESPONSE_PATTERNS checked with some(). Test fixtures updated to match terminal-notifier variants; new tests verify both matching and non-matching prose/code scenarios.
Finding flags filtering
src/agent-output-parser-audit.ts, src/agent-output-parser-audit.test.ts
Backward iteration removes any flag not in options.flags when filter is provided. Test confirms countsByFlag and findings restrict to requested flags.
CLI argument parsing and wiring
scripts/audit-agent-output-parser.mjs
Adds --flag <flag> option with validFlags set for validation. Argument parser collects flags and passes them to auditAgentOutputParserCorpus call.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • TraderSamwise/aimux#81: Both modify the same parser-audit tooling components, with main PR extending the corpus audit CLI by adding --flag filtering and status-leak pattern handling.
  • TraderSamwise/aimux#79: Updates parsing/classification of terminal-notifier/status text variants which affects audit detection and test expectations.

Poem

🐇 I sniffed the flags, I hopped the tests,
Patterns wide, I checked the nests,
CLI ears twitch, arguments in line,
Findings filtered, counted fine,
Hooray — the parser's garden's mine!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title "Fix parser audit finding filters" directly reflects the main change: adding a --flag filtering option to restrict detected finding flags in the parser audit logic.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/chat-parser-candidate-mining

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/agent-output-parser-audit.ts`:
- Around line 46-47: The two fallback regexes /(?:^|\n)\s*Bash\(/ and
/(?:^|\n)\s*Read\d/i are too permissive and produce false positives; tighten
them to only match true terminal/status lines by anchoring to the start of a
line (use ^ rather than (?:^|\n)), require explicit terminal markers or
delimiters (e.g., require a trailing closing delimiter, colon, prompt symbol, or
end-of-line), and add word-boundary or digit-exactness checks so they don’t
match ordinary prose — update the occurrences of these regex literals in the
parser to the stricter forms.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0aa79f36-cb44-425c-925f-fbca3fecbdb0

📥 Commits

Reviewing files that changed from the base of the PR and between 47bc074 and 01f9ded.

📒 Files selected for processing (3)
  • scripts/audit-agent-output-parser.mjs
  • src/agent-output-parser-audit.test.ts
  • src/agent-output-parser-audit.ts

Comment thread src/agent-output-parser-audit.ts Outdated
@TraderSamwise
TraderSamwise merged commit c1eb0d0 into master Jun 6, 2026
3 checks passed
@TraderSamwise
TraderSamwise deleted the test/chat-parser-candidate-mining branch June 6, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant