Skip to content

docs(agents): add Fail Loud, Never Fake error-handling rule#101

Merged
don-petry merged 53 commits into
mainfrom
feat/agents-fail-loud
May 11, 2026
Merged

docs(agents): add Fail Loud, Never Fake error-handling rule#101
don-petry merged 53 commits into
mainfrom
feat/agents-fail-loud

Conversation

@don-petry

@don-petry don-petry commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a new Fail Loud, Never Fake subsection to AGENTS.md under Coding Standards & Principles, immediately after Defensive Coding at System Boundaries. The new rule codifies a long-standing failure mode: agents (and humans) hiding errors behind silent fallbacks, placeholder data, or false success reports — turning what should be a 5-minute stack-trace fix into a day lost chasing wrong data downstream.

The section establishes:

  • A priority ladder for failure handling: works → disclosed fallback → clear error → never silent
  • Forbidden patterns (bare except: swallowing, synthetic data substitution, false "I've set up X" reports, weakening tests to make them pass)
  • A clear carve-out that disclosed fallbacks are fine — the rule is against hidden fallbacks, not all fallbacks
  • Agentic directives mirroring the style of the existing Breaking Changes section

Why

This complements the existing Fail fast bullet, which covers invariant violations, by addressing the adjacent (and more insidious) problem of work that appears to succeed but didn't. The two rules reinforce each other.

Test plan

  • Markdown lint passes (.markdownlint-cli2.yaml)
  • Section renders correctly on github.com (heading hierarchy, tables, code formatting)
  • No broken cross-references — this section is self-contained and adds no new links

Summary by CodeRabbit

  • Documentation
    • Introduced a “Fail Loud, Never Fake” standard to development guidelines: require surfacing, logging, or propagating exceptions; forbid silent degradation, swallowing errors, undisclosed synthetic or hardcoded fallback data, and reporting success after errors; mandate observable fallbacks (response flags/fields, logs/metrics, or UI indicators); and require fixing failing tests rather than weakening them to hide failures.

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 9, 2026 00:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new documentation rule to the agent/human contribution guidelines to prevent silent fallbacks and “fake success” outcomes, complementing the existing defensive coding and fail-fast principles.

Changes:

  • Introduces a “Fail Loud, Never Fake” subsection with a clear priority ladder for handling failures.
  • Documents explicitly forbidden error-handling patterns (swallowed exceptions, synthetic substitutions, false success reporting, test weakening).
  • Clarifies that disclosed fallbacks are acceptable when observability is maintained (logs/flags/metrics/UI).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@coderabbitai

coderabbitai Bot commented Apr 9, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 678cc495-54be-47b7-baab-e19e5d19f837

📥 Commits

Reviewing files that changed from the base of the PR and between f247b89 and 9dd9174.

📒 Files selected for processing (1)
  • AGENTS.md

📝 Walkthrough

Walkthrough

Adds a "Fail Loud, Never Fake" section to AGENTS.md defining ordered failure-handling preferences, enumerating forbidden failure-hiding patterns, and adding agent directives to surface/log/propagate exceptions and make any fallbacks observable.

Changes

Cohort / File(s) Summary
Error Handling Guidelines
AGENTS.md
Added "Fail Loud, Never Fake" section: explicit failure-handling priority (real correct data → observable degraded fallback → clear error → no silent degradation); enumerated forbidden behaviors (swallowed exceptions, synthetic/placeholder data, hardcoded success after errors, undisclosed fallback success, weakened tests); agentic directives requiring exceptions be surfaced/logged/propagated and fallbacks be observable.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title accurately describes the main change: adding a new 'Fail Loud, Never Fake' error-handling rule to the AGENTS.md documentation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/agents-fail-loud

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@don-petry

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Apr 9, 2026

Copy link
Copy Markdown
✅ Actions performed

Full review triggered.

@don-petry

Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: f247b896c91336b74de79c8b4b6434a1e5771e95
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Summary

Documentation-only change adding a 'Fail Loud, Never Fake' error-handling guideline to AGENTS.md. All CI checks pass (Lint, ShellCheck, CodeQL, SonarCloud, AgentShield), SonarCloud reports zero issues and zero security hotspots, and CodeRabbit has already approved. The content is well-structured, internally consistent, and complementary to the existing 'Fail fast' and defensive-coding sections with no security, correctness, or maintainability concerns.

Findings

Info

  • AGENTS.md:288 — The 'Fail Loud, Never Fake' section is coherent and well-scoped: priority ladder, forbidden patterns, disclosed-fallback carve-out, and agentic directives all align with the surrounding AGENTS.md style. No broken cross-references; section is self-contained.
  • Triage escalated due to 'triage-output-invalid' (triage tier crashed), not because of actual risk signals. Deep review confirms no code, config, or security-sensitive files were modified.

CI status

All status checks passed: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED). No failures or warnings.


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

@don-petry don-petry enabled auto-merge (squash) April 16, 2026 12:11
@don-petry

Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: d82732af0cc1ab8142c365fdb6e39b4196fb5790
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Note: Formal approval skipped — reviewer is the PR author. Verdict: APPROVE.

Summary

Documentation-only PR adding a 'Fail Loud, Never Fake' error-handling section to AGENTS.md. Triage escalated due to a triage-tier crash (triage-output-invalid), not any actual risk signal — the prior review at f247b89 already confirmed this is LOW risk. All CI checks pass (Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud with 0 issues/0 hotspots, CodeRabbit APPROVED), no code or config files modified, no new dependencies.

Findings

Info

  • [info] AGENTS.md:289 — 'Fail Loud, Never Fake' section is well-structured with a clear priority ladder (works correctly → visible fallback → clear error → never silent), explicit forbidden patterns, a carve-out for disclosed fallbacks, and five numbered agentic directives. Style is consistent with surrounding sections (e.g., Defensive Coding, Breaking Changes). Section is self-contained with no new cross-references to break.
  • [info] Triage escalated with signal 'triage-output-invalid' — the triage tier crashed rather than detecting a genuine risk. This deep review confirms no security, correctness, or maintainability concerns exist.
  • [info] All status checks passed at head SHA d82732a: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL (actions), SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED). Dependency audit jobs skipped correctly (no dependency files changed).
  • [info] Only file changed: AGENTS.md (+32/-0 lines). No code, config, secrets, CI workflow, schema, or dependency files were modified. Risk classification LOW is appropriate.

CI status

All CI checks passed at head SHA d82732af0cc1ab8142c365fdb6e39b4196fb5790: Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud (0 new issues, 0 security hotspots), CodeRabbit (APPROVED).


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

@don-petry don-petry disabled auto-merge April 16, 2026 17:08
@don-petry don-petry enabled auto-merge (squash) April 16, 2026 17:09
@don-petry

Copy link
Copy Markdown
Contributor Author

Automated review — APPROVED

Risk: LOW
Reviewed commit: 9dd91748e727a346dbeb7660e0291d923fd7b118
Cascade: triage → deep (see triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6 for models)

Summary

Documentation-only PR adding a 'Fail Loud, Never Fake' error-handling section to AGENTS.md (+32/-0). Triage escalated due to a triage-tier crash (triage-output-invalid), not any genuine risk signal. All CI checks pass (Lint, ShellCheck, AgentShield, Agent Security Scan, CodeQL, SonarCloud with 0 new issues and 0 security hotspots, CodeRabbit APPROVED), no code/config/secrets/CI-workflow files were modified, and no new dependencies introduced.

Findings

Info

  • AGENTS.md:289 — New 'Fail Loud, Never Fake' section is well-structured: priority ladder (works correctly → visible fallback → clear error → never silent), explicit forbidden patterns, a disclosed-fallback carve-out, and five numbered agentic directives. Style and heading hierarchy are consistent with surrounding AGENTS.md sections (Defensive Coding, Breaking Changes). Section is self-contained with no new cross-references.
  • (triage-signal) Triage escalated with signal 'triage-output-invalid' — the triage tier crashed rather than detecting a real risk. This deep review (and the two prior deep reviews at f247b89 and d82732a) confirm no security, correctness, or maintainability concerns exist. Head SHA 9dd9174 is a merge-from-main commit with no substantive diff beyond the already-reviewed documentation.

CI status

All status checks passed at head SHA 9dd91748e727a346dbeb7660e0291d923fd7b118: Lint (SUCCESS), ShellCheck (SUCCESS), AgentShield (SUCCESS), Agent Security Scan (SUCCESS), CodeQL/actions (SUCCESS), SonarCloud (SUCCESS, 0 new issues, 0 security hotspots), CodeRabbit (APPROVED). Dependency audit jobs correctly skipped — no dependency files changed.


Reviewed by the don-petry PR-review cascade (triage: haiku 4.5 → deep: sonnet 4.6 + duck: gpt-5.4 → audit: opus 4.6). Reply with @don-petry if you need a human.

@sonarqubecloud

sonarqubecloud Bot commented May 7, 2026

Copy link
Copy Markdown

@sonarqubecloud

Copy link
Copy Markdown

@don-petry don-petry merged commit 00c6cf3 into main May 11, 2026
20 checks passed
@don-petry don-petry deleted the feat/agents-fail-loud branch May 11, 2026 20:19
don-petry added a commit that referenced this pull request Jun 8, 2026
* docs(agents): add Fail Loud, Never Fake error-handling rule

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(agents): wrap long lines to satisfy MD013 (max 200)

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: don-petry <don-petry@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
don-petry added a commit that referenced this pull request Jun 10, 2026
* docs(agents): add Fail Loud, Never Fake error-handling rule

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(agents): wrap long lines to satisfy MD013 (max 200)

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: don-petry <don-petry@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
don-petry added a commit that referenced this pull request Jun 11, 2026
* docs(agents): add Fail Loud, Never Fake error-handling rule

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(agents): wrap long lines to satisfy MD013 (max 200)

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: don-petry <don-petry@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
don-petry added a commit that referenced this pull request Jun 11, 2026
* docs(agents): add Fail Loud, Never Fake error-handling rule

Adds a new subsection under Coding Standards & Principles establishing
that visible failure is preferred over silent fallbacks. Codifies a
priority ladder (works → disclosed fallback → clear error → never
silent), forbidden patterns, and agentic directives forbidding swallowed
exceptions, fake/placeholder data substitution, and false success
reports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(agents): wrap long lines to satisfy MD013 (max 200)

Two new lines exceeded the 200-char markdownlint limit. Wrap them at
sentence boundaries to match the style used in the surrounding sections.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: don-petry <don-petry@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants