Skip to content

fix: reduce public content credential fixture false positives - #1929

Merged
HanShaoshuai-k merged 1 commit into
mainfrom
fix/public-content-credential-fixtures
Jul 20, 2026
Merged

fix: reduce public content credential fixture false positives#1929
HanShaoshuai-k merged 1 commit into
mainfrom
fix/public-content-credential-fixtures

Conversation

@HanShaoshuai-k

@HanShaoshuai-k HanShaoshuai-k commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reduce false positives in public-content credential detection while preserving detection of real credentials.

Changes

  • Require value evidence instead of blocking on credential-like field names alone.
  • Recognize provider tokens, encoded credentials, high-entropy values, and typed assignments.
  • Allow metadata fields, placeholders, synthetic prefixes, masked values, and security terminology.
  • Preserve review-comment file paths for context-aware scanning.
  • Add regression tests for reported false positives and real credential cases.

Test Plan

  • Qualitygate tests and race tests

Summary by CodeRabbit

  • Bug Fixes
    • Improved credential auditing for public comments by considering both the comment text and its contextual path, so findings are attributed to the correct location.
    • Reduced false positives by tightening credential-shape, evidence, and parsing logic across additional assignment syntaxes and encoded/provider formats.
    • Enhanced detection accuracy for credential-like content embedded in structured inputs (for example, diagrams).
  • Tests
    • Expanded and strengthened scanning/auditing coverage with dynamic fixture values and broader allow/deny edge cases to prevent regressions.

@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The public-content scanner now uses provider-specific credential evidence, refined assignment parsing, and broader benign-value handling. Comment auditing carries review-comment paths into scanning and diagnostics, with expanded tests for detection and fixture behavior.

Changes

Public content auditing

Layer / File(s) Summary
Credential evidence and assignment rules
internal/qualitygate/publiccontent/credential.go, internal/qualitygate/publiccontent/rules.go, internal/qualitygate/publiccontent/scan.go
Provider identifiers, encoding and entropy checks, assignment parsing, typed RHS handling, and benign source-code classification were updated.
Scanner control flow and validation
internal/qualitygate/publiccontent/scan_test.go
Tests cover provider detection, fixture handling, typed assignments, placeholder values, source-code expressions, and generic credential decision matrices.
Comment path propagation
internal/qualitygate/cmd/comment-audit/main.go, internal/qualitygate/publiccontent/comment_audit.go, internal/qualitygate/cmd/comment-audit/main_test.go, internal/qualitygate/publiccontent/comment_audit_test.go
Review-comment paths are extracted from events, passed to ScanCommentAtPath, and preserved in diagnostics.
Collection and fixture validation
internal/qualitygate/publiccontent/collect_test.go, internal/qualitygate/rules/run_test.go, tests/e2e/new-public-workflow.test.sh
Fixtures use generated provider-like values, excerpt checks avoid fixed secrets, and expected findings and allow cases are updated.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EventFile
  participant CommentAudit
  participant PublicContent
  participant Diagnostics
  EventFile->>CommentAudit: read event body and review-comment path
  CommentAudit->>PublicContent: ScanCommentAtPath(kind, path, body)
  PublicContent->>Diagnostics: return diagnostics with the resolved path
  CommentAudit->>Diagnostics: print diagnostics and exit with report.ExitCode
Loading

Possibly related PRs

  • larksuite/cli#1700: Both modify credential and placeholder detection heuristics in scan.go and rules.go.
  • larksuite/cli#1718: Both adjust token and credential evidence detection logic.

Suggested labels: feature

Suggested reviewers: liangshuo-1

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: reducing false positives in public-content credential detection.
Description check ✅ Passed The description matches the template structure and covers summary, changes, and test plan, though Related Issues is omitted.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/public-content-credential-fixtures

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the size/L Large or sensitive change across domains or core paths label Jul 17, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/qualitygate/publiccontent/scan.go`:
- Around line 62-64: Update sourceCodeTypedCredentialRHS and both its
evidence-extraction and benign-expression classification call sites to accept
and use the current match offset location[0], rather than searching from the
start of the line. Propagate that offset through all affected paths, including
the additional occurrences noted in the review. Add an adjacent regression test
with two identical typed assignments on one line where only the second RHS is
provider-shaped, and verify the second assignment is evaluated correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 31834fb3-2579-4c41-95ba-f650b46bc221

📥 Commits

Reviewing files that changed from the base of the PR and between 4e0a6a9 and 8465147.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go

Comment thread internal/qualitygate/publiccontent/scan.go
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

🚀 PR Preview Install Guide

🧰 CLI update

npm i -g https://pkg.pr.new/larksuite/cli/@larksuite/cli@2ede076dde0e04336618291110b89ecc84730d51

🧩 Skill update

npx skills add larksuite/cli#fix/public-content-credential-fixtures -y -g

@codecov

codecov Bot commented Jul 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.12500% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.97%. Comparing base (44514ad) to head (2ede076).

Files with missing lines Patch % Lines
internal/qualitygate/publiccontent/scan.go 86.56% 5 Missing and 4 partials ⚠️
internal/qualitygate/cmd/comment-audit/main.go 53.33% 6 Missing and 1 partial ⚠️
internal/qualitygate/publiccontent/credential.go 96.07% 1 Missing and 1 partial ⚠️
internal/qualitygate/publiccontent/rules.go 95.45% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1929      +/-   ##
==========================================
- Coverage   75.02%   74.97%   -0.06%     
==========================================
  Files         894      895       +1     
  Lines       94294    94379      +85     
==========================================
+ Hits        70746    70761      +15     
- Misses      18137    18200      +63     
- Partials     5411     5418       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from 8465147 to 0ee2e94 Compare July 17, 2026 06:51

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/qualitygate/publiccontent/scan_test.go`:
- Around line 417-418: Replace aggregate count-only assertions with individual
classification assertions in internal/qualitygate/publiccontent/scan_test.go at
lines 417-418, 394-395, 597-598, 1150-1163, 1183-1184, and 1656-1657: verify
each benign-key, provider, readable, placeholder, expression, and source-literal
fixture directly as allowed or rejected, then retain combined deduplication
checks where applicable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c8e13390-295e-4117-9338-57fe18486bd3

📥 Commits

Reviewing files that changed from the base of the PR and between 8465147 and 0ee2e94.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go
🚧 Files skipped from review as they are similar to previous changes (7)
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/collect_test.go

Comment thread internal/qualitygate/publiccontent/scan_test.go Outdated
@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from 0ee2e94 to 037667f Compare July 17, 2026 07:03
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

PR Quality Summary

CI did not complete successfully. Use the failed check links below to decide whether this PR needs a code change or a rerun.

Failed checks

@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch 2 times, most recently from f6a0c8d to ee7d781 Compare July 20, 2026 03:16

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
internal/qualitygate/cmd/comment-audit/main_test.go (1)

47-56: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Extract working directory setup into a helper.

This os.Getwd, os.Chdir, and t.Cleanup boilerplate is repeated across multiple tests in this file (here, lines 22-31, and lines 74-83) to safely supply a relative path. Extracting this into a small test helper (or using t.Chdir(dir) if this project uses Go 1.24+) would reduce duplication.

♻️ Proposed helper extraction
// Add this helper to the bottom of the file
func chdirTemp(t *testing.T) string {
	dir := t.TempDir()
	origDir, err := os.Getwd()
	if err != nil {
		t.Fatal(err)
	}
	if err := os.Chdir(dir); err != nil {
		t.Fatal(err)
	}
	t.Cleanup(func() {
		_ = os.Chdir(origDir)
	})
	return dir
}

Then replace the boilerplate in the tests:

-	dir := t.TempDir()
+	dir := chdirTemp(t)
 	if err := writeTestFile(filepath.Join(dir, "event.json"), `{"comment":{"body":"test suggestion","path":"cmd/agent/list_test.go"}}`); err != nil {
 		t.Fatal(err)
 	}
-	origDir, err := os.Getwd()
-	if err != nil {
-		t.Fatal(err)
-	}
-	if err := os.Chdir(dir); err != nil {
-		t.Fatal(err)
-	}
-	t.Cleanup(func() {
-		_ = os.Chdir(origDir)
-	})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/qualitygate/cmd/comment-audit/main_test.go` around lines 47 - 56,
Extract the repeated working-directory setup into a shared test helper, such as
chdirTemp, in main_test.go. Have it create and return a temporary directory,
change into it, and register cleanup restoring the original directory; replace
the duplicated os.Getwd, os.Chdir, and t.Cleanup blocks in the affected tests
while preserving their existing directory usage.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/qualitygate/publiccontent/rules.go`:
- Around line 385-388: Update credentialShapedValue to apply strings.TrimSpace
to value before removing boundary quotes and angle brackets with strings.Trim,
ensuring padded credential strings are normalized before
credentialShapedIdentifier is called.

---

Nitpick comments:
In `@internal/qualitygate/cmd/comment-audit/main_test.go`:
- Around line 47-56: Extract the repeated working-directory setup into a shared
test helper, such as chdirTemp, in main_test.go. Have it create and return a
temporary directory, change into it, and register cleanup restoring the original
directory; replace the duplicated os.Getwd, os.Chdir, and t.Cleanup blocks in
the affected tests while preserving their existing directory usage.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b0e07eb4-ca3a-41bd-aba5-bbce1af6a789

📥 Commits

Reviewing files that changed from the base of the PR and between f6a0c8d and ee7d781.

📒 Files selected for processing (10)
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/cmd/comment-audit/main_test.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/rules.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go
  • internal/qualitygate/rules/run_test.go
🚧 Files skipped from review as they are similar to previous changes (8)
  • internal/qualitygate/publiccontent/comment_audit.go
  • internal/qualitygate/rules/run_test.go
  • internal/qualitygate/publiccontent/comment_audit_test.go
  • internal/qualitygate/cmd/comment-audit/main.go
  • internal/qualitygate/publiccontent/credential.go
  • internal/qualitygate/publiccontent/collect_test.go
  • internal/qualitygate/publiccontent/scan.go
  • internal/qualitygate/publiccontent/scan_test.go

Comment thread internal/qualitygate/publiccontent/rules.go
@HanShaoshuai-k
HanShaoshuai-k force-pushed the fix/public-content-credential-fixtures branch from ee7d781 to 2ede076 Compare July 20, 2026 03:28
@HanShaoshuai-k
HanShaoshuai-k merged commit 21cff2e into main Jul 20, 2026
26 checks passed
@HanShaoshuai-k
HanShaoshuai-k deleted the fix/public-content-credential-fixtures branch July 20, 2026 05:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Large or sensitive change across domains or core paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants