Adding fix-flaky-go-test skill - #22010
Conversation
|
👋 kalverra, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
kalverra
changed the title
|
✅ No conflicts with other open PRs targeting |
There was a problem hiding this comment.
Pull request overview
Risk Rating: LOW — documentation-only change, but it directly guides automated workflows so correctness matters.
Adds a Chainlink-specific fix-flaky-go-test agent skill with repo-specific reproduction and fixing guidance for intermittent Go test failures.
Changes:
- Introduces a new
.agents/skills/fix-flaky-go-test/SKILL.mddescribing setup, reproduction steps, common root causes, and fix patterns. - Aligns the guidance with Chainlink CI tooling (
tools/bin/*) and database setup (.dbenv,make testdb). - Documents optional CI parity knobs (e.g.,
GODEBUG=goindex=0) and race/stress testing workflows.
| Append --tags integration to every go test below if the file has //go:build integration. | ||
| deployment/ CCIP: use tools/bin/go_core_ccip_deployment_tests pattern (cd deployment, CL_RESERVE_PORTS=128). | ||
| Optional CI parity: GODEBUG=goindex=0 on go test (see ci-core.yml). | ||
| If the file uses //go:build dev or trace, add matching --tags when reproducing. |
There was a problem hiding this comment.
go test uses the -tags flag (e.g., -tags=integration), not --tags. As written, --tags integration will be rejected by go test and would break the reproduction instructions (and any agent automation relying on them).
| Append --tags integration to every go test below if the file has //go:build integration. | |
| deployment/ CCIP: use tools/bin/go_core_ccip_deployment_tests pattern (cd deployment, CL_RESERVE_PORTS=128). | |
| Optional CI parity: GODEBUG=goindex=0 on go test (see ci-core.yml). | |
| If the file uses //go:build dev or trace, add matching --tags when reproducing. | |
| Append -tags integration to every go test below if the file has //go:build integration. | |
| deployment/ CCIP: use tools/bin/go_core_ccip_deployment_tests pattern (cd deployment, CL_RESERVE_PORTS=128). | |
| Optional CI parity: GODEBUG=goindex=0 on go test (see ci-core.yml). | |
| If the file uses //go:build dev or trace, add matching -tags when reproducing. |
| <verify> | ||
| Write the exact repro go test line including -run and --tags integration when relevant. | ||
| Race: GORACE log_path, go test -race -shuffle on, confirm no race.* or document skip. |
There was a problem hiding this comment.
This section again references --tags integration, but go test expects -tags (e.g., -tags=integration). Please update the example wording so users copy/pasting the repro line don’t get an "unknown flag: --tags" error.
sebawo
changed the title
CORA - Analysis SkippedReason: The number of code owners (2) is less than the minimum required (5) and/or the number of CODEOWNERS entries with changed files (1) is less than the minimum required (2). |
ba64942 to
526e5e5
Compare
Documents how to: - extract the Trunk test case link from Jira tickets via atlassian MCP - fetch failure history and rate from the Trunk test page (Scrapling) - call the Trunk fix-flaky-test MCP tool with the correct fix/investigation ID (distinct from the test-case UUID in the page URL) - distinguish "investigation not found" from a real result Also adds Docker/Solana sync.Once root cause pattern from CCIP-10960. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Made-with: Cursor
526e5e5 to
f5d27be
Compare
|
sebawo
changed the title
|
@kalverra what is required to merge this change? |
|
Closed in favor of the superior successor: #22125 |
This pull request adds comprehensive documentation for the
fix-flaky-go-testskill, which provides step-by-step guidance and best practices for identifying, reproducing, and fixing flaky Go tests in the Chainlink codebase. The documentation covers how to leverage Trunk.io for flake tracking and AI-assisted fixes, environment setup, classification of test types, workflow for reproducing and fixing flakes, common root causes, and recommended fix patterns.Key documentation and process improvements:
Trunk.io integration and flake tracking:
Test environment setup and classification:
integration,dev, ortrace.Reproduction and debugging workflow:
-count,-shuffle,-race, etc.), and advice for narrowing down failure cases.Root cause analysis and fix patterns: