feat(scorecard): add code-coverage module - #3476
Open
fullsend-ai-coder[bot] wants to merge 7 commits into
Open
Conversation
fullsend-ai-coder
Bot
requested review from
a team,
Eswaraiahsapram,
christoph-jerolimov and
its-mitesh-kumar
as code owners
June 19, 2026 13:27
Missing ChangesetsThe following package(s) are changed by this PR but do not have a changeset:
See CONTRIBUTING.md for more information about how to add changesets. Changed Packages
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3476 +/- ##
==========================================
+ Coverage 57.93% 57.95% +0.02%
==========================================
Files 2396 2401 +5
Lines 96104 96175 +71
Branches 26803 26801 -2
==========================================
+ Hits 55675 55737 +62
- Misses 40233 40242 +9
Partials 196 196
*This pull request uses carry forward flags. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
|
🤖 Finished Review · ✅ Success · Started 1:30 PM UTC · Completed 1:42 PM UTC |
ReviewFindingsHigh
Medium
Low
Info
Previous runReviewFindingsMedium
Low
Info
|
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
| if (!response.ok) { | ||
| throw new Error( | ||
| `Code coverage API error: ${response.status} ${response.statusText} for ${url}`, | ||
| ); |
There was a problem hiding this comment.
[medium] Missing service-to-service authentication
The client uses bare fetch() without Backstage service-to-service authentication tokens. The code-coverage API is an internal Backstage plugin discovered via DiscoveryService.
Suggested fix: Inject AuthService (from coreServices.auth) into the client. Before each fetch, call auth.getPluginRequestToken() and pass the token as a Bearer header.
| type: this.getMetricType(), | ||
| history: true, | ||
| }; | ||
| } |
There was a problem hiding this comment.
[low] edge-case
The calculateMetric method accesses report.aggregate[mapping.section][mapping.field] without null checks.
| }); | ||
| }); | ||
|
|
||
| describe('getCatalogFilter', () => { |
There was a problem hiding this comment.
[low] missing-test
No test case for calculateMetric when the API returns a report with missing or null aggregate section data.
| it('should call the correct URL and return the report', async () => { | ||
| jest.spyOn(global, 'fetch').mockResolvedValueOnce({ | ||
| ok: true, | ||
| json: async () => sampleReport, |
There was a problem hiding this comment.
[low] test-inadequate
The error-case test does not verify the URL is included in the error message due to substring matching.
| * Creates a single code-coverage metric provider for the given metric ID. | ||
| */ | ||
| export function createCodeCoverageMetricProvider( | ||
| discovery: DiscoveryService, |
There was a problem hiding this comment.
[info] logic-error
Creates a new CodeCoverageClient instance for each of the 8 metric providers, causing 8x API calls per entity.
| if (!response.ok) { | ||
| throw new Error( | ||
| `Code coverage API error: ${response.status} ${response.statusText} for ${url}`, | ||
| ); |
There was a problem hiding this comment.
[info] data-exposure
Error message includes the full internal URL. Consistent with existing patterns.
| ); | ||
| } | ||
| return response.json() as Promise<CodeCoverageReport>; | ||
| } |
There was a problem hiding this comment.
[info] No response body validation
JSON response cast directly to CodeCoverageReport without runtime validation. Consistent with other modules.
fullsend-ai-review
Bot
added
the
requires-manual-review
Member
|
/fs-fix regenerate the api reports and commit the changes (or new files) |
Contributor
Author
|
🤖 Finished Fix · ❌ Failure · Started 6:42 AM UTC · Completed 6:42 AM UTC |
Member
|
/fs-fix
|
Contributor
Author
|
🤖 Finished Fix · ❌ Failure · Started 8:17 AM UTC · Completed 8:17 AM UTC |
Member
|
/fs-fix |
Contributor
Author
|
🤖 Fix · ❌ Terminated · Started 4:19 PM UTC · Ended 4:20 PM UTC |
|
🤖 Review · ❌ Terminated · Started 4:53 PM UTC · Ended 5:08 PM UTC |
CI Feedback 🧐A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
|
|
|
||
| this.logger.debug(`Fetching code coverage report for entity ${entityRef}`); | ||
|
|
||
| const response = await fetch(url); |
There was a problem hiding this comment.
[high] Missing backend-to-backend authentication
The fetch call to the code-coverage backend plugin does not include a Backstage service-to-service authentication token. In the Backstage new backend system, backend plugins enforce authentication by default. The Jira scorecard module correctly uses coreServices.auth to obtain a getPluginRequestToken and sends it as a Bearer token. This module omits authentication entirely and will fail at runtime in any deployment with default auth settings.
Suggested fix: Add coreServices.auth to the module's deps in module.ts. Pass the AuthService into CodeCoverageClient. In getReport(), call auth.getOwnServiceCredentials() then auth.getPluginRequestToken({ onBehalfOf: ownCredentials, targetPluginId: 'code-coverage' }) and include the resulting token as Authorization: Bearer *** in the fetch headers.
| ): MetricProvider<'number'> { | ||
| const client = new CodeCoverageClient(discovery, logger); | ||
| return new CodeCoverageMetricProvider(client, metricId); | ||
| } |
There was a problem hiding this comment.
[medium] resource-duplication
The createCodeCoverageMetricProvider function creates a new CodeCoverageClient instance for every metric. 8 separate client objects are instantiated when 1 would suffice. All 8 metrics extract data from the same API call for a given entity.
Suggested fix: Create a single CodeCoverageClient instance in createCodeCoverageMetricProviders and pass it to all 8 CodeCoverageMetricProvider instances.
| }; | ||
| } | ||
|
|
||
| async calculateMetric(entity: Entity): Promise<number> { |
There was a problem hiding this comment.
[medium] missing-null-handling
In calculateMetric, the code accesses report.aggregate[mapping.section][mapping.field] without null/undefined checks. The runtime data comes from an external API whose response shape is not validated. If the API does not have branch coverage data, aggregate.branch could be undefined, causing a TypeError.
Suggested fix: Add a defensive check before accessing the nested property.
| if (!response.ok) { | ||
| throw new Error( | ||
| `Code coverage API error: ${response.status} ${response.statusText} for ${url}`, | ||
| ); |
There was a problem hiding this comment.
[info] unvalidated-response
The JSON response is cast to CodeCoverageReport without runtime validation. Consistent with other modules in the workspace.
fullsend-ai-review
Bot
removed
the
requires-manual-review
|
🤖 Finished Review · ✅ Success · Started 4:53 PM UTC · Completed 5:08 PM UTC |
Contributor
|
This PR has been automatically marked as stale because it has not had recent activity from the author. It will be closed if no further activity occurs. If the PR was closed and you want it re-opened, let us know and we'll re-open the PR so that you can continue the contribution! |
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 27, 2026
- Add Backstage service-to-service authentication to CodeCoverageClient using AuthService (getOwnServiceCredentials + getPluginRequestToken) with Bearer token in fetch headers - Move CODE_COVERAGE_METRICS, CODE_COVERAGE_METRIC_CONFIG, PERCENTAGE_THRESHOLDS and related constants from CodeCoverageConfig into CodeCoverageMetricProvider - Update metric IDs from snake_case to lowerCamelCase per repo convention (e.g. codeCoverage.linePercentage instead of code-coverage.line_percentage) - Use 'codeCoverage' for getProviderDatasourceId - Share single CodeCoverageClient instance across all 8 metric providers instead of creating 8 separate instances - Add null/undefined checks for report.aggregate section data - Add test for missing aggregate section data - Add test verifying auth token request targets code-coverage plugin Addresses review feedback on #3476
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 27, 2026
- Set package version to 0.0.0 for new code-coverage module - Align README metric IDs with implementation (lowerCamelCase) - Add generated API report file (report.api.md) Addresses review feedback on #3476
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 27, 2026
Ran yarn dedupe to deduplicate the scorecard workspace lockfile as requested. Version in package.json was already 0.0.0. Addresses review feedback on #3476
fullsend-ai-coder
Bot
force-pushed
the
agent/3474-code-coverage-module
branch
from
July 27, 2026 21:48
1f99af1 to
e97ef41
Compare
Member
|
/fs-fix rebase |
Contributor
Author
|
🤖 Finished Fix · ✅ Success · Started 10:07 AM UTC · Completed 10:27 AM UTC |
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 28, 2026
Rebase onto main and update the code-coverage module to match the current MetricProvider interface: - Replace getMetric() with getMetrics() returning Metric[] with thresholds - Replace calculateMetric() with calculateMetrics() returning Map - Remove getMetricType() and getMetricThresholds() (no longer in interface) - Update all tests to use the new API methods Addresses review feedback on #3476
Contributor
Author
🔧 Fix agent — iteration 5 (human-triggered)Rebased branch onto latest main, resolved yarn.lock conflict, verified all 28 tests pass. Fixed (1):
Tests: passed Updated by fullsend fix agent |
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 28, 2026
- Add Backstage service-to-service authentication to CodeCoverageClient using AuthService (getOwnServiceCredentials + getPluginRequestToken) with Bearer token in fetch headers - Move CODE_COVERAGE_METRICS, CODE_COVERAGE_METRIC_CONFIG, PERCENTAGE_THRESHOLDS and related constants from CodeCoverageConfig into CodeCoverageMetricProvider - Update metric IDs from snake_case to lowerCamelCase per repo convention (e.g. codeCoverage.linePercentage instead of code-coverage.line_percentage) - Use 'codeCoverage' for getProviderDatasourceId - Share single CodeCoverageClient instance across all 8 metric providers instead of creating 8 separate instances - Add null/undefined checks for report.aggregate section data - Add test for missing aggregate section data - Add test verifying auth token request targets code-coverage plugin Addresses review feedback on #3476
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 28, 2026
- Set package version to 0.0.0 for new code-coverage module - Align README metric IDs with implementation (lowerCamelCase) - Add generated API report file (report.api.md) Addresses review feedback on #3476
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 28, 2026
Ran yarn dedupe to deduplicate the scorecard workspace lockfile as requested. Version in package.json was already 0.0.0. Addresses review feedback on #3476
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 28, 2026
Regenerate yarn.lock to resolve conflicts from rebasing onto origin/main. All 28 code-coverage module tests pass. Addresses review feedback on #3476
fullsend-ai-coder
Bot
force-pushed
the
agent/3474-code-coverage-module
branch
from
July 28, 2026 10:26
e97ef41 to
7f62cfa
Compare
Member
|
/fs-fix run yarn dedupe and commit the yarn.lock |
1 similar comment
Member
|
/fs-fix run yarn dedupe and commit the yarn.lock |
Contributor
Author
|
🤖 Finished Fix · ✅ Success · Started 6:38 PM UTC · Completed 6:51 PM UTC |
Contributor
Author
🔧 Fix agent — iteration 6 (human-triggered)Ran yarn dedupe in the scorecard workspace as requested. The yarn.lock was significantly reduced (714 lines removed) by deduplicating transitive dependencies. Fixed (1):
Tests: passed Updated by fullsend fix agent |
fullsend-ai-coder Bot
added a commit
that referenced
this pull request
Jul 29, 2026
Run yarn dedupe in the scorecard workspace to deduplicate transitive dependencies in yarn.lock. Addresses review feedback on #3476
Member
|
/fs-fix rebase |
Contributor
Author
|
🤖 Finished Fix · ✅ Success · Started 9:24 PM UTC · Completed 9:38 PM UTC |
Create a new scorecard-backend-module-code-coverage that integrates with the Backstage Community code-coverage plugin to provide 8 metrics: line/branch percentage, available, covered, and missed counts. The module fetches data from the code-coverage-backend API using the Backstage discovery service and maps the aggregate line and branch fields to individual MetricProviders. Entities are filtered by the backstage.io/code-coverage annotation. Percentage metrics include default thresholds (>80 success, 50-80 warning, <50 error). Includes: - CodeCoverageClient for API communication - Type definitions for the code-coverage API response - 8 MetricProvider implementations via factory pattern - Unit tests for client, providers, and factory (28 tests) - Example entity with code-coverage annotation - Backend app integration Closes #3474
Rebase onto main and update the code-coverage module to match the current MetricProvider interface: - Replace getMetric() with getMetrics() returning Metric[] with thresholds - Replace calculateMetric() with calculateMetrics() returning Map - Remove getMetricType() and getMetricThresholds() (no longer in interface) - Update all tests to use the new API methods Addresses review feedback on #3476
- Add Backstage service-to-service authentication to CodeCoverageClient using AuthService (getOwnServiceCredentials + getPluginRequestToken) with Bearer token in fetch headers - Move CODE_COVERAGE_METRICS, CODE_COVERAGE_METRIC_CONFIG, PERCENTAGE_THRESHOLDS and related constants from CodeCoverageConfig into CodeCoverageMetricProvider - Update metric IDs from snake_case to lowerCamelCase per repo convention (e.g. codeCoverage.linePercentage instead of code-coverage.line_percentage) - Use 'codeCoverage' for getProviderDatasourceId - Share single CodeCoverageClient instance across all 8 metric providers instead of creating 8 separate instances - Add null/undefined checks for report.aggregate section data - Add test for missing aggregate section data - Add test verifying auth token request targets code-coverage plugin Addresses review feedback on #3476
- Set package version to 0.0.0 for new code-coverage module - Align README metric IDs with implementation (lowerCamelCase) - Add generated API report file (report.api.md) Addresses review feedback on #3476
Ran yarn dedupe to deduplicate the scorecard workspace lockfile as requested. Version in package.json was already 0.0.0. Addresses review feedback on #3476
Regenerate yarn.lock to resolve conflicts from rebasing onto origin/main. All 28 code-coverage module tests pass. Addresses review feedback on #3476
Run yarn dedupe in the scorecard workspace to deduplicate transitive dependencies in yarn.lock. Addresses review feedback on #3476
Contributor
Author
🔧 Fix agent — iteration 7 (human-triggered)Rebased PR branch onto latest origin/main. All 7 commits applied cleanly. Verified: build passes, 28 tests pass, secret scan clean, lockfile dedupe not needed. No new commit created — the rebase rewrote existing commits onto the new base. Fixed (1):
Tests: passed Updated by fullsend fix agent |
fullsend-ai-coder
Bot
force-pushed
the
agent/3474-code-coverage-module
branch
from
July 29, 2026 21:38
bbda591 to
0e792d2
Compare
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Create a new scorecard-backend-module-code-coverage that integrates with the Backstage Community code-coverage plugin to provide 8 metrics: line/branch percentage, available, covered, and missed counts.
The module fetches data from the code-coverage-backend API using the Backstage discovery service and maps the aggregate line and branch fields to individual MetricProviders.
Entities are filtered by the backstage.io/code-coverage annotation. Percentage metrics include default thresholds (>80 success, 50-80 warning, <50 error).
Includes:
Closes #3474
Post-script verification
agent/3474-code-coverage-module)7ccaff17753df64c7ab288cdcba34cee5a657254..HEAD)