Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
bca1a70
feat(scorecard): add `average` aggregation type
imykhno Apr 26, 2026
36f6193
fix(scorecard): average gauge out-of-range
imykhno Apr 27, 2026
11f5d25
fix(scorecard): default aggregation threshold value usage
imykhno Apr 27, 2026
bcd6186
fix(scorecard): sonarqube issues
imykhno Apr 27, 2026
bed8060
refactor(scorecard): update type definitions for tooltip and state ma…
imykhno Apr 27, 2026
fd15c6c
fix(scorecard): update tooltip translations and aggregation configura…
imykhno Apr 27, 2026
4e483aa
refactor(scorecard): rename `aggregationResultThresholds` to `thresho…
imykhno Apr 27, 2026
52f98a5
refactor(scorecard): consolidate AggregationConfig imports across ser…
imykhno Apr 27, 2026
b0d716c
Merge branch 'main' into scorecard-average-as-aggregation-type
imykhno Apr 27, 2026
a87361d
fix(scorecard): issues after merging main
imykhno Apr 27, 2026
b264e55
feat(scorecard): enhance aggregation KPI configuration with new types
imykhno Apr 27, 2026
f531700
Revert "feat(scorecard): enhance aggregation KPI configuration with n…
imykhno Apr 27, 2026
0e94a12
refactor(scorecard): update README and aggregation configuration for …
imykhno Apr 28, 2026
b9dae06
feat(scorecard): use the i18n for loading indicators and remove hardc…
imykhno Apr 28, 2026
da54b6d
refactor(scorecard): rename `aggregationKinds` to `aggregationTypes`
imykhno Apr 28, 2026
5a3f3e6
feat(scorecard): add detailed documentation for threshold rules in co…
imykhno Apr 28, 2026
ea941a5
feat(scorecard): enhance threshold configuration with aggregation rules
imykhno Apr 28, 2026
ba41570
fix(scorecard): average aggregation card percentage value
imykhno Apr 28, 2026
05cc855
refactor(scorecard): rename `ThresholdRuleAggregationConfig` to `Aggr…
imykhno Apr 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions workspaces/scorecard/.changeset/twenty-trains-fetch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@red-hat-developer-hub/backstage-plugin-scorecard-backend': minor
'@red-hat-developer-hub/backstage-plugin-scorecard-common': minor
'@red-hat-developer-hub/backstage-plugin-scorecard-node': minor
'@red-hat-developer-hub/backstage-plugin-scorecard': minor
---

Adds `**average**` as an aggregation KPI type alongside `**statusGrouped**`, with configurable `**options.statusScores**` and optional `**options.thresholds**` (same shape as metric thresholds) for homepage donut coloring against `**averageScore × 100**`, with built-in defaults when omitted.
10 changes: 10 additions & 0 deletions workspaces/scorecard/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,13 @@ yarn install
- **app-legacy:** Run `yarn start:legacy` to start the legacy frontend with the backend. Use the Scorecard tab on entity pages or the scorecard homepage card.

> **Notice:** The guest user has admin permissions in this application for quick setup. For better control, specify more users and groups in `app-config.local.yaml` and define a separate admin/admins permission instead of using the guest user. Using the guest user as admin is not recommended for permission management.

## Documentation

| Topic | Location |
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| Aggregation KPIs (`statusGrouped`, `average`), API, ownership | [plugins/scorecard-backend/docs/aggregation.md](plugins/scorecard-backend/docs/aggregation.md) |
| Backend installation and RBAC, **`scorecard.aggregationKPIs`** examples | [plugins/scorecard-backend/README.md](plugins/scorecard-backend/README.md) |
| Drill-down (entity list for a metric) | [plugins/scorecard-backend/docs/drill-down.md](plugins/scorecard-backend/docs/drill-down.md) |
| Metric thresholds, annotations, **average KPI result colors** | [plugins/scorecard-backend/docs/thresholds.md](plugins/scorecard-backend/docs/thresholds.md) |
| Frontend (homepage cards, NFS) | [plugins/scorecard/README.md](plugins/scorecard/README.md) |
21 changes: 21 additions & 0 deletions workspaces/scorecard/app-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,27 @@ scorecard:
type: statusGrouped
description: This KPI is provide information about GitHub open PRs grouped by status.
metricId: github.open_prs
openPrsWeightedKpi:
title: GitHub Open PRs (weighted health)
type: average
description: Weighted health average for open PRs by threshold status across your entities.
metricId: github.open_prs
options:
statusScores:
success: 100
warning: 40
error: 0
thresholds:
rules:
- key: success
expression: '>=80'
color: '#6bb300' # green
- key: warning
expression: '30-79'
color: 'rgb(224, 189, 108)' # light orange
- key: error
expression: '<30'
color: '#be1ec7' # purple
openIssuesKpi:
title: Jira Open Issues KPI
type: statusGrouped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export const AGGREGATED_CARDS_METRIC_IDS = {
withDefaultAggregation: 'github.open_prs',
withGithubOpenPrs: 'openPrsKpi',
withJiraOpenIssuesKpi: 'openIssuesKpi',
withOpenPrsWeightedKpi: 'openPrsWeightedKpi',
} as const;

export const AGGREGATED_CARDS_WIDGET_TITLES = {
Expand All @@ -27,4 +28,5 @@ export const AGGREGATED_CARDS_WIDGET_TITLES = {
withDefaultAggregation: 'Scorecard: With default aggregation config (GitHub)',
withGithubOpenPrs: 'Scorecard: GitHub open PRs',
withJiraOpenIssuesKpi: 'Scorecard: Jira open blocking tickets',
withOpenPrsWeightedKpi: 'Scorecard: GitHub open PRs (weighted health)',
} as const;
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ export const ScorecardRoutes = {
OPEN_ISSUES_KPI_METADATA_ROUTE:
'**/api/scorecard/aggregations/openIssuesKpi/metadata',
OPEN_PRS_KPI_AGGREGATION_ROUTE: '**/api/scorecard/aggregations/openPrsKpi',
OPEN_PRS_WEIGHTED_KPI_METADATA_ROUTE:
'**/api/scorecard/aggregations/openPrsWeightedKpi/metadata',
OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE:
'**/api/scorecard/aggregations/openPrsWeightedKpi',
OPEN_ISSUES_KPI_AGGREGATION_ROUTE:
'**/api/scorecard/aggregations/openIssuesKpi',
/** Default aggregation when aggregationId is the metric id (no KPI entry). */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
*/

import { Locator, Page, expect } from '@playwright/test';
import { AGGREGATED_CARDS_WIDGET_TITLES } from '../constants/homepageWidgetTitles';
import {
ScorecardMessages,
getEntitiesLabel,
Expand Down Expand Up @@ -62,6 +63,11 @@ export class HomePage {
cardPattern = /Scorecard:\s*GitHub open PRs|ScorecardGithubHomepage/i;
} else if (cardName === 'Scorecard: Jira open blocking') {
cardPattern = /Scorecard:\s*Jira open blocking|ScorecardJiraHomepage/i;
} else if (
cardName === AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi
) {
cardPattern =
/Scorecard:\s*GitHub open PRs \(weighted health\)|ScorecardOpenPrsWeightedKpi/i;
} else {
cardPattern = new RegExp(escapeRegex(cardName), 'i');
}
Expand Down Expand Up @@ -125,6 +131,21 @@ export class HomePage {
}

async clickDrillDownLink() {
await this.page.getByText(getEntitiesLabel(this.translations)).click();
// CardSubheader renders the count as a Link (e.g. "10 entities"). The card
// description can also contain the word "entities" (see API metadata), so
// getByText(entitiesLabel) is ambiguous. MUI Tooltip also sets the link’s
// accessible name to the long tooltip, so getByRole('link', { name }) is
// locale‑fragile. Match only links whose *visible* text is "{{count}} <label>".
const entitiesLabel = getEntitiesLabel(this.translations);
await this.page
.getByRole('link')
.filter({
hasText: new RegExp(
String.raw`^\d+\s*${escapeRegex(entitiesLabel)}$`,
'i',
),
})
.first()
.click();
}
}
181 changes: 181 additions & 0 deletions workspaces/scorecard/packages/app-legacy/e2e-tests/scorecard.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ import {
emptyGithubAggregatedResponse,
emptyJiraAggregatedResponse,
openPrsKpiMetadataResponse,
openPrsWeightedAggregatedResponse,
emptyOpenPrsWeightedAggregatedResponse,
openPrsWeightedKpiMetadataResponse,
openPrsWeightedUnsupportedAggregationResponse,
notAllowedAggregationErrorBody,
githubEntitiesDrillDownResponse,
jiraEntitiesDrillDownResponse,
Expand All @@ -58,6 +62,11 @@ import {
mockAllDefaultHomepageAggregationsSuccess,
mockHomepageAggregationsPermissionDenied,
} from './utils/mockHomepageAggregations';
import {
expectAverageCardCenterPercent,
verifyAverageDonutCenterTooltip,
verifyAverageLegendTooltipForStatus,
} from './utils/averageCardAssertions';
import { runAccessibilityTests } from './utils/accessibility';
import { ScorecardRoutes } from './constants/routes';
import {
Expand All @@ -83,6 +92,7 @@ async function addAggregatedScorecardWidgets(homePage: HomePage) {
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withDefaultAggregation);
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withGithubOpenPrs);
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withJiraOpenIssuesKpi);
await homePage.addCard(AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi);

await homePage.saveChanges();
}
Expand Down Expand Up @@ -965,5 +975,176 @@ test.describe('Scorecard Plugin Tests', () => {
);
});
});

test.describe('Average aggregation KPI (openPrsWeightedKpi)', () => {
test('Verify title and description from API metadata', async () => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
openPrsWeightedAggregatedResponse,
);

await addWidgets(
homePage,
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await page.reload();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other comments: why? Can we follow up ok this with a bug please?


const card = homePage.getCard(
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
);
await expect(card).toBeVisible();
await expect(card).toContainText(
openPrsWeightedKpiMetadataResponse.title,
);
await expect(card).toContainText(
openPrsWeightedKpiMetadataResponse.description,
);
});

test('Verify center score and average tooltips', async () => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
openPrsWeightedAggregatedResponse,
);

await addWidgets(
homePage,
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await page.reload();

const card = homePage.getCard(
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
);
await expectAverageCardCenterPercent(card, '50%');
await verifyAverageDonutCenterTooltip(
page,
card,
translations,
500,
1000,
);
await verifyAverageLegendTooltipForStatus(
page,
card,
translations,
currentLocale,
'success',
);
});

test('Verify empty aggregated response shows no data', async () => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
emptyOpenPrsWeightedAggregatedResponse,
);

await addWidgets(
homePage,
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await page.reload();

await homePage.expectCardHasNoDataFound(
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
);
});

test('Accessibility on weighted average card', async ({
browser: _browser,
}, testInfo) => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
openPrsWeightedAggregatedResponse,
);

await homePage.navigateToHome();
await homePage.enterEditMode();
await homePage.clearAllCards();
await homePage.addCard(
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await homePage.saveChanges();
await page.reload();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this page.reload needed? Can you please create a bug so that we follow-up. Users should see the right cards and latest data after they save the homepage.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will create a bug to follow and resolve all your comments related to e2e tests

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bug was created - link


await runAccessibilityTests(page, testInfo);
});

test('GitHub weighted KPI: drill-down, average card, and table', async () => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
openPrsWeightedAggregatedResponse,
);
await mockScorecardEntitiesDrillDownWithSort(
page,
githubEntitiesDrillDownResponse,
'github.open_prs',
);

await homePage.navigateToHome();
await page.reload();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed?

await homePage.enterEditMode();
await homePage.clearAllCards();
await homePage.addCard(
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await homePage.saveChanges();

await homePage.clickDrillDownLink();
await scorecardDrillDownPage.expectOnPage('github.open_prs', {
aggregationId: AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
});
await scorecardDrillDownPage.expectPageTitle(
'github.open_prs',
openPrsWeightedKpiMetadataResponse.title,
);

const drillCard = scorecardDrillDownPage.getDrillDownCard(
'github.open_prs',
{
aggregationId: AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
},
);
await expectAverageCardCenterPercent(drillCard, '50%');
await scorecardDrillDownPage.verifySomeEntitiesNotReportingTooltip();
await scorecardDrillDownPage.expectTableHeadersVisible();
await scorecardDrillDownPage.expectEntityNamesVisible([
'all-scorecards-service',
'Red Hat Developer Hub',
'github-scorecard-only-service',
'all-scorecards-service-different-owner',
'backend-api',
]);
});
});

test.describe('Unsupported aggregation type', () => {
test('Shows unsupported message when aggregationType is unknown', async () => {
await mockApiResponse(
page,
ScorecardRoutes.OPEN_PRS_WEIGHTED_KPI_AGGREGATION_ROUTE,
openPrsWeightedUnsupportedAggregationResponse,
);

await addWidgets(
homePage,
AGGREGATED_CARDS_WIDGET_TITLES.withOpenPrsWeightedKpi,
);
await page.reload();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same


const card = homePage.getCard(
AGGREGATED_CARDS_METRIC_IDS.withOpenPrsWeightedKpi,
);
await expect(card).toContainText(
translations.errors.unsupportedAggregationType,
);
await expect(card).toContainText('customUnknownAggregationKind');
});
});
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
import { Page, expect } from '@playwright/test';
import { ScorecardRoutes } from '../constants/routes';

/** Metric-id aggregation URLs (drill-down uses `github.open_prs` / `jira.open_issues`). */
const GITHUB_AGGREGATION_ROUTE =
ScorecardRoutes.GITHUB_OPEN_PRS_METRIC_AGGREGATION_ROUTE;
const JIRA_AGGREGATION_ROUTE =
Expand Down
Loading
Loading