Decouple Test Analytics from Coverage Requirement, with Optional Integration

[codecovdesign]

Problem to solve

Currently, Test Analytics (TA) requires code coverage to be configured. However, some users may not be interested in coverage reporting, which can slow down TA adoption. Decoupling these features while suggesting both can encourage growth and improve user onboarding speed.

Proposed Solution

• Make coverage configuration optional for TA and vice versa
• Allow TA to work independently without requiring coverage setup
• Ensure PR comments are displayed even when only TA is configured

Figma: https://www.figma.com/design/pCkCgq3vTuBxH8wUkuX8pd/GH-3079?node-id=1-2

[Adal3n3]

Figma: https://www.figma.com/design/pCkCgq3vTuBxH8wUkuX8pd/GH-3079?node-id=1-2

Two places to make updates:

1. PR comment

2. Doc: link

[rohan-at-sentry]

Some questions to consider before we make this truly dev-ready.

• Do we want this to respect comment: false? Currently, the ability for TA to comment on PRs can be turned off by turning off PR comments for Codecov via the Codecov YAML. Do we want to extend this to TA without coverage as well (I suspect the answer is yes, but we will have to test this)

• Do we want the UI to be any different if a repo has TA but not Coverage. Some things we may want to test

  • How do we want the repo list to look if a repo has TA but not Coverage
    

[codecovdesign]

How do we want the repo list to look if a repo has TA but not Coverage

This is a non-blocker, could we solve this problem in these issues: related discovery #2263 and TA specific: #2796. TLDR, where we landed with 2263 with to leave only coverage for now (since we have visibility of the feature on the repo itself, instead scope iteration show recently visited), and 2796 discovery if we want to add new features such as TA.

[rohan-at-sentry]

@codecovdesign @Adal3n3 how would we feel about a quick iteration where we have a column next to Test Coverage Called Test Analytics with text that says enabled vs not.

That said I don't feel too strongly on this being a blocker for this epic

[codecovdesign]

@rohan-at-sentry do you recall we did some exploration around this, see here >
in the spirit of iteration, can we move this to this issue: #2796?
regarding TA, IF it's is configured, what data point do you want to show? or it's basically binary configured/not-configured?