Codecov can read from multiple YAML's in a single repo

[rohan-at-sentry]

Background

This proposal attempts to fix two problems with the current codecov.yml that are acutely felt by our largest customers:

1. Customers looking to set up components on large-scale monorepos can sometimes find manipulating a single repo level settings file (the Codecov YAML) cumbersome.
   1. Often in large scale monorepos, centralized configuration — like a single codecov.yml — leads to enormous (e.g., 10,000+ lines) configuration files that are cumbersome to update and prone to errors when doing so.
   2. Many developers in monorepos don’t have a full understanding of individual projects, how they work, nor how they’re configured. Therefore, engineering leaders tend to want to decentralize configuration so configuration can be more easily edited on a “per project” basis within the monorepo. This keeps individual configurations smaller, and prevents unintentional “global” configuration changes from occurring.
2. Components are a way to declare how to split up reporting of code coverage by “area” - and in a large-scale monorepo, customers are telling us they see of “areas” as “ownership areas”. Components become strategic to such customers as (when they work well), they allow a centralized way to see what teams are doing well against a global code coverage goal.
   1. However, defining components in a single codecov.yml leads to enormous configuration files that are difficult to reason about and update.

Proposed Solutions

Codecov can read from multiple YAMLs in a single repo

This directly attacks the “Separation of Concerns” problem stated above. By allowing customers to define a separate YAML file such as

component-definition:
    back-end :
      component_id : back-end
        paths: 
                - #some-path-to-file.java
                - #some glob
     front-end :
      component_id : back-end
         paths: 
                 - #some-path-to-file.js
                 -  #some glob
      mobile :
       component_id : mobile
           paths:
                  - #some-path-to-file.apk
                  -  #some glob

the “main” codecov.yaml can look very manage-able

comment:
layout: "header, diff, flags, components"  # show component info in the PR comment
component_management: 
	default_rules: 
		statuses:
      - type: project # in this case every component that doens't have a status defined will have a project type one
        target: auto
        branches:
          - "!main"
  individual_components : auto # a new keyword that instructs the CLI to search for component-definition files to then process  

The component-definition.yml can be located at different parts of the repo (see example) allowing for developers to “own” Codecov settings within their field of file/code ownership on the monorepo.

|-- front-end/
|  	|
|   |-- tests/
|   |-- main.js
|   |-- component-definition.yml
|
|-- back-end/
|   |
|   |-- tests/
|   |-- __init__.py
|   |-- ascii.py
|   |-- emoji.py
|   |-- component-definition.yml
|
|-- mobile/
    |
    |-- tests/
    |-- __init__.py
    |-- ascii.py
    |-- emoji.py
    |   |-- component-definition.yml

Risks

• The onus now sits with the individual team to set the component-definition.yml file instead of all this happening centrally - leading to less pain for the admin (or manager) but more friction on rollout.

Technical notes

[Analysis - Proposal 1](https://www.notion.so/Analysis-Proposal-1-1538b10e4b5d80efb8f1ce513e7efcc8?pvs=21)