Skip to content

Flaky Test Detection Heuristic Improvements #3139

Description

@rohan-at-sentry

We will do the following improvements

Better flake expiry (1 SP)

  • Calculate and use the “flake percentage” (how often does this test fail on main) to determine when to expire a flake. A 100% flake would be expired after 10 passes
    • a 1% flake would be expired after 1000 passes

Error Message Parsing (3 SP)

  • Use the work done for framework detection in [TA] Error message parsing #1487 to improve our flake normalization system
  • Alternatively find and implement other approaches

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions