Skip to content

Fuzzy dedup falsely merges similar-but-distinct entities (chip models, codenames with shared prefixes) #878

Description

@myf-cpu

Fuzzy dedup uses Jaro-Winkler string similarity (_MERGE_THRESHOLD = 92.0) and
incorrectly merges entities that are similar but semantically distinct.

Example cases:

Label A Label B Jaro-Winkler Result
───────── ───────── ───────────── ─────────────────
ASR1603 ASR1605 ~95.2 falsely merged
ASR1603 ASR1606 ~95.2 falsely merged
cranel cranelr ~96.3 falsely merged
cranel cranelrh ~95.2 falsely merged
cranelr cranelrh ~96.3 falsely merged

These are different chip name / hardware variants, not typos or formatting
variants. Merging them causes:

Nodes disappear from the graph (only the winner survives)
All edges are rewired to the survivor, creating false relationships
The merged entity appears connected to everything all variants were connected to
Do you have any suggestions regarding this issue?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions