Skip to content

dedup._norm() crashes with TypeError: normalize() argument 2 must be str, not None on a node with label: null — aborts the whole build #1194

Description

@freiit

Summary

A single node with label: null — which OpenAI-compatible LLM backends occasionally emit during semantic extraction — crashes the entire build in deduplicate_entities()_norm() with TypeError: normalize() argument 2 must be str, not None. No graph.json is written.

Worse for extract: the per-file semantic results are cached before the build/dedup step runs, so every subsequent extract re-loads the bad node from cache and re-crashes. There is no --no-dedup flag to work around it, so the corpus becomes un-buildable until the cache is wiped.

Version

  • Reproduced on graphifyy==0.8.31.
  • The code path is unchanged on the latest release v0.8.35 and on the default branch v8 (HEAD, checked today), so current installs are affected.

Traceback

File ".../graphify/__main__.py", line 4153, in main
    G = _build_merge(...)
File ".../graphify/build.py", line 360, in build_merge
    G = build(all_chunks, directed=directed, dedup=dedup, ...)
File ".../graphify/build.py", line 264, in build
    combined["nodes"], combined["edges"] = deduplicate_entities(...)
File ".../graphify/dedup.py", line 174, in deduplicate_entities
    key = _norm(node.get("label", node.get("id", "")))
File ".../graphify/dedup.py", line 20, in _norm
    label = unicodedata.normalize("NFKC", label)
TypeError: normalize() argument 2 must be str, not None

Root cause

_norm() assumes a str:

def _norm(label: str) -> str:
    label = unicodedata.normalize("NFKC", label)   # <-- None -> TypeError
    return re.sub(r"[\W_]+", " ", label.casefold(), flags=re.UNICODE).strip()

The callers pass node.get("label", node.get("id", "")). dict.get(key, default) returns the default only when the key is absent — for a node that literally has "label": None, it returns None, not the id/"" fallback. So one null label takes down the whole run. On v8 there are 5 such call sites: dedup.py lines 174, 204, 216, 226, 362.

This is the same class of bug as #454 (sanitize_label crashing when source_file is None).

Reproduction

Run graphify extract <docs> --backend <openai-compatible> --mode deep on a moderately large document corpus where the model sometimes returns an entity with "label": null. Hit reliably on a ~6,250-file Markdown corpus via a vLLM/gpt-oss-120b OpenAI-compatible backend.

Proposed fix

Guard _norm() against a non-str at the single chokepoint (covers all 5 call sites and _entropy, which also routes through _norm):

def _norm(label: str) -> str:
    """Lowercase + collapse non-alphanumeric runs to space (Unicode-aware)."""
    if not isinstance(label, str):
        label = "" if label is None else str(label)
    label = unicodedata.normalize("NFKC", label)
    return re.sub(r"[\W_]+", " ", label.casefold(), flags=re.UNICODE).strip()

A null/empty label then normalizes to "", which the existing if key: guards already skip — so the offending node is simply not considered for merging (it stays in the graph) instead of aborting the build.

Optionally, also make the callers prefer id over a null label: _norm(node.get("label") or node.get("id") or "").

Verified the one-line guard resolves the crash on 0.8.31. Happy to open a PR if that's useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions