Bug
_stamped_manifest_files() (added in #1897 to fix zero-node docs being stamped incorrectly) decides whether a semantic document "produced output" by checking only the nodes and edges collections of sem_result. It never checks hyperedges.
sem_extracted: set[Path] = set()
for coll in ("nodes", "edges"):
for item in sem_result.get(coll, []):
sf = item.get("source_file", "")
if sf:
sem_extracted.add(_resolve(sf))
If a chunk's only output for a given document is a hyperedge (a coherent group of 3+ existing nodes/concepts spanning that doc — the extraction skill prompt explicitly supports this: "if 3 or more nodes clearly participate together in a shared concept... add a hyperedge"), that document is left out of sem_extracted.
This means:
_response_is_hollow() in llm.py correctly treats hyperedges as valid output (not nodes and not edges and not hyperedges), so the chunk is NOT flagged as a failure/truncation.
- But
_stamped_manifest_files() disagrees — it excludes the doc from the manifest anyway.
semantic_hash never gets stamped for that file.
detect_incremental() re-queues the file as "changed" on every subsequent graphify extract run, even though it was successfully extracted.
Observed with
- graphifyy 0.9.16/0.9.17, code reading (not yet reproduced against a live LLM backend — filing from source-level analysis, repro below is a direct unit-level trigger)
Reproduction
from graphify.cli import _stamped_manifest_files
files_by_type = {"document": [str(doc_path)]}
sem_result = {
"nodes": [],
"edges": [],
"hyperedges": [{"id": "h1", "label": "...", "nodes": ["a", "b", "c"], "source_file": "doc.md"}],
}
out = _stamped_manifest_files(files_by_type, sem_result, root)
# out["document"] == [] — doc.md is dropped even though it produced a hyperedge
Expected behavior
A document whose only extraction output is a hyperedge should be treated the same as one with a node or edge: included in sem_extracted, stamped in the manifest, and not re-queued on the next incremental run.
Suggested fix
Add "hyperedges" to the iterated collections in _stamped_manifest_files, matching _response_is_hollow's definition of "produced output":
for coll in ("nodes", "edges", "hyperedges"):
for item in sem_result.get(coll, []):
...
Impact
Documents whose semantic value is purely relational (a synthesis/overview doc describing how several existing entities relate) never get a stable semantic_hash, so they're silently re-sent to the LLM backend on every graphify extract run — wasted tokens/cost with no visible symptom other than persistently non-zero "changed files" counts.
Bug
_stamped_manifest_files()(added in #1897 to fix zero-node docs being stamped incorrectly) decides whether a semantic document "produced output" by checking only thenodesandedgescollections ofsem_result. It never checkshyperedges.If a chunk's only output for a given document is a hyperedge (a coherent group of 3+ existing nodes/concepts spanning that doc — the extraction skill prompt explicitly supports this: "if 3 or more nodes clearly participate together in a shared concept... add a hyperedge"), that document is left out of
sem_extracted.This means:
_response_is_hollow()inllm.pycorrectly treatshyperedgesas valid output (not nodes and not edges and not hyperedges), so the chunk is NOT flagged as a failure/truncation._stamped_manifest_files()disagrees — it excludes the doc from the manifest anyway.semantic_hashnever gets stamped for that file.detect_incremental()re-queues the file as "changed" on every subsequentgraphify extractrun, even though it was successfully extracted.Observed with
Reproduction
Expected behavior
A document whose only extraction output is a hyperedge should be treated the same as one with a node or edge: included in
sem_extracted, stamped in the manifest, and not re-queued on the next incremental run.Suggested fix
Add
"hyperedges"to the iterated collections in_stamped_manifest_files, matching_response_is_hollow's definition of "produced output":Impact
Documents whose semantic value is purely relational (a synthesis/overview doc describing how several existing entities relate) never get a stable
semantic_hash, so they're silently re-sent to the LLM backend on everygraphify extractrun — wasted tokens/cost with no visible symptom other than persistently non-zero "changed files" counts.