`.graphifyignore` is bypassed for `graphify-out/converted/` sidecars produced from `*.xlsx` / `*.docx` source files

[adamrossmyers-coder]

Affected version

graphifyy 0.4.23 (Python 3.14.0 on Windows 11; reproducible regardless of OS based on code path).

Summary

When graphify.detect() encounters an office file (*.xlsx, *.docx, etc.), it converts the file to markdown in graphify-out/converted/<source>_<hash>.md and appends the converted path to the corpus file list. The converted path is added without being routed through _is_ignored(), so a user's .graphifyignore rule cannot suppress the converted sidecar even when the rule clearly targets it.

This makes patterns like graphify-out/, graphify-out/converted/, **/converted/*.md, etc. effectively ineffective for suppressing the auto-conversion output. The source *.xlsx file itself must be ignored instead.

Repro

Requires graphifyy[office] extras installed (otherwise the conversion branch isn't reached). The xlsx must contain at least one populated cell — an empty workbook produces empty text and is short-circuited at the if not text.strip(): return None check inside convert_office_file, never reaching the bug path.

mkdir /tmp/repro && cd /tmp/repro
pip install 'graphifyy[office]'
python -c "
import openpyxl
wb = openpyxl.Workbook()
ws = wb.active
ws['A1'] = 'biomarker'; ws['B1'] = 'value'
ws['A2'] = 'LDL';       ws['B2'] = 4.8
wb.save('test.xlsx')
"
cat > .graphifyignore <<'EOF'
graphify-out/
graphify-out/**
**/graphify-out/**
**/converted/*.md
EOF
python -c "
from graphify.detect import detect
from pathlib import Path
r = detect(Path('.'))
print('graphifyignore_patterns:', r.get('graphifyignore_patterns'))
print('total_files:', r.get('total_files'))
print('docs:', r['files']['document'])
"

Expected: graphifyignore_patterns: 4, total_files: 0, docs: [] — the only source file (test.xlsx) is matched by the office-conversion path, but the converted output sits inside graphify-out/converted/ which is ignored four different ways.

Actual (confirmed on graphifyy 0.4.23, Python 3.14.0, Windows 11, 2026-05-14): graphifyignore_patterns: 4, total_files: 1, docs: ['graphify-out\\converted\\test_<hash>.md'] — the converted sidecar appears in the corpus despite multiple matching ignore patterns.

Workaround confirmed: adding *.xlsx to .graphifyignore returns total_files: 0. The source *.xlsx is filtered before the conversion branch, so no sidecar is produced. The user has to know the leak originates at the source-file level — none of the four obvious "ignore the output directory" patterns work.

Root cause

graphify/detect.py, inside the main file loop of detect(). The ignore check fires against the source path p, then the office-conversion branch appends the converted path md_path straight into files[ftype]:

for p in all_files:
    ...
    if _is_ignored(p, root, ignore_patterns):
        continue
    ...
    ftype = classify_file(p)
    if ftype:
        if p.suffix.lower() in OFFICE_EXTENSIONS:
            md_path = convert_office_file(p, converted_dir)
            if md_path:
                files[ftype].append(str(md_path))    # ← never goes through _is_ignored
                total_words += count_words(md_path)
            ...

The source p may pass the ignore check (the user did not ignore test.xlsx), but the converted output lands in a directory the user clearly did ignore.

Suggested fix

Re-run _is_ignored against the converted path before appending, e.g.:

if md_path:
    if _is_ignored(md_path, root, ignore_patterns):
        # Source not ignored, but the converted output is — respect that.
        continue
    files[ftype].append(str(md_path))
    total_words += count_words(md_path)

Alternative: have _is_ignored evaluated against p test for the would-be md_path if p.suffix in OFFICE_EXTENSIONS and skip the source itself in that case. Equivalent behaviour, fewer paths to maintain.

Why this matters

Users who maintain a .graphifyignore to keep their corpus clean (typical in mature knowledge bases that have run graphify multiple times and have stale graphify-out/ artefacts in the source tree on Dropbox, OneDrive, etc.) will see "ignored" converted-xlsx outputs reappear on every run. There is no way to find this out from the docs or from the warnings — the output silently includes them, inflates token cost, and (in my medical-KB use case where I hit this) introduces duplicate biomarker nodes that compete with the canonical CSV-extracted entries.

The workaround is non-obvious: you have to discover that the leak originates in *.xlsx source files and exclude *.xlsx instead of (or in addition to) graphify-out/converted/.

Related observation (not the same bug)

graphify-out/cache/ does not appear to be persistent across runs in 0.4.23 — files cached in a prior run did not produce hits when the same files were re-extracted weeks later under the same root. May be a related code path worth a second look while in the area, but I have not independently confirmed the cache invariants. Filing separately if I can pin it down.

[safishamsi]

Hey
Can you rebase to 0.7.18 and test

[safishamsi]

Fixed in 972fdc1 — added _is_ignored() check before appending converted office/Google Workspace sidecars in detect.py.