_is_ignored applies a nested .gitignore to the entire scan tree, silently dropping files (detect() returns 0 on a healthy repo)

[nadiadatepe-eng]

Summary

_is_ignored matches a non-anchored pattern against the scan root instead of against the directory the pattern was declared in. The pattern's own anchor is only tried as a fallback — it is never used to limit scope.

Git's rule is the opposite: a pattern in <dir>/.gitignore governs only paths under <dir>, matched relative to <dir>.

So one project's .gitignore silently governs every sibling project in the scan. Because the drop site is a bare continue, the lost files appear in no diagnostic field — not walk_errors, not unclassified, not skipped_sensitive. detect() reports success and the graph is quietly incomplete.

Reproduction

repro/
├── run.py
├── project_a/
│   └── data/loader.py        # unrelated project — must be indexed
└── project_b/
    ├── .gitignore            # contains exactly: data/
    └── data/dump.csv         # this is what the rule is for

from pathlib import Path
from graphify.detect import detect
r = detect(Path(__file__).parent.resolve())
print(r['total_files'], r['files']['code'])
print(r['skipped_sensitive'], r['unclassified'], r['walk_errors'])

Expected: project_a/data/loader.py is indexed. project_b/.gitignore cannot govern project_a.

Actual (0.9.15):

total_files      : 1
code files       : ['run.py']
skipped_sensitive: []
unclassified     : ['project_b/.gitignore']
walk_errors      : []

project_a/data/loader.py is gone and is reported nowhere.

Git on the identical tree disagrees, as it should:

$ git check-ignore -v project_a/data/loader.py
(no output — not ignored)
$ git check-ignore -v project_b/data/dump.csv
project_b/.gitignore:1:data/    project_b/data/dump.csv

Root cause

detect.py:965-976 in _is_ignored._eval:

else:
    try:
        rel = str(target.relative_to(root)).replace(os.sep, "/")   # <-- root, not anchor
        matched = _matches(rel, p, anchored=False)
    except ValueError:
        pass
    if not matched and anchor != root:                              # <-- anchor only a fallback
        try:
            rel_anchor = str(target.relative_to(anchor)).replace(os.sep, "/")
            matched = _matches(rel_anchor, p, anchored=False)
        except ValueError:
            pass

The first branch makes every non-anchored pattern global to the scan. _matches then walks path components, so a bare data/ matches the data component anywhere. Combined with the ancestor-exclusion loop at detect.py:993-997, matching a directory component removes the entire subtree.

Severity scales with the pattern:

• A pattern like data/ or output/ (declared in one subproject) silently deletes that category tree-wide. Nothing goes to zero, so it is invisible.
• A bare * — which is what a venv, a .hypothesis dir, or any number of tool caches ship in their auto-generated .gitignore — makes fnmatch(<anything>, "*") true for every file, and detect() returns total_files: 0 for a large, healthy repo.

This is amplified by ordering: ignore_patterns grows during the walk (detect.py:1219), while the classification loop (detect.py:1258) runs afterwards against the fully-grown list. So probing _is_ignored(f, root, _load_graphifyignore(root)) by hand returns False (kept) while the real loop drops the same file — the standalone probe and the production path evaluate different pattern sets, which makes this very hard to diagnose.

Suggested fix

Scope every pattern to its anchor, which also unifies the anchored/non-anchored branches:

try:
    rel_anchor = str(target.relative_to(anchor)).replace(os.sep, "/")
except ValueError:
    continue                      # pattern's directory does not contain target -> cannot match
if _matches(rel_anchor, p, anchored=anchored):
    result = not negated

Verified against a ~800-file tree: restores it from total_files: 0 to full, with no regression on trees that were already correct.

Related, in the same area

1. detect.py:1265 is a silent continue. Every other drop path records itself (skipped_sensitive, unclassified, walk_errors), so an ignored file is the only way a scan can lose data with no trace. Appending to an ignored: [] in the return dict would have made this bug self-evident instead of undiagnosable. Given detect() already goes to some length to surface partial enumeration (the _on_walk_error comment at detect.py:1181-1187 makes exactly this argument), this looks like an oversight rather than a decision.
2. _is_included (detect.py:1063-1069) has the same root-relative shape and presumably the same bug.
3. include_patterns is loaded at detect.py:1170 and never used — _load_graphifyinclude's result is assigned and then dropped, so .graphifyinclude appears to be a no-op in detect().

Happy to open a PR for the anchor-scoping fix and the ignored list if useful.

Version: graphifyy 0.9.15, Python 3.12, Linux.

[safishamsi]

The nested-ignore scoping bug itself was fixed in 0.9.16 (#1873) — a nested .gitignore is now matched relative to its declaring directory, so a sibling project is no longer dropped. The remaining silent-drop gap this issue raised is addressed on v8 in cb96bda (ships in 0.9.17): detect() now records ignored files/dirs in a new "ignored" diagnostic field (per-directory where a subtree is pruned). Regression test added.