docs: update embedding benchmarks (3.3.0)

[github-actions]

Automated embedding benchmark update for 3.3.0 from workflow run #381.

[greptile-apps]

Greptile Summary

This is an automated documentation PR that updates generated/benchmarks/EMBEDDING-BENCHMARKS.md with embedding benchmark results for version 3.3.0. However, the committed data is empty: symbols: 0 and "models": {}, with a completely blank results table — indicating the source workflow run (#381) did not successfully execute any benchmarks.

Key concerns:

• The 3.3.0 entry records "symbols": 0 and "models": {} — no symbols were indexed and no model benchmarks were run. The previous entry (3.1.4) had 1,095 symbols with full model data.
• The "Latest results" markdown table renders with no rows, and the **Symbols:** 0 headline is misleading.
• Merging overwrites a valid baseline with an empty entry as the authoritative latest result, degrading the value of the benchmark history.

Confidence Score: 1/5

• Not safe to merge — the benchmark data is empty (0 symbols, no models), indicating a failed workflow run was committed as a valid result.
• The sole change introduces a benchmark entry with no actual data (symbols: 0, models: {}), which corrupts the "latest results" display and replaces a meaningful 1,095-symbol baseline with an empty record. This is a documentation regression, not a valid benchmark update.
• generated/benchmarks/EMBEDDING-BENCHMARKS.md — the only changed file contains the empty benchmark entry that should be investigated before merging.

Important Files Changed

Filename	Overview
generated/benchmarks/EMBEDDING-BENCHMARKS.md	Adds a 3.3.0 benchmark entry with symbols: 0 and an empty models: {} object, producing a blank results table; strongly suggests the benchmark workflow run completed without actually running any benchmarks.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Release workflow triggers benchmark run] --> B[codegraph embed - index symbols]
    B --> C{Symbols indexed > 0?}
    C -- Yes --> D[Run embedding benchmarks per model]
    D --> E[Record Hit@1/3/5/10, Misses, timing]
    E --> F[Commit populated entry to EMBEDDING-BENCHMARKS.md]
    C -- No or Error --> G[symbols: 0, models: empty]
    G --> H[Commit empty entry as latest results - THIS PR]
    H --> I[Latest results table renders blank]
    F --> J[Latest results table shows valid benchmark data]

Loading

_{Last reviewed commit: "docs: update embeddi..."}

[greptile-apps]

Empty benchmark result committed — symbols: 0, models: {}

The 3.3.0 entry records symbols: 0 and "models": {}, meaning no symbols were indexed and no model benchmarks were executed. The previous version (3.1.4) had 1,095 symbols across populated model results. An empty result like this indicates the workflow run #381 either failed during the benchmark phase (before any symbols were embedded) or completed a dry-run with no data, and the CI still committed these empty results as a valid benchmark update.

As it stands, the "Latest results" table renders completely empty and the symbols: 0 headline is misleading — it implies the codebase has no indexable symbols rather than reflecting a measurement failure. Merging this will overwrite a valid 1,095-symbol baseline with an empty entry as the authoritative "latest" result.

Suggested actions before merging:

• Verify why the workflow run produced 0 symbols (check run logs for errors in the embedding/indexing phase).
• If the run genuinely failed, close this PR and re-trigger the benchmark workflow once the issue is resolved.
• If symbols: 0 is intentional for some reason (e.g., a no-symbol build), add a comment in the JSON entry explaining it so future readers aren't confused.

[carlos-alm]

Agreed — this is a failed benchmark run. The PR should be closed and re-triggered after diagnosing why the workflow produced 0 symbols.

[carlos-alm]

This PR should not be merged — the 3.3.0 embedding benchmark recorded 0 symbols and empty models, indicating the workflow run failed during the indexing phase. Merging would overwrite the valid 3.1.4 baseline (1,095 symbols) with empty data. Recommend closing and re-triggering the benchmark workflow once the issue is diagnosed.

[carlos-alm]

Root cause identified and fixed in #528: @huggingface/transformers is a devDependency, not a regular/optional dependency. In --npm mode, the benchmark installs codegraph into a temp dir where devDeps are absent. All 6 model workers crash on import, producing symbols: 0, models: {}.

Fix: bench-config.js now explicitly installs @huggingface/transformers in the temp dir (matching the existing pattern for native platform packages). Also added a guard in update-embedding-report.js that rejects empty results instead of silently overwriting valid data.

This PR should still be closed — once #528 merges, re-trigger the benchmark workflow to get valid 3.3.0 embedding data.