Skip to content

Constrained query expansion#998

Merged
safishamsi merged 1 commit into
Graphify-Labs:v8from
eugene-krivosheyev:v8
May 24, 2026
Merged

Constrained query expansion#998
safishamsi merged 1 commit into
Graphify-Labs:v8from
eugene-krivosheyev:v8

Conversation

@eugene-krivosheyev

Copy link
Copy Markdown
Contributor

Add constrained query expansion step to /graphify query skill

Problem

graphify query matches via case-folded substring + IDF — no stemming, no synonyms, no cross-language match. When the user's question uses different vocabulary than the graph labels (Slavic → English, "handlers" → "handler", "обработчик" → "handler"), the literal matcher returns 0 hits and the LLM consumer either gets empty subgraph or improvises an ungrounded keyword list from training memory (e.g. expanding "auth" to {passport, sso, saml, oauth, jwt, scim, …} regardless of whether those tokens exist in the corpus).

Fix

Adds a Step 0 — Constrained query expansion block to the skill's /graphify query section. The LLM consumer extracts vocabulary from graph labels (CamelCase/snake_case split, length-filtered) and is instructed to pick only tokens present in that vocabulary, explicitly forbidden from inventing terms.

Effects:

  • Bounded improvisation — fantom tokens (terms not in corpus) cannot be expanded, even when LLM "knows" they're related to the intent.
  • Honest negative signal — if vocab is poor on a query's topic, expansion returns [] and the LLM tells the user, instead of fabricating a search.
  • Auditability — selected tokens are printed to the user, and saved into save-result for the next --update to graph as Q&A nodes.

Scope

Patches the canonical graphify/skill.md. The 11 host-variant skills (skill-codex.md, skill-aider.md, …) follow the same query-section contract but inline Python rather than calling graphify query CLI; those need a parallel patch with the inline form. Happy to follow up in a separate PR after review on the canonical patch.

Test

On a graph built from the graphify repo itself (1284 nodes, 1454 vocab tokens), an unconstrained expansion of "укрупненная архитектура аутентификации" yields {auth, oauth, jwt, saml, sso, ldap, scim, mfa, 2fa, pin, passport, session, login, token} — of which 11/15 are absent from the corpus. Constrained expansion against the actual vocab yields {credential, security, token, signature, user, architecture, component, module, overview} — 9 tokens, 0 fantom. Same retrieval, dramatically higher precision.

Add constrained query expansion step to /graphify query skill

## Problem

`graphify query` matches via case-folded substring + IDF — no stemming, no synonyms, no cross-language match. When the user's question uses different vocabulary than the graph labels (Slavic → English, "handlers" → "handler", "обработчик" → "handler"), the literal matcher returns 0
hits and the LLM consumer either gets empty subgraph or improvises an ungrounded keyword list from training memory (e.g. expanding "auth" to `{passport, sso, saml, oauth, jwt, scim, …}` regardless of whether those tokens exist in the corpus).

## Fix

Adds a `Step 0 — Constrained query expansion` block to the skill's `/graphify query` section. The LLM consumer extracts vocabulary from graph labels (CamelCase/snake_case split, length-filtered) and is instructed to pick **only** tokens present in that vocabulary, explicitly forbidden from inventing terms.

Effects:
- Bounded improvisation — fantom tokens (terms not in corpus) cannot be expanded, even when LLM "knows" they're related to the intent.
- Honest negative signal — if vocab is poor on a query's topic,  expansion returns [] and the LLM tells the user, instead of  fabricating a search.
- Auditability — selected tokens are printed to the user, and saved into `save-result` for the next --update to graph as Q&A nodes.

## Scope

Patches the canonical `graphify/skill.md`. The 11 host-variant skills (skill-codex.md, skill-aider.md, …) follow the same query-section contract but inline Python rather than calling `graphify query` CLI; those need a parallel patch with the inline form. Happy to follow up in a separate PR after review on the canonical patch.

## Test

On a graph built from the graphify repo itself (1284 nodes, 1454 vocab tokens), an unconstrained expansion of "укрупненная архитектура аутентификации" yields {auth, oauth, jwt, saml, sso, ldap, scim, mfa, 2fa, pin, passport, session, login, token} — of which 11/15 are absent
from the corpus. Constrained expansion against the actual vocab yields {credential, security, token, signature, user, architecture, component, module, overview} — 9 tokens, 0 fantom. Same retrieval, dramatically higher precision.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants