Skip to content

[draft] feat: PPR v2 โ€” node specificity (IDF) re-ranking#75

Draft
TaskerJang wants to merge 1 commit into
devfrom
feat/ppr-node-specificity
Draft

[draft] feat: PPR v2 โ€” node specificity (IDF) re-ranking#75
TaskerJang wants to merge 1 commit into
devfrom
feat/ppr-node-specificity

Conversation

@TaskerJang

Copy link
Copy Markdown
Owner

โš ๏ธ ์ธก์ • ์ „ โ€” ๋ฐ•์ œ์šฉ draft (๋จธ์ง€ ๋ณด๋ฅ˜)

PPR v1์˜ faithfulness ํ•˜๋ฝ(97โ†’84%) ๋ถ€๊ฒ€์—์„œ ๋“œ๋Ÿฌ๋‚œ metric ๋…ธ์ด์ฆˆ hub(degree ๋†’์€ "๋งค์ถœ์•ก 2์กฐโ€ฆ" ๋ฅ˜๊ฐ€ PPR ์ƒ์œ„ ์ ๋ น โ†’ ๊ณ๊ฐ€์ง€ ์ฒญํฌ ํ˜ผ์ž…) ์ฒ˜๋ฐฉ.

๋ฌด์—‡

node specificity ์žฌ๋žญํ‚น: ์—”ํ‹ฐํ‹ฐ e ์˜ spec(e) = log(1 + N/df(e)) (df = e ๋ฅผ ์–ธ๊ธ‰ํ•œ ์ฒญํฌ ์ˆ˜, N = ์ „์ฒด ์ฒญํฌ ์ˆ˜, IDF ๊ณ„์—ด). raw PPR ์€ ๊ทธ๋Œ€๋กœ ๋‘๊ณ (ํ™œ์„ฑํ™” ํ™•์‚ฐ ๋ณด์กด) re-ranking ์ธต์œผ๋กœ๋งŒ ๊ณฑํ•œ๋‹ค โ€” eff(e) = pr(e) ยท spec(e). top-15 ์—”ํ‹ฐํ‹ฐ ์„ ์ • + ์ฒญํฌ ์ ์ˆ˜ ํ•ฉ์‚ฐ ๋ชจ๋‘ eff ์‚ฌ์šฉ โ†’ generic hub ๊ฐ€ ๋Œ์–ด์˜จ ๊ณ๊ฐ€์ง€ ์ฒญํฌ๊ฐ€ ๋ฐ€๋ ค๋‚œ๋‹ค.

  • NODE_SPECIFICITY ํ”Œ๋ž˜๊ทธ๋กœ v1(off)/v2(on) ํ† ๊ธ€
  • _compute_specificity ์ถ”๊ฐ€, _load_entity_graph๊ฐ€ spec ๋™๋ด‰ ์บ์‹œ, readout์ด eff ๊ธฐ์ค€
  • top_entities์— ppr/spec ์ง„๋‹จ ํ•„๋“œ ๋…ธ์ถœ

๊ฐ€์„ค (์ธก์ • ๋Œ€๊ธฐ)

  • AC ์œ ์ง€/์†Œํญโ†‘ (๊ณ๊ฐ€์ง€ ์ฒญํฌ ์ •๋ฆฌ)
  • faithfulness ํšŒ๋ณต (graph_012๋ฅ˜ ungrounded ๊ฐ์†Œ) โ† ํ•ต์‹ฌ ๊ธฐ๋Œ€
  • ์œ„ํ—˜: specific metric ๊ณผ๋„ down-weight ์‹œ metric ์งˆ๋ฌธ(graph_012๋ฅ˜) AC ์†์‹ค ๊ฐ€๋Šฅ

๊ฒ€์ฆ

container mock e2e: hub ์—”ํ‹ฐํ‹ฐ PPR 0.103โ†’eff 0.098๋กœ ์–ต์ œ, specific 0.224โ†’0.492๋กœ ์ƒ์Šน (entity ๋žญํ‚น ์—ญ์ „ ํ™•์ธ). ์‹ค์ธก์€ --tag gpt52_e1_ppr_v2๋กœ v1(gpt52_e1_ppr, 24๊ฐœ 2.25 / faithfulness 84%) ๋Œ€๋น„ ์˜ˆ์ •.

๋จธ์ง€ ์กฐ๊ฑด: ์ธก์ •์—์„œ faithfulness ํšŒ๋ณต + AC ๋น„ํ‡ดํ–‰ ํ™•์ธ ์‹œ.

raw PPR์— spec(e)=log(1+N/df) ๊ณฑํ•ด generic hub(metric ๋…ธ์ด์ฆˆ) down-weight.
re-ranking ์ธต์œผ๋กœ๋งŒ ์ž‘๋™(ํ™œ์„ฑํ™” ํ™•์‚ฐ ๋ณด์กด), NODE_SPECIFICITY ํ”Œ๋ž˜๊ทธ ํ† ๊ธ€.
์ธก์ • ์ „ โ€” ๋ฐ•์ œ์šฉ (unmerged).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant