Skip to content

MarcelLeon/graph-rag-java

BranchesTags

Repository files navigation

graph-rag-harness

English · 中文

A pluggable Java GraphRAG harness with a runnable Spring AI + Ollama demo, side-by-side plain-RAG vs graph-RAG comparison, and an agent-friendly graph retrieval SPI.

CI License: Apache 2.0 Java 17 Spring AI Status: public preview

A small, honest, pluggable GraphRAG layer for any Java agent stack — with a runnable Spring AI + Ollama demo that lets you experience plain-RAG vs graph-RAG side by side from your terminal.

Demo (25 seconds)

One short DAU question, real local Ollama, graph-RAG output with explicit relation chains.

Hero demo

Real run, real local Ollama (qwen3:8b chat + nomic-embed-text embeddings), --variant=graph. JVM warm-up was hidden so only CLI interaction is visible (~22 s of streaming on top of typing).

Want the full plain-RAG vs graph-RAG side-by-side experience? See the longer extended demo (60 s, both variants).

Reproduce locally with the recipe in 5-minute quick start below; rerecord either GIF yourself with the included vhs tapes (short / long) — see Record the demo GIF yourself.

The graph variant pulls explicit relations out of a typed knowledge graph and feeds them to the LLM as deterministic context, so multi-hop / lineage / impact questions stay structural instead of drifting into prose.

Why this project exists

Most RAG demos stop at semantic similarity over prose chunks. This project takes a more structured path:

  • keep a knowledge graph as the retrieval layer,
  • expose it through a stable, ChatGPT-style tool / SPI contract,
  • and ship a runnable plain-RAG vs graph-RAG comparison in one CLI.

What you get today:

  • a reusable GraphDatabaseClient abstraction (Neo4j default + zero-dep in-memory backend),
  • a Spring Boot starter with Ops APIs,
  • a Spring AI example app pre-wired to local Ollama,
  • a dogfooding CLI for plain | graph | both side-by-side experience,
  • and an LLM-as-judge evaluation harness for multi-hop / lineage / impact-analysis questions.

Current status: public preview / demo-ready. Latest real Ollama eval shows positive L3 uplift of +0.50 for graph-RAG over plain-RAG (overall +0.20). Honest target tracker: NORTH_STAR milestone is L3 ≥ +1.0, see STATUS.md.

Highlights

  • Backend-agnostic — graph operations go through GraphDatabaseClient. Swap Neo4j for an in-memory backend or your own implementation without touching agents.
  • Agent-framework-agnostic — integration surface is GraphRagToolSpi with two stable methods (graphKnowledgeSearch, metricRelationQuery). Wire it into Spring AI @Tools, LangChain4j tools, or any custom agent harness.
  • Zero internal-dependency — open-source-bound. No com.kuaishou.*, no private SDK, no surprise classpath collisions.
  • Noop-fallback everywhere — every SPI ships a Noop default so the demo runs offline out of the box.
  • Honest evaluation — LLM-as-judge over 10 manually authored questions (L1 / L2 / L3), with NORTH_STAR uplift tracking and per-question latency.

5-minute quick start

Prerequisites

  • Java 17
  • Maven 3.9+
  • Ollama running locally
  • Local models:
    • qwen3:8b (or any 7-8B chat model, e.g. llama3.1:8b, qwen2.5:7b)
    • nomic-embed-text:latest

Check your models:

ollama list

Run the side-by-side dogfooding CLI

From the repository root:

export JAVA17_HOME=/Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home

# First-time setup: install the harness modules into your local Maven cache.
# Only required once after `git clone` (and again any time graph-rag-core
# or graph-rag-spring-boot-starter change). Running `spring-boot:run` on
# graph-rag-examples will otherwise fail to resolve its upstream deps.
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-core,graph-rag-spring-boot-starter -am -DskipTests install -q

# Then launch the dogfooding CLI (note: NO -am here, see Troubleshooting below).
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples spring-boot:run \
  -Dspring-boot.run.arguments="--mode=cli --variant=both --server.port=8086"

Prefer a single binary instead of going through Maven each time? Build the fat jar once and run it directly:

JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples -am -DskipTests package -q
JAVA_HOME=$JAVA17_HOME java -jar graph-rag-examples/target/graph-rag-examples-1.0.0-SNAPSHOT.jar \
  --mode=cli --variant=both --server.port=8086

Why 8086? The example app is still a Spring Boot web application under the hood, so a non-default port avoids common 8080 conflicts during local dogfooding.

When startup succeeds you should see:

=== graph-rag-harness dogfooding CLI ===
Variant: both
Type a question and press Enter.
Special commands: :help  :examples  exit

Then ask:

Walk me through the full data lineage from the physical storage layer up to the DAU number on the executive dashboard.

Exit with:

exit

Try other variants

# graph-only (recommended for the GIF)
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples spring-boot:run \
  -Dspring-boot.run.arguments="--mode=cli --variant=graph --server.port=8086"

# plain-only baseline
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples spring-boot:run \
  -Dspring-boot.run.arguments="--mode=cli --variant=plain --server.port=8086"

Recommended dogfooding questions

What is Daily Active Users (DAU)?
Which sub-metrics are used in the formula for DAU?
What is the most common dimension used to slice DAU, and why?
If fact_user_activity_daily fails to load tomorrow, which top-level metrics will be impacted? Trace the lineage.
Walk me through the full data lineage from the physical storage layer up to the DAU number on the executive dashboard.

In graph mode, watch for:

  • whether graphContext appears,
  • whether relation chains such as DERIVED_FROM, BELONGS_TO, FORMULA_USES show up,
  • and whether the answer feels structural rather than a plain prose summary.

Run the evaluation harness

export JAVA17_HOME=/Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home

# Make sure the upstream modules are installed locally (see First-time setup
# above). Then run eval — again, NO -am on the spring-boot:run line.
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples spring-boot:run \
  -Dspring-boot.run.arguments="--mode=eval --server.port=8087"

Or, with the pre-built fat jar:

JAVA_HOME=$JAVA17_HOME java -jar graph-rag-examples/target/graph-rag-examples-1.0.0-SNAPSHOT.jar \
  --mode=eval --server.port=8087

You will get:

  • graph-rag-examples/target/eval-report.md — summary table + L1/L2/L3 aggregates + NORTH_STAR uplift check
  • graph-rag-examples/target/eval-report.raw.md — verbatim plain / graph answers and judge verdicts

A latest reference report (real Ollama, May 26 2026):

Level N plain avg graph avg Δ
L1 3 3.00 3.00 +0.00
L2 3 3.00 3.00 +0.00
L3 4 2.50 3.00 +0.50
all 10 2.80 3.00 +0.20

NORTH_STAR target: L3 ≥ +1.0. Currently at +0.50 — public preview story is positive but not yet at the milestone.

Architecture at a glance

graph-rag-harness/
├── graph-rag-core/                  # core engine, no hard Spring coupling
├── graph-rag-spring-boot-starter/   # Spring Boot auto-config + REST Ops API
└── graph-rag-examples/              # Spring AI demo + Ollama dogfooding CLI + eval harness

Core flow:

External data ──▶ GraphIngestionService ──▶ GraphDatabaseClient
                                               │
Agent / LLM ──▶ GraphRagToolSpi ──▶ GraphRetrievalService
                                               │
                              GraphContext / MetricDependencyContext
                                               │
                                       markdown / tool context

Integration surface

If you want to adopt the graph retrieval layer in your own Spring Boot app or Java agent harness, start here:

  • docs/playbooks/integration-guide.md — five-minute integration, four ingestion paths, Spring AI / LangChain4j / custom-agent wiring, output contract, enterprise-concerns matrix.

Key types:

  • GraphDatabaseClient — backend abstraction (21 methods)
  • GraphIngestionService — write path, fed by *DataPort SPIs
  • GraphRetrievalService — read path
  • GraphRagToolSpi — agent-facing tool contract (append-only)

Project status

What is already solid

  • zero-dependency in-memory graph backend
  • startup graph loader for the example app
  • Spring AI + Ollama example wiring
  • plain / graph answer services
  • evaluation harness (--mode=eval)
  • interactive dogfooding CLI (--mode=cli)
  • positive L3 uplift on real Ollama (+0.50, see eval table above)

What is still in progress

  • pushing L3 uplift from +0.50 to ≥ +1.0 (NORTH_STAR milestone)
  • richer fixture graph for stronger lineage / ownership / consumption questions (see docs/playbooks/sample-graph-v2-design.md)
  • polishing GitHub-facing assets and release posture

Record the demo GIF yourself

The repo ships two GIFs:

Both are produced by vhs tapes that live next to them (short tape / long tape). To rerecord:

# 1. Make sure the example app is built (fat jar) and Ollama is up with the right models.
export JAVA17_HOME=/Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples -am -DskipTests package -q
ollama run qwen3:8b "ok" >/dev/null     # warm up the chat model

# 2. Install vhs once.
brew install vhs                        # macOS; see https://github.com/charmbracelet/vhs for other OS

# 3. Re-render either GIF.
vhs docs/assets/demo-short.tape         # writes docs/assets/demo-short.gif         (hero, ~25 s)
vhs docs/assets/demo.tape               # writes docs/assets/demo-side-by-side.gif  (extended, ~60 s)

Both tapes:

  • run ./docs/assets/run-cli.sh (the in-repo launcher) to start the dogfooding CLI on port 8088,
  • hide JVM warm-up so only CLI interaction is visible,
  • type one short question and let the answer fully render,
  • exit cleanly.

The short tape sets VARIANT=graph; the long tape uses the default VARIANT=both. If your machine produces slightly different latencies, tweak the trailing Sleep in either tape.

Honest caption suggestion: "Side-by-side dogfooding of plain-RAG vs graph-RAG. graph-RAG surfaces explicit relation chains (FORMULA_USES, FILTERS_BY, DERIVED_FROM) instead of free-form prose."

Avoid over-claims like "GraphRAG consistently outperforms plain RAG" or "Benchmark-proven multi-hop uplift" — current eval is L3 +0.50, the NORTH_STAR target is +1.0.

Documentation map

Need Read this
Project direction / non-negotiables NORTH_STAR.md
Current project status STATUS.md
User-visible change history CHANGELOG.md
Integration guide for adopters docs/playbooks/integration-guide.md
Demo / fixture roadmap docs/playbooks/demo-design.md · docs/playbooks/sample-graph-v2-design.md
Agent / contributor handoff protocol AGENTS.md
ADRs and project decisions docs/decisions/
Journal / pitfalls / blockers docs/journal/
How to contribute CONTRIBUTING.md
Security policy SECURITY.md
Code of conduct CODE_OF_CONDUCT.md

Development quick check

export JAVA17_HOME=/Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home
JAVA_HOME=$JAVA17_HOME mvn test -pl graph-rag-core -q 2>&1 | tail -5

Expected:

Tests run: 65, Failures: 0, Errors: 0, Skipped: 0

Troubleshooting

Unable to find a suitable main class when running spring-boot:run

Symptom:

Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:3.2.5:run
  (default-cli) on project graph-rag-harness: Unable to find a suitable main class,
  please add a 'mainClass' property

Cause: you added -am (Maven's "also make") to spring-boot:run. With -am, the reactor includes every dependency module and the root pom — so Maven tries to execute spring-boot:run on the root graph-rag-harness (packaging=pom, no main class) before it ever gets to graph-rag-examples, and the build fails on module #1.

Fix: run install and spring-boot:run as two separate invocations:

# Step 1 (one-time, or whenever upstream modules change) — install upstream into local cache
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-core,graph-rag-spring-boot-starter -am -DskipTests install -q

# Step 2 — run the example app on its own (NO -am)
JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples spring-boot:run \
  -Dspring-boot.run.arguments="--mode=cli --variant=both --server.port=8086"

Alternative — skip Maven at runtime entirely with the pre-built fat jar:

JAVA_HOME=$JAVA17_HOME mvn -pl graph-rag-examples -am -DskipTests package -q   # build once
JAVA_HOME=$JAVA17_HOME java -jar graph-rag-examples/target/graph-rag-examples-1.0.0-SNAPSHOT.jar \
  --mode=cli --variant=both --server.port=8086

(The reason mvn ... -am package is safe but mvn ... -am spring-boot:run is not: package on a pom-packaging module is a no-op success, while spring-boot:run is a directly invoked goal that fires on every reactor project regardless of packaging.)

Port 8086 / 8087 / 8088 already in use

The Spring Boot web server in the example app is enabled by default, so the dogfooding CLI still binds a port. Earlier dev sessions sometimes leave orphan JVMs behind. Clear them:

lsof -ti:8086 -ti:8087 -ti:8088 | xargs -r kill -9
# or, more aggressive:
pkill -9 -f "graph-rag-examples-1.0.0-SNAPSHOT.jar"

License

Apache-2.0. See LICENSE.

About

A pluggable Java GraphRAG harness with a runnable Spring AI + Ollama demo, side-by-side plain-RAG vs graph-RAG comparison, and an agent-friendly graph retrieval SPI.

github.com/MarcelLeon/graph-rag-harness#5-minute-quick-start

Topics

java spring-boot neo4j knowledge-graph ai-agents rag llm retrieval-augmented-generation ollama llm-evaluation spring-ai graphrag

Resources

Readme

License

Apache-2.0 license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages