GraphRAG is having its moment. After Microsoft's research drop and a dozen open-source frameworks shipping in 2025, every enterprise AI proposal we receive in 2026 mentions a "knowledge graph layer." Most of those proposals shouldn't.

This is the framework we use on client work to decide between the three real options: flat RAG, hybrid RAG, and GraphRAG. With cost numbers from production deployments.

The 30-second version

Pattern Use when Avoid when
Plain RAG Documents are mostly independent. Questions are mostly retrieval. Answers require multi-hop reasoning across linked entities.
Hybrid RAG (vector + keyword + metadata filter) Most enterprise search scenarios. You need explicit relationship traversal.
GraphRAG Entities have meaningful, queryable relationships. Questions are structural ("show me all X that touch Y"). You don't yet have a clear schema for those relationships.

If you can answer the user's question by reading a single chunk plus its neighbours, you don't need a graph. If the question is "which of our deliverables are downstream of regulator-flagged requirements," you do.

Where flat RAG silently fails

The classic flat-RAG pipeline — embed everything, retrieve top-k, pass to LLM — has three failure modes that don't show up in demos:

  1. Multi-hop questions. "Who owns the team responsible for the system that processes the data flagged in last quarter's audit?" Five hops. Vector similarity flattens that to "stuff related to audits." The model hallucinates the rest.
  2. Aggregation questions. "How many of our active risks affect EU customers?" Vector search returns the top-k most-similar risks, not all relevant ones. You get a confident wrong answer.
  3. Updates. Update a single source document and you must re-embed every chunk that referenced it. With no graph, you don't even know which chunks referenced it.

Flat RAG is a great fit for FAQ-style retrieval, support-doc lookup, and "summarise this section." Nothing more.

!
The demo trap. Every flat-RAG demo looks great because the demo dataset is curated to make top-k similarity work. The failure modes above only emerge once a real user types a real question against real data. Build evals before you build retrieval.

Where GraphRAG genuinely wins

GraphRAG isn't just RAG with a graph database stapled on. The pattern is:

  1. Entity + relation extraction at ingest time (LLM-driven, schema-guided).
  2. Community detection to cluster related subgraphs.
  3. Hierarchical summarisation of those clusters.
  4. Subgraph retrieval at query time — return the neighbourhood, not a chunk list.
  5. LLM reasoning over the structured subgraph.

This pattern is genuinely better — sometimes 3–5× more accurate on multi-hop benchmarks — for:

  • Compliance and audit trails (where "derived from" matters legally).
  • Portfolio-level analytics (which is why we use it in Vero).
  • Document derivation chains (why we use it in DocEngine).
  • Investigative search across linked records (KYC, fraud, due diligence).
  • Engineering knowledge bases where traceability is the core question.

The cost reality

Across three client deployments we ran in 2025 — on roughly comparable corpora (~2M tokens, ~50k entities) — the relative costs were:

Cost dimension Flat RAG Hybrid RAG GraphRAG
Indexing tokens 1× ~1.1× ~6–8×
Storage 1× ~1.5× ~3×
Per-query tokens 1× ~1.2× ~1.5–2×
Engineering time to ship v1 2 weeks 4 weeks 10–14 weeks
Maintenance overhead Low Low High

The headline: GraphRAG costs 6–8× more to index and 3× more to operate. If your problem is genuinely structural, that cost is worth it. If it isn't, you've spent a quarter and a six-figure budget to ship something a $200/month managed-RAG product would have done.

Three of our last five "AI" projects shipped without a vector database at all

Most of the lift in real client work doesn't come from the retrieval layer — it comes from a clean schema, a small evaluator, and prompt engineering. We've shipped LLM features on top of plain Postgres full-text search and a metadata filter, and the client never noticed.

If you take one thing from this piece, take the question we ask first on every AI engagement:

Can we answer the user's questions with structured queries against the data we already have, plus a small LLM on top? — First question on every AI engagement

If yes, build that first. Then, only if the structural-reasoning ceiling becomes a real ceiling, reach for the graph.

GraphRAG is a real tool. It's not a default.