There's a category of problems where standard RAG - embed documents, retrieve by semantic similarity, generate an answer - simply doesn't work well enough. Regulatory compliance is one of them.
The issue isn't retrieval accuracy on individual chunks. It's that compliance problems are often relational: a sub-clause in Appendix C inherits a definition from Section 2.4, which was superseded by an addendum issued six months later, which contradicts a project-specific deviation approved in the technical meeting minutes. No flat similarity search will surface that chain of relationships, because no single chunk contains it.
This is the problem we solved for a construction procurement client in Sweden, and this post is a detailed account of how we did it.
Why Knowledge Graphs for compliance?
The fundamental insight is that regulatory and procurement documents are not just collections of text - they are webs of entities and relationships. Requirements reference specifications. Specifications supersede earlier standards. Clauses inherit definitions from other sections. Deviations override defaults. Parties have roles that constrain their obligations.
A Knowledge Graph makes these relationships first-class. Once you've modeled the domain correctly in a graph, compliance queries become graph traversals: "find all clauses in the submitted tender that conflict with any requirement reachable within 3 hops from this regulatory section." That's a Cypher query in Neo4j. It's not possible to express it meaningfully in a vector similarity search.
Neo4j AuraDB: the managed option
We chose Neo4j AuraDB (the managed cloud version) rather than self-hosting Neo4j for this engagement. The reasons were pragmatic: the client had no existing graph infrastructure, AuraDB handles backups, upgrades, and availability SLAs without operational overhead, and the performance characteristics for our query patterns were well within AuraDB's capabilities.
The schema we designed has five core node types: Requirement, Specification, Clause, Standard, and Party. Edge types include REQUIRES, REFERENCES, SUPERSEDES, CONTRADICTS, and APPLIES_TO. Getting the schema right upfront was the most important design decision - changing graph schemas in production is expensive.
The ingestion pipeline
Document ingestion is a four-stage pipeline:
1. Parsing and segmentation. Swedish construction documents come in heterogeneous formats. We built format-specific parsers for the three most common types (structured XML exports, Word documents, scanned PDFs with OCR). Each parser outputs a canonical JSON format with section hierarchy preserved.
2. Named Entity Recognition. A fine-tuned NER model extracts entities (parties, standards references, defined terms) and their positions. For Swedish-language documents, we found that a multilingual XLM-RoBERTa model fine-tuned on a small labeled set of construction contracts outperformed general-purpose models significantly.
3. Relationship extraction. This is where a standard NLP pipeline falls short. We used GPT with a structured output schema to extract relationships between entities, with the surrounding section as context. Each extracted relationship includes a confidence score; low-confidence extractions go to a human review queue before being committed to the graph.
4. Graph upsert. Entities and relationships are upserted into Neo4j using idempotent Cypher MERGE statements. The pipeline is designed to be re-run safely - documents can be re-ingested without creating duplicate nodes or edges.
The GraphRAG query layer
Standard RAG retrieves document chunks via vector similarity. GraphRAG does that, but also traverses the graph to retrieve structurally related context.
Our implementation works in two phases:
Phase 1 - Anchor node retrieval. The query is embedded and used to find the most relevant entity nodes in the graph (via a vector index on node descriptions stored in Neo4j 5.x). This gives us a set of anchor nodes.
Phase 2 - Graph traversal. From each anchor node, we run configurable-depth Cypher traversals to retrieve related entities and relationships. The traversal depth and edge type filters are controlled by query routing logic - a simple classifier that identifies whether the query is asking about a single clause, a cross-document comparison, or a hierarchical requirement chain.
The combined context (vector-retrieved chunks + graph traversal context) is assembled and passed to the LLM for synthesis. Every response is grounded with explicit graph-path citations: "Clause 7.3.2 contradicts Standard EN-15643 Section 4.1 via the inheritance chain: Project Spec § 3.1 → references → EN-15643 → supersedes → earlier deviation."
Production deployment
The system runs on GCP with a FastAPI async backend and a streaming response API. Each compliance check request triggers a LangGraph workflow that coordinates the vector retrieval, graph traversal, and LLM synthesis steps.
Multi-tenant isolation is implemented at the graph layer: each procurement agency gets a dedicated Neo4j database within the AuraDB instance, with application-layer authentication enforcing tenant boundaries. Shared infrastructure reduces costs; data isolation maintains compliance with procurement confidentiality requirements.
Results
The numbers were striking. A compliance consistency check that previously required 2 weeks of expert review was completed in under 2 hours. More importantly, the system surfaced 3x more inconsistencies than a flat RAG baseline we ran in parallel for the first month - all real issues, all grounded with full provenance.
The production system now handles 10+ simultaneous procurement agencies on the shared infrastructure, with consistent sub-8-second response times for complex multi-hop queries.
When should you use GraphRAG vs. standard RAG?
The honest answer is: standard RAG is simpler, cheaper, and sufficient for many use cases. GraphRAG adds meaningful complexity - schema design, ingestion pipeline, graph query logic. You should consider it when:
- Your domain has important entity relationships that flat chunks don't capture
- Users ask relational questions: "compare X against Y," "what does this inherit from?"
- Provenance and explainability are requirements, not nice-to-haves
- Your document corpus has genuine semantic overlap that vector similarity conflates
Compliance, legal, financial analysis, and technical documentation are the domains where we've seen GraphRAG consistently outperform flat RAG. If you're working in one of these areas, the added complexity is usually worth it.