From PDFs to Truth: Why Document Reconciliation Requires Knowledge Graphs
29m
A chatbot by CSI-Piemonte serves millions of citizens consulting public recruitment competitions and exam schedules. Competitions publish official notices as PDFs defining exam sessions segmented by region, surname ranges, and candidate codes. These notices are continuously amended by corrections and cancellations—often hundreds per day across national procedures.
A naive architecture feeding all documents into a single LLM context window fails predictably. When a session appears at 14:00 in an original notice and 10:00 in a correction, the LLM has no model of supersession. It hedges, guesses, or hallucinates—returning cancelled sessions as active or mixing outdated information.
We introduce a knowledge graph as a data quality layer between raw documents and the vector database. LLMs extract structured facts from individual PDFs. The graph handles what LLMs cannot—reconciling documents through explicit event chains, applying amendments, tracking cancellations, and preserving historical state.
The schema separates identity (session defined by competition, region, surname range, code) from versioned state (date, time, venue), connected through version and amendment relationships. Resolving current truth becomes deterministic graph traversal: follow the version chain and check for cancellations, no temporal reasoning is delegated to the LLM.
Only resolved, conflict-free information is indexed. The chatbot answers accurately because contradictory data never reaches vector store.