Back to blogApprofondimenti

    Enterprise knowledge graph: from Graph RAG to a virtual graph over existing data

    Graph RAG reads an archive of documents; it does not represent the company's data. The limits of graph databases, the virtual graph alternative and the decisions on ownership, writing and access.

    ZeroFive.AI November 20, 2026 8 min

    In short. Graph RAG builds a graph of what documents say and is useful for questions about a whole archive, but it is not the same thing as a representation of company data. The real difficulty lies in connecting the conceptual model to the instances that live in the systems, without replicating the data platform inside a graph database. A virtual graph, which leaves the data where it is and exposes it through mappings, is a route worth assessing. Before the technology come the decisions on who owns the graph, who may write to it and who sees what.

    In recent months almost every architecture proposal for AI agents that crosses our desk has contained the word graph. The knowledge graph is presented as the layer that will give agents the company's knowledge, and Graph RAG as the technique for querying it in natural language. Both things are true within a certain perimeter, and it is worth knowing that perimeter before approving the investment.

    We have written about ontologies several times, from the glossary to the taxonomy through to the enterprise ontology as a model of the entities that matter. Here we are interested in the next step, the one where the conceptual model has to meet the real data, because that is where projects slow down and decisions stop being technical.

    Graph RAG and its field of application

    The technique was described by Microsoft Research in a paper from April 2024. The idea is to have a model read a set of documents, extract entities and relationships, group them into communities and prepare a summary for each. When a question arrives that concerns the whole corpus, for example which themes recur in the past year's minutes, the system answers from the summaries, something similarity-based retrieval cannot do because no single fragment contains the answer.

    The limit that emerged straight away is the cost of preparation, since every document is read by a model several times. In November 2024 the same authors presented a variant, LazyGraphRAG, which defers extraction to the moment of the question and brings the indexing cost down to 0.1% of the full version's. The fact that the correction came from the group that had proposed the method says how much of an obstacle that cost was.

    Graph RAG, in every variant, works on documents. It builds a graph of what the texts say, and it does so with a model that can get an extraction wrong or merge two entities with the same name. It remains a tool for reading an archive, useful within that boundary, and it is not the same thing as a representation of the company's data.

    The conceptual layer and the instances

    An ontology describes the surface. It says that the customer entity exists, that a customer has contracts, that a contract belongs to a product line. Those who worked with expert systems in the years when ontologies were built by hand remember that they worked well as long as domain experts were there to maintain them, and that in any case they stopped at that level, without going down to the data.

    Beneath the surface are the instances, the millions of records that live in the business systems, the data warehouse, the data platform. An agent asked which customers are exposed to a supplier's delay needs both things: the map that tells it how customer and supplier are connected, and the up-to-date data on orders and deliveries.

    The passage from one to the other is where the difficulty concentrates. Every column in a table has a technical name decided years ago by whoever designed the system, and working out which attribute of which entity it corresponds to is reconciliation work that generative models now speed up considerably and that still has to be validated by people who know the data. Without that correspondence the ontology remains a drawing.

    The limits of graph databases

    The first solution that comes to mind is to load everything into a graph database. Anyone who has tried it on enterprise volumes knows the outcome, because these engines are very effective when the question concerns the structure of relationships, such as recognising a recurring pattern or finding entities that are similar in the way they are connected, and much less so on the aggregations and bulk reads for which analytical platforms have been optimised over twenty years.

    Replicating the whole data platform inside a graph also means keeping two copies aligned, with the costs and delays every copy brings. The risk is ending up with a complete, well-drawn map that takes minutes to answer a question the data warehouse answers in seconds.

    The same reasoning, reversed, applies to document databases, which gain speed by giving up relationships and present the bill when a query crosses several collections. Every technology was designed for one type of question, and a company's information assets ask questions of every type.

    Which framework does your company actually need?

    AI Rating measures maturity across the four areas of the model and shows where to start, with priorities and estimated effort.

    Start your AI Rating

    The virtual graph over existing platforms

    There is an intermediate route, less well known outside academia, which consists of leaving the data where it is and building a virtual graph on top of it. The ontology is connected to the tables through a set of mappings, and when a question arrives expressed in the ontology's terms the system translates it into queries on the underlying databases.

    One of the references in this field is Italian. Ontop, developed at the Free University of Bozen-Bolzano and distributed under an open licence, exposes the content of relational databases as a knowledge graph without moving the data, and in recent versions also connects to the main cloud analytical platforms. We mention it as an example of the approach and not as a product recommendation, because the choice of tool depends on what the company already has.

    The advantage of this set-up is that it separates two different kinds of work. The conceptual model and the mappings are an asset of the organisation, one that survives a change of platform. Execution stays with the engines that already do it well. The cost lies in writing and maintaining the mappings, which is exactly the reconciliation work described above and which no architecture lets you avoid.

    The graph on top, the data where it is: ontology, mappings, instances and governance decisions

    The governance decisions

    Whatever the technology, an enterprise knowledge graph raises questions the vendor cannot answer.

    Who owns it comes first. An ontology that spans sales, operations and finance belongs to none of the three functions, and without an owner with a cross-functional mandate each area maintains its own version. The writing rules come straight after: if agents can add entities and relationships as they work, someone has to decide which proposals enter the graph automatically and which wait for a person's approval.

    Then there is the matter of who sees what. The same node may hold information that a salesperson may read and an external supplier may not, and a graph that connects everything makes it very easy to reach by a side route data one would not have direct access to. Permissions have to be carried into the graph with the same granularity they have in the source systems, a subject we touched on in RAG versus connectors.

    Finally there is maintenance. A graph describes the company at a given date, and the company changes. It takes a process for retiring entities that no longer exist and an indicator of how much of the graph has been verified recently.

    A starting perimeter

    In the projects we follow the advice is the same we give for the ontology: one process, twenty entities, one owner. You pick a business question that today requires consulting three systems and two people, model the entities involved, write the mappings to the tables that hold them and measure whether an agent can answer with the same result as an analyst.

    On that narrow perimeter the real problems show up at once: the field names nobody can interpret any more, the two customer master files that do not match, the missing owner. Solving them there costs little and produces a method that can be extended to the next process, as we describe in the article on the AI Company OS.

    In the AI Rating, the presence of a shared model of entities, with an owner and an update procedure, is one of the elements of the Readiness dimension. It is rare to find it complete, and common to find organisations that bought a graph platform before deciding who governs its content. To work out which process to start from you can book an assessment meeting or write to us at hello@zerofive.ai.

    Written with the support of Claude, reviewed and approved by Fabio Lalli on November 11, 2026. How we use AI

    Frequently asked questions

    What is the difference between Graph RAG and an enterprise knowledge graph?

    Graph RAG extracts entities and relationships from a set of documents to answer questions about the whole archive. An enterprise knowledge graph represents business entities and connects them to the data in the systems.

    Why not load all the data into a graph database?

    Because these engines are effective on questions about relationships and much less so on aggregations and bulk reads, and because replicating the data platform means keeping two copies aligned.

    What is a virtual knowledge graph?

    A graph in which the data stays in the source databases. The ontology is connected to the tables through mappings, and questions are translated into queries on the underlying systems.

    Who should own the graph?

    An owner with a cross-functional mandate, because an ontology that spans several functions belongs to none of them. Without that role each area maintains its own version.

    Where is it best to start?

    From one process, about twenty entities and one owner, measuring whether an agent answers a business question with the same result as an analyst.

    Want to discuss this for your company?

    30 minutes with us to figure out where to start, or an AI Rating to measure your starting point.

    #knowledge graph#Graph RAG#ontology#data platform#data governance#AI Rating
    Share

    Keep reading