From complexity to clarity
Graphs with complex data structures, featuring various entity types and relationships, can hold a wealth of information—some intentionally modelled, others hidden and undiscovered. To illustrate, consider a city logistics scenario where ANPR cameras are strategically positioned to monitor vehicle movements in and out of specific zones. These cameras capture key data, such as the timestamp and license plate, uniquely identifying each vehicle. Additional details, like fuel type and emission class, can be linked to the registered vehicles, while company-owned vehicles bring in more layers of insight by adding information about their parent companies. This enriched dataset enables the analysis of traffic patterns by vehicle type, emission levels, or fuel usage, and even reveals which industries frequent the area most and when. With such granular insights, informed decisions can be made to design low-emission zones or even transition to entirely car-free areas.
So, how does that translate to a graph based digital twin representation?
In Figure 1, a snippet of the graph designed for the City Logistics use-case is illustrated. This includes sensor nodes that serve as digital twins of physical ANPR cameras monitoring vehicle movements. Each sensor node has a madeObservation relationship connecting it to observation nodes, which capture details of specific observations, such as observation time. These observation nodes are further linked via observedVehicle relationships to the observed vehicle’s digital twin. Each vehicle node is enriched with attributes like fuel type and emission class. The real graph also features company nodes (not depicted in Figure), representing vehicle-owning companies, each characterised by details such as the sectors they operate in and their principle place of business. The hasCompanyOwner relationship links vehicle nodes to their respective company nodes.
Exploring such a complex graph can be daunting. You need to master the query language to interact with the database and understand the graph’s logic. It feels far more intuitive for most people if the graph could be queried in one’s native language. This makes the graph accessible to a much broader audience, including those without technical expertise.
This is where Large Language Models (LLMs), or AI Agents, come into play, making it possible to query information naturally.
Large Language Models and AI Agents
This section aims to clarify the basics and key characteristics of LLMs and AI Agents, highlighting their differences, the necessity of AI Agents, and their role in retrieving information from complex graphs.A Large Language Model is an Artificial Intelligence that processes and generates human-like text. It’s trained on vast amounts of text data, learning patterns, grammar, and context. When you ask a question, the LLM uses this knowledge to predict and generate a relevant response based on mathematical and statistical methods to find the most relevant answer. It doesn’t have access to new information beyond what it was trained on, so its responses are limited to the scope of the training data.
While interacting with an LLM might feel natural, it’s important to remember that it doesn’t truly understand text like humans do. Its coherent and contextually relevant responses aren’t the result of logical reasoning, and it lacks genuine comprehension or awareness of the questions or its answers.
Although disruptive technology, that many of us are now using daily (sometimes unaware), there are also notable disadvantages to consider.
As mentioned, an LLM’s “knowledge” is limited to the data it was trained on. Therefore, it does not include the most recent information, nor will it have access to private or restricted data.
The lack of transparency in how LLMs substantiate their answers can make it challenging to verify the accuracy and source of the information. Furthermore, LLMs can sometimes generate incorrect or nonsensical information, a phenomenon known as hallucination.
To avoid a thorough technical deep-dive, one can view an AI Agent as a project manager in a large office, coordinating a team of workers that each can handle a specialised task efficiently. The LLM can be viewed as a knowledgeable assistant, excelling at understanding and generating human-like text. When a request comes in, the project manager assesses it with the help of the LLM and determines which specialised workers are needed to answer the request. The project manager then assigns tasks to the appropriate workers, who perform their specialised task. The responses of the workers are collected by the project manager, and together with the LLM checked on their relevance with respect to the overall request. Finally, a natural language result is formulated and returned to the user. This coordinated effort ensures that the request is handled efficiently and accurately, utilising the strengths of both the LLM and the specialised tools to achieve the desired outcomes. The worker inputs and results can also be passed to the user for verification and traceability purposes. This technique also hugely diminishes hallucinations.
To illustrate: the earlier presented “1 + 1 =” request will now be recognized by the AI Agent and LLM as a calculation that is best handled by the specialised Calculator worker. The returned result, namely the integer 2, is than packed into a human readable answer like “The result of the calculation 1 + 1 is 2”, indicating that the Agent has interpreted the string as a calculation.
Modern AI systems like Copilot or ChatGPT have evolved beyond being just pure LLMs. They function more like AI agents, equipped with access to various specialised tools and APIs. For instance, Copilot can perform web searches, access databases, and use other APIs to gather real-time information, enhancing its ability to provide up-to-date and substantiated responses. This allows them to provide more accurate and contextually relevant answers by leveraging these additional resources.
AI Agent and Graph Databases
In the previous section, we explored how AI Agents can access external sources of information. One such source could be a graph database containing information that was not used for training. This could be due to several reasons:
- The data wasn’t available at the time of training.
- The data cannot be exposed to others.
- Query results returned by the AI Agent need to be traceable and verifiable.
As previously explained, the AI agent, aware of the various tools at its disposal, informs the LLM (knowledgeable assistant) about these tools, their tasks, and their interfaces. For a graph database, query access is the most flexible access method, allowing full access to the graph content using a query language like SPARQL. SPARQL, a standard for over 16 years, is well-documented and widely discussed, making it familiar to the LLM.
The LLM can construct correct SPARQL queries based on a request. However, it needs to know what content is stored in the graph and how it is structured. This is where ontologies come in. Ontologies provide semantic information about the content and its structure. Therefore, along with a description of the database, the ontology is provided as input to the LLM, enabling it to understand the information stored in the graph and its organisation.
Let's consider the following request based on the previously sketched City Logistics example:
Provide the number of times vehicles of each fuel type are observed per day of the week. Display the results in a line chart. Put the days of the week along the x-axis and the number of observations along the y-axis. Use the English labels where you can.
While I don't claim to fully understand how an LLM provides its answers (since I use reasoning, which LLMs don’t), let’s try to roughly explain what happens within the AI Agent and its LLM companion:
- The request is passed to the AI Agent, which shares it with the LLM along with information about a SPARQL endpoint that provides access to data described by the provided ontology.
- The LLM extracts key elements from the query, subjects such as “observations”, “vehicles” and “fuel type”, actions such as “provide number of times”, and time-related elements “per day of the week”. It then determines whether it has enough knowledge and information to answer the question itself or if one of the tools provided could provide more info.
- Since the ontology provides human-readable labels and comments, where terms like Vehicle, Fuel type, and Observation are given, a match with the key elements is easily found. It then decides to use the SPARQL endpoint and constructs a query. The ontology provides enough information on the query structure.
- The SPARQL query is handed to the AI Agent, which executes it. Assuming everything goes smoothly, the result is shared with the LLM along with the original request. This request specifies a line-chart as the preferred presentation format. By providing the LLM with information on how to format table, line-, bar- and pie-charts (JSON, XML or in another format), the delivered result will be consistent and can immediately be used for visualisation.
This is an oversimplified and likely not entirely accurate representation of the real process. Also, in general, the process is iterative, where subsequent queries (hopefully) converge to a satisfactory answer. Proper error handling is also crucial but not discussed here.
To conclude and demonstrate that an AI Agent indeed can handle such queries, below is the generated result by our Royal HaskoningDHV Automate AI Agent. Through the use of LLM’s we are able to understand our data more clearly, and create more valuable digital twins. Get in touch to discover more about the work we are doing and how it can support your digital twin aspirations!
Do you want to know moreor have a question?
Contact our experts!