Semantic Layer for Hybrid Multicloud: Beyond the Metrics Layer
In a Data Science Innovators Conference 2023 keynote, Stardog's Navin Sharma said metrics layers capture only part of what a semantic layer can do. He argued a knowledge graph-powered layer that federates data, infers relationships and keeps business meaning can unify hybrid multicloud data.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Metrics layers are limited. Sharma said BI semantic layers focus on consistent KPIs and resemble a logical data warehouse.
- Meaning over storage. A knowledge graph layer lets people find data by business concepts such as customers and orders.
- Federate, don't copy. Data can stay at source, with queries translated to optimized SQL.
- Knowledge graph is not a graph database. Sharma cited federation, open standards, abstraction and inferencing as differences.
- Know the limits. It is not meant for OLAP-style aggregations.
In a keynote at the Data Science Innovators Conference 2023, Navin Sharma, VP of Product at Stardog, argued that a semantic layer for hybrid and multicloud data should go well beyond the metrics layer familiar from business intelligence. His proposal was a semantic layer powered by an enterprise knowledge graph. Sharma, who the host said had also served as VP of product at Precisely (formerly Syncsort) and Pitney Bowes, spoke as a vendor executive, and his product views reflect Stardog's approach.
Why is enterprise data still hard to use?
Sharma said data work has become harder as organizations moved from on-premises systems to multiple clouds while keeping much of their data and applications on-premises. Data is no longer only structured. It is unstructured, semi-structured, relational or graph-based.
He said data management practices have not kept pace. Pipelines are still built on brittle ETL, processing is pushed into one cloud or another, creating proprietary lock-in, and updates are slow. Self-service is limited because specialized skills are still needed, data sprawl creates governance problems, and consumers cannot see context across domains. Taking SAP data out of SAP, he said, strips away its context.
He cited an Accenture study of executives that found only 32 percent were positioned to take advantage of the data available to them.
What is wrong with a metrics layer?
Semantic layers are not new, Sharma said. In business intelligence they have existed for about 20 years, built on the relational paradigm and useful for high-speed aggregates. But they focused on consistent KPIs and metrics, an idea he linked to the recent metrics layer led by tools such as dbt, MicroStrategy and Tableau. In his view, that is a logical data warehouse. It delivers value but limits what an organization can get from a semantic layer.
What is a knowledge graph-powered semantic layer?
Sharma's alternative is finding and accessing data by its business meaning. People in the business think in terms of customers, suppliers, products and orders, he said, not strings and columns in tables. A semantic layer that keeps meaning and relationships separate from where data is stored makes it easier to make decisions.
He noted that Google coined the term "knowledge graph" with its 2012 search changes and said companies such as Facebook and LinkedIn structure data this way, as do assistants such as Alexa. He cited Gartner's 2023 impact radar, which described knowledge graphs as becoming mainstream within one to three years, and a Gartner finding that organizations using graphs and semantic approaches for natural language projects would have 75 percent less AI technical debt. With large language models and ChatGPT in the news, he said, knowledge graphs can serve as a foundation for using enterprise data with LLMs.
According to Sharma, a transformative semantic layer should do several things:
- Abstract data models from where the data is stored
- Reuse schemas and ontologies across use cases
- Federate access over physical data, so data stays in place
- Apply business logic as inference rules, such as inferring that two children of the same parent are siblings
- Treat relationships as first-class citizens, which he said relational databases do not
Which industries were using the approach?
Sharma gave examples from the time of the talk. A large pharmaceutical company, Boehringer Ingelheim, he said, was speeding drug discovery by letting researchers reuse past research and combine it with preclinical and clinical R&D information. Financial services firms were using it for credit risk, operational risk and anti-money laundering, following the money trail across connected data. He also said NASA's mission to the Moon relied on a knowledge graph foundation that captures systems engineering data from early flight modeling through test, validation and launch.
How does a knowledge graph differ from a graph database?
Answering audience questions, Sharma said a graph database requires moving data into the database. An enterprise knowledge graph platform can leave data at its source, map it at the metadata level and translate queries into optimized SQL for each source at query time. He also listed open standards, an abstracted data model and inferencing, meaning relationships need not all be defined explicitly, as differences.
On data lakes versus data mesh, he said a lake is a cheap source of information that lacks context, while in a data mesh a semantic layer can define domain-specific data products with their own rules, quality constraints and access policies.
Asked about downsides, Sharma said a semantic layer is "not the answer to everything." He said it is not meant for OLAP-style aggregations for BI, which relational semantic layers handle better, and that a knowledge graph platform has to interoperate with existing tools such as data lakes, catalogs and BI reports.
Where to follow semantic layer and context topics
The semantic layer has since become a regular theme at AI Connect events and webinars. The 2027 schedule includes 24 free live webinars in four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. Details are at /webinars/, and registration is coming soon.
Frequently asked questions
What is a semantic layer?
Sharma described it as a layer that separates business meaning and relationships from where data is stored, so people can find data by concepts like customers and products.
How is a knowledge graph different from a graph database?
A graph database moves data into the database. Sharma said a knowledge graph platform can leave data at source, map it at the metadata level, and infer relationships.
How does a semantic layer fit a data mesh?
He said it can define domain-specific data products with their own inference rules, quality constraints and access policies.
What are the limits of a knowledge graph semantic layer?
Sharma said it is not for OLAP-style aggregations for BI and must integrate with existing data stack tools.
This article is based on remarks by Navin Sharma, VP, Product, Stardog at Data Science Innovators Conference 2023 on April 12, 2023. More coverage: More AI Connect Events.
Filed under: Context, Semantics & RAG, Data & AI Infrastructure
