In collaboration withSAP
In the pursuit of adopting AI and demonstrating its value, businesses are accelerating efforts to implement agentic AI as co-pilots, assistants, and autonomous task managers. By late 2025, almost two-thirds of organizations were trialing AI agents, with 88% utilizing AI in at least one operational function, a rise from 78% in 2024, as reported by McKinsey’s yearly AI analysis. However, although initial trials frequently show promise, merely 10% of companies have managed to scale their AI agents effectively.
A significant challenge: the effectiveness of AI agents hinges on the quality of the underlying data structures. Specialists contend that many organizations encounter delays in AI implementation, not due to deficiencies in the algorithms, but because they lack data frameworks that provide the business context essential for human and agent utilization.
Organizations must prepare with the appropriate data architecture, and the coming months — or years at most — will be crucial, states Irfan Khan, president and chief product officer of SAP Data & Analytics.
“The only reliable forecast anyone can make is that we can’t predict what will unfold in the forthcoming years, months — or even weeks — with AI,” he emphasizes. “To achieve quick victories at this moment, you need to embrace an AI mindset and … underpin your AI models with trustworthy data.”
While data has always been vital for enterprises, its importance amplifies in the AI era. The capabilities of agentic AI will be dictated more by the robustness of enterprise data frameworks and governance than by model advancements. To effectively scale the technology, businesses must invest in a contemporary data architecture that offers context alongside the data.
Increased business context, not merely more data
Conventional perspectives often equate structured data with greater worth and unstructured data with lesser value. However, AI nuances that separation. High-value data for agents is characterized less by its format and more by its business relevance. Data relevant for essential business operations — like supply-chain management and financial forecasting — is context-dependent. While detailed, high-volume data, such as IoT, logs, and telemetry, can provide value, it requires accompanied business context.
Consequently, the true risk for agentic AI isn’t the absence of data, but the lack of grounding, asserts Khan.
“Any data that is business contextual will inherently offer greater value and reliability regarding business outcomes,” he states. “It’s not simply a matter of saying structured data is high-value while repeated data is low-value — both can hold immense value when leveraged correctly, which is a key distinction in AI.”
Context can be established through software integration, in-situ analysis and enrichment, or via a governance pipeline. Data that lacks these qualities is likely to be deemed untrustworthy — a significant reason why two-thirds of business leaders do not fully trust their data, according to the Institute for Data and Enterprise AI (IDEA). This “trust deficit” has hindered businesses in their pursuit of AI readiness. Addressing this trust gap necessitates shared definitions, semantic consistency, and dependable operational context to align data with business significance.
Data expansion necessitates a semantic, business-centric layer
In the last ten years, a pivotal transformation in enterprise data architecture has been the distinct separation of compute from storage, and the flexibility provided by cloud-scale, according to Khan. Yet, that separation and transition to cloud have also resulted in data sprawl, with information stored across various clouds, data lakes, warehouses, and numerous SaaS applications.
As organizations ramp up their AI initiatives, this sprawl does not diminish. On the contrary, it’s becoming a greater issue, with over two-thirds of companies recognizing data silos as a primary obstacle in AI adoption, and more than half of enterprises struggling with 1,000 data sources or even more. The previous phase focused on developing the foundation for software-as-a-service — separating compute from storage and constructing lakes — while the current phase centers on delivering suitable data to autonomous AI agents managing various business tasks.
“The most substantial innovation in data management was the dissociation of compute from storage,” Khan remarks. “What’s truly setting us apart now is how we harmonize data and extract value from it across various content sources.”
Achieving this requires a semantic or knowledge layer that accommodates multiple platforms, encodes business rules and relationships, provides a governed and business-contextual view of data, and enables both humans and agents to access data appropriately. Legacy data architectures are ill-equipped to support the autonomous AI systems of the future, as noted by consultancy Deloitte in its State of AI in the Enterprise report. Only 40% of businesses believe their data management processes can support AI, a decline from 43% the prior year, indicating that as organizations investigate AI implementation, they recognize their infrastructure’s limitations.
Agentic AI does not eliminate SaaS
Some investors and tech innovators propose that AI agents could render SaaS applications obsolete. Khan fundamentally disagrees. Over the past 15 years, value has progressively ascended through the stack, from on-premises infrastructure to infrastructure as a service (IaaS), to platform as a service (PaaS), and onto SaaS. Agentic AI represents merely the next tier. Agentic AI will establish its own layer to access data and engage with business logic. The value continues to rise, yet nothing below it ceases to exist, he asserts.
“SaaS remains relevant,” he insists. “It simply means that SaaS and these agents will work collaboratively. Companies are not likely to discard their entire general ledger and swap it for an agent. What can the agent possibly do? It lacks comprehension without business context and processing.”
In this developing paradigm, the software stack is being reconfigured so that applications and data furnish governed context within which AI can operate efficiently. SaaS applications continue to serve as the systems of record, while the semantic layer emerges as the business-context source of truth. AI agents evolve into a new engagement layer, facilitating interaction among systems, with both humans and agents acknowledged as “first-class citizens” in how they access business logic, he explains.
Importantly, agents cannot directly interface with every operational system. “If we’re claiming that agents are set to revolutionize everything … it’s unfeasible for an agent to communicate with every operational backend system,” Khan explains. “It simply isn’t feasible.”
This underscores the necessity for a semantic or business-fabric layer.
Initiating the process
The majority of enterprises should commence where their data currently resides — within platforms like Snowflake, Databricks, Google BigQuery, or an existing SAP system. Khan observes that this is standard practice but advises against falling into previous vendor lock-in patterns.
He recommends that organizations focus on the most crucial data by prioritizing the preservation and provision of business context for operational and application data. Enterprises should also invest early in governance and semantics by establishing shared policies, access regulations, and semantic frameworks prior to scaling pilot projects. Lastly, businesses should advocate for openness and interoperability in fabric-style rather than attempting to consolidate all data into a single stack.
Khan cautions against striving for complete automation too soon. “There exists a fresh, bold opportunity to engage with the agentic and AI landscape,” Khan mentions, “Fully automating [key business processes] may be overly ambitious, as significant oversight will be required.” Initial successes will likely derive from less-critical processes and from agents utilizing recent, stateful data instead of outdated dashboards, he adds. As AI begins to provide value and adoption grows, leaders will need to determine how to reinvest those benefits to enhance top-line efficiency or explore new markets.
Sign up for “The Fabric of Data & AI” online event scheduled for March 24, 2026. Gain insights from executives and thought leaders shaping the trajectory of data and AI.
This material was developed by Insights, the custom content division of MIT Technology Review. It was not authored by the editorial team of MIT Technology Review. It was researched, crafted, and produced by human writers, editors, analysts, and illustrators. This includes survey creation and data collection for surveys. Any AI tools possibly employed were confined to secondary production procedures that underwent thorough human evaluation.
