Why AI Agents Are Forcing Enterprises To Rebuild Their Data Foundations

Enterprises have spent years accumulating data. Making it usable for AI is proving harder.

Systems built for different teams, functions, and use cases have left information scattered across databases, applications, and workflows. For AI systems expected to reason across an organisation, having data somewhere is not enough. They need to find it, understand what it means, and use it in the right context.

The gap is becoming visible as Indian businesses move AI from pilots to production. According to a Dun & Bradstreet survey, 73% of surveyed Indian businesses report some measurable returns from AI investments, but only 4% say their enterprise data is fully ready to support AI at scale.

Closing this readiness gap means addressing years of accumulated data debt. Customer identities may be fragmented across systems, operational data may remain siloed, and information organised for reporting may not meet the needs of AI systems expected to retrieve it and act on it.

Bringing this scattered information together is only the first step. To make it usable for AI, enterprises must determine which data needs to be available in real time, how agents will understand its context, and what permissions should govern access. These decisions also shape the infrastructure and models needed to serve each use case, influencing whether the value of deploying AI justifies its cost.

How enterprises should approach these decisions was the focus of “Building The Data Foundation For AI At Scale”, a roundtable hosted by Inc42 in partnership with Oracle at its CTO Summit 2026.

The roundtable featured:

• Abhimanyu Saxena, cofounder, InterviewBit & Scaler
• Dhruv Gupta, cofounder, Orchard Healthcare
• Himanshu Garg, CTO, Kapture CX
• Naveen Dachuri, CTO, Yulu 
• Palanivel Saravanan, head of infrastructure cloud engineering, Oracle
• Priyanka Nahata, founder, CodeKarma Technologies
• Syed Mohd Bilal, head of data science, Perfios
• Vinay Murthy, senior director of product, Zeta

Moderated by Alexy Thomas, partner, technology consulting, EY India, the discussion examined how business needs should guide data architecture, agent access, and deployment costs.

Outcomes Before Architecture

For Orchard Healthcare’s Gupta, the starting point is the problem the business wants to solve. That means working backwards from the intended outcome, rather than adopting a technology and searching for a use case.

“The way we think about it is backwards. What is the problem to be solved?”

Gupta said his team followed this approach when building its customer data platform. The six-month project began only after the team established how it could improve customer understanding, retention, and the wider business.

Perfios’ Bilal echoed this emphasis on defining the outcome before choosing the tools.

“Start from an outcome focus and then figure out what to build rather than, you know, thinking about tools first.”

Agreeing on an outcome, however, still leaves enterprises with an architectural choice: build for that specific use case or invest in a platform that supports multiple applications. Bilal outlined the trade-off: use-case-led development addresses immediate needs, while a platform-first approach provides a broader foundation for future applications.

Zeta’s Murthy argued that AI is making the broader platform harder to ignore. As employees and systems begin creating agents, companies may struggle to anticipate every use case or the data each agent will require.

“What AI gets in is that the use cases are no longer in your control,” Murthy said.

This unpredictability puts greater demands on the underlying platform. It must serve known applications while allowing new agents to find and interpret data for needs the enterprise has yet to define.

Building Context Into Data

Supporting these new uses requires more than a database connection. An agent must understand how the information is organised, what it means, and how to retrieve it efficiently.

Oracle’s Palanivel pointed to information spread across databases such as Oracle, MySQL, PostgreSQL, and MongoDB, as well as unstructured knowledge. Connecting an agent directly to these systems introduces another challenge — it may not understand the database schema or generate queries that the database can handle efficiently.

The layer between the agent and the data therefore needs to interpret its structure and meaning and plan how to retrieve information.

Palanivel explained that traditional systems relied on humans to connect operational data with business decisions. For autonomous agents, some of that reasoning needs to be encoded into the data layer itself, making a semantic layer increasingly important.

“The semantic layer is only required or is becoming mandatory to support the reasoning for the agent,” he said.

Beyond helping agents interpret information, semantics can also make retrieval more focused. Perfios’ Bilal suggested that it can narrow the information an AI model needs to search through, helping it retrieve relevant data.

Creating this context requires input from people who understand the data. Bilal argued that those working closest to it are best placed to define its meaning, and that understanding should be consistent across tools and platforms.

But making information understandable does not mean every agent should be able to use it. Enterprises also need boundaries around whose data an agent can access and for what purpose.

Governing Agent Access

InterviewBit & Scaler’s Saxena illustrated this challenge with a question — if a model learns from Bank A’s data, can that knowledge be used for Bank B? Using enterprise data to train or improve AI systems, he pointed out, carries both legal and technical implications.

Similar questions arise within an organisation when employees use AI tools to work with business information. Kapture CX’s Garg argued that organisations cannot realistically prevent employees from using frontier models, making controls around that usage critical.

“So how do you build governance around this right? That is the most critical thing,” he said.

For Kapture, this means applying access controls to the knowledge and context graphs its agents use. Alongside these controls, Garg said the company is building agent harnesses to automate testing and evaluation.

Making The Economics Work

Permissions and testing address how agents operate, but enterprises must also establish whether each deployment delivers enough value to justify its running costs.

Yulu’s Dachuri distinguished between internal AI tools, where companies can set budgets, and AI embedded in customer-facing products, where usage and costs are harder to predict.

For Yulu’s bike-repair use case, the calculation is more measurable. The company can weigh reductions in manual touchpoints and turnaround time against the cost of deploying AI.

“So the value versus cost balance. It is most important,” he said.

Keeping this balance in check also requires choosing models suited to the task. Kapture, for instance, is using smaller models for predictable tasks, such as order-status queries, rather than relying on frontier models for every interaction.

Oracle’s Palanivel noted that organisations are moving away from a one-size-fits-all frontier-model approach, with open-weight models offering a more predictable cost structure for some workloads.

The choice of model, then, sits alongside the data and governance decisions that determine whether AI can work reliably in production. Enterprises need to provide relevant context, enforce access controls, and evaluate agent behaviour, while keeping infrastructure and model costs aligned with business value.

For businesses already seeing returns from early AI projects, scaling those gains means getting these elements to work together. The foundation must support both the use cases they can define today and the demands that new agents will place on their data.

The post Why AI Agents Are Forcing Enterprises To Rebuild Their Data Foundations appeared first on Inc42 Media.