How to Build the Architecture Enterprise AI Agents Actually Need

Enterprises are deploying artificial intelligence (AI) agents faster than they are building the systems those agents depend on. Business leadership now expects visible AI capability within one or two quarters, a fraction of the 18 to 24 months a core enterprise resource planning (ERP) rollout once took. That compression stems from competitive pressure, despite potential shortcomings in technical readiness. The result is a familiar pattern of disconnected pilots solving local problems without an enterprise strategy to guide them, leading to unnecessary costs.

Research from MIT’s Project NANDA found that 95% of generative AI pilots have little impact on profit and loss. The shortcomings, however, rarely stem from model capabilities. More often, the surrounding architecture was not built for autonomous action, leaving no clear answer as to which system holds authoritative data, what an agent may do, or how a decision is reconstructed afterward. Understanding the warning signs, the critical foundations, and the limits of architecture-first thinking gives teams a defensible project architecture.

Warning Signs That Agents Are Outpacing Architecture

The first sign of trouble often appears in the data itself. Multiple business units might pilot against the same data sources but with inconsistent field definitions. Then the marketing customer segment does not match the finance customer segment, yet both claim to be working from the same customer relationship management (CRM) system. According to Forrester’s The State of Agentic AI in 2026, companies are chasing agentic AI capabilities, but few are gaining the value they expected, and more than half report governance gaps that allow agents to proliferate unsupervised. A second warning sign emerges when no one can definitively answer which system is authoritative when a record exists in two places. The ERP system or the procurement system may hold the canonical vendor record, and nobody can definitively say which.

Perhaps the most telling warning sign is that security and identity teams are consulted only after a pilot already touches production data. This pattern of retrofitting governance creates the risk that architecture-first thinking can effectively prevent. The agents are already acting on real systems before anyone has established how they should be permitted to act.

Why Fragmented Data Blocks Autonomous Decisions

The same customer or vendor carries different attributes across ERP, CRM, and unowned departmental sources. Human reviewers often know which number to trust because they have institutional knowledge, context, and judgment. Agents lack that instinct unless it is built into their architecture. Gartner has identified three interdependent journeys that help data and analytics leaders scale AI, covering business outcomes, data and analytics capability, and behavioral change. The first journey calls for trust models that rate data by lineage and curation.

An authoritative-looking wrong answer costs more than no answer, because it clears the review process without being challenged. The agent does not know that the CRM’s annual revenue field might include only closed-won deals while the ERP’s version includes all contracted revenue, so it simply reads and acts, and the organization bears the consequences.

Foundations to Establish Before the First Agent Ships

Before deploying autonomous agents, organizations need to establish several critical foundations (Figure 1). The first is a trusted data layer: documented ownership for each entity type, so someone knows who is responsible for the data, when it was last updated, and which system serves as the system of record, paired with a shared vocabulary maintained centrally, because the same piece of information carries a different name in each system it touches. What the ERP calls a customer number, the CRM calls an account ID, and an agent that has not been given those mappings will treat them as two different customers.

The second is a delegated identity that carries a person’s varying access across systems, distinct from a static service account. It is crucial for an agent acting on behalf of a procurement manager to inherit that manager’s permissions and restrictions rather than operate with blanket system privileges. The third is policy and guardrails: the runtime checkpoints that decide, action by action rather than role by role, whether an agent may proceed on its own or must stop for a human. That distinction, between what an identity is allowed to touch and what a given action is allowed to do, separates secure agentic implementations from those that inevitably create security holes. The fourth is observability that captures the decision context and execution trace rather than the final action alone. If a transaction is disputed six months later, an auditor will want to see what the agent was asked, which systems it queried, what those systems returned, and who approved the result. If a log records only the outcome, everything else is left unanswered.

Building a Shared Foundation Rather Than Isolated Projects

Organizations that succeed with enterprise AI take a different approach than those that fail. They start by cataloging every pilot already running, including informal ones that may have begun as departmental experiments, to provide visibility into where agents are already operating and which shared infrastructure they would benefit from. Next, they build identity, logging, and a shared data layer once, centrally, which avoids the trap of repeatedly solving the same problems across different business units. Giving business units a paved path instead of solving data access from scratch accelerates deployment while maintaining consistency, since those units need a well-defined route to the data, in the form they need it, with governance already applied, and without having to build their own data engineering expertise.

In one enterprise implementation, an invoicing agent attempted to parse PDF invoices sent by email. The agent misread amounts often enough to undermine business confidence, because the documents arrived in varying formats, languages, and number conventions. Rather than investing in better parsing, the organization created an event-driven feed of structured data, keyed to a customer reference table. Based on transaction volumes in the initial rollout, the structured feed eliminated the need for AI-based PDF extraction for roughly 85%-90% of invoices. Feeding the agent a trustworthy, structured path restored reliability.

Where Architecture-First Thinking Goes Wrong

A platform built before a validated use case is a cost the business cannot easily justify, and architecture work drifting too far in that direction becomes endless preparation. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, with escalating costs and unclear business value among the reasons. The design problem compounds in enterprises that run several platforms at once, where an overarching architecture drawn up in advance tends to fit none of them especially well. The ERP and CRM teams, and half a dozen departmental owners each arrive with requirements that are legitimate on their own terms but irreconcilable inside a single design phase.

Narrower work still delivers value, and a single agent operating inside one platform, with defined boundaries and rules it follows every time, can satisfy authorization, auditability, and traceability before the wider architecture is implemented. Teams learn a great deal from running one of those in production, and the results provide finance with a reason to fund the foundation while a retrofit is still cheap. That sequence sits between deploying recklessly and preparing endlessly, which is where most organizations need to operate.

Getting the Sequence Right

The pressure to show AI results within short timeframes is real and will not ease. There is a difference in timelines worth considering. A demonstration takes weeks, while an agent that can run against production data, survive an audit, and be corrected when it errs takes longer to develop. Skipping that work may save time in the short term, but not indefinitely. The expense returns as redundant rework.

Organizations already running 20 or 30 agents can take inventory of what exists now, identify the data and identity dependencies those agents share, and consolidate them before the number of agents doubles. Organizations starting now can pick one agent with write access to a business system and settle three details before it goes live: Which system is authoritative for each piece of data it reads? What can it do without a human, and what stops it at a checkpoint? Which record lets an auditor reconstruct a decision from three months ago? Whatever cannot be answered is the architecture work that comes first, and it costs far less now than it will once those agents are running in production.

References

1. Estrada, S. (2025, August 18). MIT report: 95% of generative AI pilots at companies are failing. Fortune. fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
2. (2025, March 3). Gartner identifies three areas to help data and analytics leaders scale AI [Press release]. gartner.com/en/newsroom/press-releases/2025-03-03-gartner-identifies-three-areas-to-help-data-and-analytics-leaders-scale-ai
3. (2025, June 25). Gartner predicts over 40% of agentic AI projects will be canceled by end of 2027 [Press release]. gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
4. Hopkins, B., Le Clair, C., Pollard, J., Curran, R., & Joseph, L. (2026, June 3). The state of agentic AI in 2026: Companies are chasing, few are catching. Forrester. forrester.com/blogs/the-state-of-agentic-ai-in-2026-companies-are-chasing-few-are-catching/