A senior engineer hands the coding agent a task. Add a new field to the user object, propagate it through the order pipeline, make sure the downstream payment service still reconciles correctly.
The agent edits the user-service file it can see. It guesses at the payment service’s schema based on a comment that turns out to be two years stale. The PR compiles. Unit tests pass. The change ships. Three weeks later, payment reconciliation breaks on a transaction class nobody on the team had top of mind.
The bug is not about the agent’s coding ability. The agent wrote correct code against the wrong picture of the system. It never had a picture of the system at all. It had files.
This is the gap that defines the category. AI coding agents do not understand software architecture on their own, and the question of how they come to understand it has a specific technical answer.
What the agent actually sees
Set aside everything the marketing says about codebase awareness. A coding agent at the start of a task has three inputs and only three.
- The current file, loaded directly into the prompt
- Files retrieved by similarity, ranked against the query and pulled into context
- The model’s training data, which holds general patterns from public code but nothing about this specific system
That is the entire foundation Cursor, Claude Code, GitHub Copilot, and Codex reason from when they sit at the code layer. It is enough for a function-scoped task in a known file. The agent reads the file, understands the call, writes the change, and moves on.
It breaks the moment the task depends on knowledge the three inputs do not carry. Which service actually owns the user object. Why an abstraction was introduced two years ago and what it protects against. Which downstream consumer reads a schema field whose name nobody remembers changing. What runs in production at peak load.
The agent without this context guesses. The guesses look fluent. They are expensive at scale, and the cost shows up first as wasted tokens, then as the regression that ships into production.
| What enterprise tasks require | What file-level retrieval provides |
| System-level awareness across services | Per-file content |
| Reasoning behind past architectural decisions | None |
| Intent behind the design | None |
| Runtime behaviour of the system | None |
The gap between the two columns is the category. Architectural understanding in an AI coding agent means an external layer that fills the right column and keeps it current as the system changes. The rest of this piece explains what that layer has to model, how it gets exposed to the agent, and how a team verifies whether a vendor has actually built it.
Why this gap widens at enterprise scale
The failure mode is structural, not statistical. It scales with the size of the codebase, not the difficulty of the task.
Coding agents work inside a finite context window, between 128k and 2M tokens depending on the model. Enterprise monorepos run multi-million lines, with some past 10M. The agent holds a fraction of a percent of the system at any moment, monorepo or not. Without a structural map of what exists outside its window, it guesses which slice to load. Each wrong guess generates more file reads, more shell commands, more tool calls. Each of those burns tokens.
Our SWE-Bench Pro evaluation measures the gap on Claude Opus 4.6. With native file search alone, the agent resolves 51.9% of tasks. With system-level context exposed via Bito’s AI Architect, the resolution rate climbs to 70.1%. A 35% overall lift.
The headline number compresses on the cases that matter inside enterprise codebases.
- Repositories above 1.5M lines of code, 3.8x lift
- Changes spanning 10+ files, 4.5x lift
- Critical issues across performance, security, or cross-component boundaries, 4x lift
On substantial tasks, structural context cuts cost per task by ~47% in aggregate and up to ~68% on individual tasks.
The same gap produces a longer-tail problem that benchmarks do not measure. The codebase the agent edits gradually drifts away from the design the team intended, because each individual edit looks reasonable in isolation and the agent has no view of the system-level pattern it is breaking. We unpack that failure mode and the diagnostics that surface it in Why AI coding tools break architecture in large codebases.
So if the gap is real and the failure mode is structural, what closes it.
The external context layer
What the agent needs is not more files. What it needs is a model of the system that holds four kinds of knowledge at once.
- Code, what exists today. Repositories, services, modules, APIs, dependencies, the architectural patterns the team has actually used.
- Business context, the reasoning behind decisions. Tickets in Jira and Linear, incident history, the record of why things are the way they are.
- Tribal knowledge, the intent behind the design. Confluence pages, design documents, the Slack threads where the real argument happened.
- Runtime, what actually happens under load. Observability data, deployment configuration, database schemas, message-queue topologies.
Strip out any one of the four and a specific failure mode shows up in the resulting code.
Code without business context, and the agent reinvents conventions the team already settled six months ago in a thread the agent never read. Code without tribal knowledge, and the agent ships an implementation that fits today’s surface but contradicts a direction the team committed to in a design doc. Code without runtime, and the agent reasons about how the system should behave, not how it does behave when a thousand requests per second hit the cache.
These four signals have to fuse into something the agent can actually query. A text index does not work, because a text index returns passages and the agent ends up reconstructing relationships on every call. What works is a code graph.
From four signals to a code graph
A code graph is a typed, directional model of every service, file, function, and dependency in a codebase. Each entity is a node. Each relationship is an edge.
Embedding-based codebase indexing retrieves text chunks scored by vector similarity. The agent gets passages and has to infer how they connect. A code graph models the connections directly. The agent traverses from one entity to another along defined edges and gets the full path back in one call.
The difference becomes decisive on multi-hop reasoning. A question like “which services consume this schema and which of them handle the new field correctly” requires three or four hops. Passage retrieval makes the model reconstruct the chain on every call, burning context window space on intermediate results. Graph traversal returns the chain in one query.
The agent reaches the graph through MCP. The agent issues a query, the graph returns typed entities and relationships, the agent reasons over the result.
Bito’s AI Architect as one implementation
Bito’s AI Architect builds the graph by statically analysing every repository in scope, then ingesting business context from Jira and Linear and tribal knowledge from Confluence, Slack, and design documents. Every function, class, API, configuration, and schema becomes a typed node. Every call, dependency, and schema reference becomes a typed edge.
For each repo, the graph indexes 160+ properties across 15+ dimensions of how the system is built. These include:
- Service ownership and bounded contexts
- The dependency graph across REST, gRPC, and message-queue boundaries
- Architectural patterns and shared-library conventions
- Deployment topology and runtime configuration
- Database schemas and a cross-repo symbol index that resolves references in milliseconds
The graph stays current as the system changes. Repository webhooks trigger incremental updates as merges land. Jira and Linear updates flow in on a sub-minute cadence. A nightly rebuild re-derives every typed relationship from scratch.
AI Architect runs on cloud or on-prem. The code stays at the customer’s perimeter. The platform holds SOC 2 Type II.
Where the model proves itself: microservices
Knowing the four-signal model holds in principle is different from seeing it hold under load. The hardest place to hold it is inside a microservice architecture, because the reasoning surface spans services the agent will never load into context.
A change to one service can ripple to several others through APIs, shared libraries, event streams, or database contracts. Multi-repo microservices push the surface past what any single-repo embedding store holds. The agent needs three primitives the code graph supplies.
- Service dependency graph. Maps which services call which, across REST endpoints, gRPC contracts, database connections, and message queues. Detection runs through three mechanisms: static analysis of client SDK imports, OpenAPI and gRPC IDL parsing, and configuration-file parsing for connection strings and queue topics.
- Call-chain analysis. Traces an outbound call to its source and every downstream consumer. The agent sees the chain, not the endpoint.
- Change-impact propagation. Walks outbound dependency edges from the proposed change to every entity it touches. A function signature change traces to every call site. An API contract change traces to every consumer service. A schema change traces to every query that reads it.
Without these primitives, the agent sees one service clearly and the rest only through embedding fragments. On a task that crosses three services, the agent often modifies the one it sees and leaves the others inconsistent. The PR looks clean, and the integration breaks in staging if you are fortunate and in production if you are not.
In production, this is the exact pattern Kredivo’s engineering team used to cut cross-service technical design time from 2 weeks to 4 hours. The graph held the tacit knowledge of which services were involved in a proposed change. Engineers stopped sitting in design meetings reconstructing the dependency surface, and the design closed in a single working session.
The mechanics of how dependency analysis, call-chain tracing, and change-impact propagation actually run on real refactoring tasks sit in How AI coding agents navigate service dependencies in microservices.
Where the model proves itself, again: the architect’s view
Microservice mechanics are the technical proof. The architect’s view is the political proof, because architects evaluate AI coding tools on a criterion developers tend to underweight.
Developers judge a tool by velocity. The cost of a bad suggestion is a few minutes of rewrite, and a tool that produces good suggestions fast wins the daily test.
Architects judge a tool by architectural drift. The system gradually diverges from its intended design as the agent makes hundreds of edits that each look reasonable in isolation. The drift compounds. By the time it surfaces at the system level, the architect no longer recognises the topology, and the cost of the realignment is measured in quarters, not minutes.
The criteria diverge on three questions about how the tool handles the existing architecture.
- Does the agent respect existing architectural patterns, or invent new ones each time?
- Does the agent reuse shared libraries and contracts, or duplicate them silently?
- Does the agent route changes through canonical service boundaries, or cut across them?
A coding agent operating from file-level retrieval alone cannot answer any of these reliably. The answers live at the system and dependency layers of the architectural reasoning model, which means they require the external context layer. The full architect-side framework, including the questions to ask during a tool review and the signals to watch during a pilot, sits in What software architects need from AI coding tools.
How spec-driven development fits
There is a separate movement in the category worth addressing directly, because architects and engineering leaders ask about it during the same evaluation.
Spec-driven development captures intent at the front of the engineering loop. Engineers write specifications that describe what the system should do. AI coding agents convert specs into code. The argument is sound. Specs reduce the ambiguity that ad-hoc prompting introduces, and they give the agent a target to optimise against.
The blind spot is the existing codebase. A specification describes intent in the abstract. It leaves out the system the intent has to live inside. On a greenfield codebase, the gap stays small. On a large evolving codebase, the gap is where defects originate, because the agent translates intent into code without checking whether the code fits the patterns, contracts, and conventions already in the system.
The two approaches compose rather than compete. Spec-driven defines what to build. Architecture-aware ensures the build fits the system already running. Together, the agent captures intent at the front of the loop and respects the architecture at the back.
How to verify a tool actually has the layer
The piece so far makes a structural claim. A coding agent reasoning across the system, dependency, and code layers behaves differently from one reasoning at the code layer alone. The question for a team evaluating a tool is whether a given vendor has actually built the layer, or whether the marketing is describing it.
A short pilot answers this. Three components make the pilot conclusive.
Pick a non-trivial cross-service task. The pilot task should require changes across at least two services, touch at least one API contract, and reference behaviour described in tickets rather than only in code. A task that fits inside a single repository tells the team nothing about architectural understanding, because the code layer alone is enough for it.
Watch the agent in the first few minutes. The signals that separate a code-layer tool from a tool backed by an external context layer surface immediately.
| Signal during the pilot | What it tells you |
| Agent asks which services are involved | Operating at the code layer |
| Agent names the services and traces dependencies before editing | External context layer present |
| Agent proposes a service-by-service implementation plan | System-level reasoning present |
| Agent references decisions from tickets or design docs | Business context and tribal knowledge connected |
| Agent’s plan accounts for downstream services beyond the immediate edit | Change-impact propagation working |
Ask the vendor concrete questions before the pilot starts.
- How does the graph get built, what gets indexed, and how often does it update?
- Which integrations connect the graph to issue trackers, documentation systems, and observability?
- Where does the code sit during indexing and querying, and what certifications cover that posture?
- Which coding agents connect to the graph today, and through which protocol?
- How long does the initial graph build take on a multi-million-line codebase?
A vendor that answers each of these concretely has built the layer. A vendor that hedges on indexing cadence, deployment posture, or coverage breadth has built less than the marketing suggests.
What changes when the layer is in place
The technical claim is that the agent moves from the code layer to architectural reasoning. The organisational claim is what makes the technical claim worth the engineering investment.
Architecture-level intelligence stops sitting in the heads of the two engineers who actually know the system, and becomes available to every engineer on the team.
- The senior engineer who used to be the only one able to safely break down a cross-service epic becomes one of several
- The architect who used to spend their week answering questions about service boundaries spends it on the design work that requires architectural judgement
- The new engineer who used to need six weeks to ship safely ships in a fortnight
Three production examples make the pattern concrete. PubMatic compressed technical design from 2 weeks to 3 days. Kredivo cut cross-service design time from 2 weeks to 4 hours. Privado shipped enterprise SSO in 5 hours across 4 repositories, against a 10-day baseline.
The mechanism behind every one of these outcomes is the same. Code for what exists, business context for why, tribal knowledge for intent, runtime for what actually happens, indexed into a code graph that updates as the system changes, exposed to the coding agent through MCP, and consulted before the agent starts editing.
Bito’s AI Architect is one production implementation of this architecture, and the most direct way to see whether the model holds for a specific codebase is to run the pilot framework above against it.
