When Old Architectural Patterns Meet AI Realities

Many optimization patterns from the mainframe era are being revived in agentic AI systems. When a core resource is powerful but expensive and slow, we wrap it in caching, abstraction layers, routing, and edge computing. Today, that expensive core is LLM inference.

The familiar four moves are already visible:

1. Direct invocation (raw prompts)

2. Structured access layers (Model Context Protocol as the new standardized interface)

3. Multi-tier caching (prefix + semantic)

4. Edge and hierarchical routing (SLMs at the edge + routers to frontier models)

However, the analogy breaks in important ways. Cache invalidation shifts from deterministic to probabilistic, introducing staleness and access-control risks. State management flips polarity: mainframes were stateful (we built stateless layers around them); LLMs are stateless (we must now build heavy stateful infrastructure around them). Most importantly, the cost curve keeps falling rapidly — some scaffolding may eventually be absorbed by better models.

What tends to survive is not cost savings, but the deterministic properties that surrounding layers provide: reliability, compliance, auditability, and safety.

This Ledger Entry expands how readers think about systems architecture in the agentic AI era by showing that while classic optimization patterns from the mainframe era are being revived, critical divergences in state management, probabilistic caching, and rapidly falling model costs mean we must deliberately build layers that deliver reliability, compliance, and auditability — qualities that survive even as inference becomes cheaper and more powerful.