AI data centres are now being throttled not by a shortage of compute, but by how fast memory can feed data to the chips already installed. More GPUs should mean more throughput. Instead, accelerators are spending growing fractions of their time waiting for data, and the bottleneck is memory bandwidth.
This is the analytical event behind Bernstein’s research published on 29 August 2026: a structural constraint the firm argues reaches well beyond high-bandwidth memory (HBM) to encompass conventional DRAM, NAND flash, and the full range of storage technologies. Most market attention has concentrated on GPU vendors and HBM suppliers. The broader memory stack, the layers that training, inference, enterprise retrieval, and autonomous agent workloads all draw on, has been underappreciated as a structural beneficiary.
Here is a workload-by-workload breakdown of where memory pressure is actually being created, which segments of the memory stack each workload stresses, and how you can assess where structural demand growth sits beyond HBM.
Why AI systems are running out of memory before they run out of compute
GPU performance has scaled dramatically. Memory bandwidth has not kept pace. The result is what engineers call the “memory wall”: the widening gap between how fast accelerators can process data and how fast memory can deliver it, a condition where systems spend growing fractions of time idle, waiting for data rather than computing.
That gap is not intuitive. You add more GPUs, you expect proportionally more throughput. But when the memory subsystem cannot keep up, accelerator utilisation falls below its theoretical ceiling. The chips are there. The data is not arriving fast enough to use them.
DRAM supply constraints extend well beyond any single buyer’s capital budget, with SK Hynix projecting a global shortage lasting through 2030 and all three major producers fully sold out through 2026, a structural scarcity that reinforces why the memory wall is not an engineering problem money alone can resolve.
Why Bernstein is focused on this now
Bernstein’s central argument is that AI infrastructure spending is increasingly driven by memory rather than logic semiconductors, and that this reframes where you should be looking as an investor.
“Memory is more important to AI than logic semiconductors.” — Bernstein, August 2026
The firm describes the global memory market as in a strong price upcycle driven by sustained AI demand and constrained supply, with the cycle extending through 2026-2027. Bernstein estimates that higher HBM, DRAM, and NAND prices could lift hyperscaler data centre capex by approximately 30%. The firm views pullbacks in memory names as attractive entry points.
The memory wall is not a temporary engineering inconvenience. It is the structural condition that makes the entire memory supply chain, not just HBM, an investable theme. That framing matters before you engage with any workload-specific data.
When big ASX news breaks, our subscribers know first
Four AI workloads, four distinct memory stress patterns
Each major AI workload type places distinct pressure on a different layer of the memory stack. Understanding which workload stresses which tier gives you a framework for evaluating supplier positioning that goes beyond headline GPU or HBM narratives.
Training
Training remains HBM-led and bandwidth-bound. Large models need massive parallel data throughput during training runs, and HBM sits closest to the accelerator, delivering the bandwidth those runs require.
But training is not storage-indifferent. At scale, training operations draw on system DRAM for intermediate caching, locally attached SSDs for staging datasets ahead of computation, and networked storage to handle checkpoint management (the periodic saving of model state so that a failed training run does not need to restart from zero). HBM dominates, but the full stack participates.
Inference and the KV-cache problem
Inference, where a trained model generates outputs in response to prompts, splits into two distinct phases with fundamentally different memory profiles.
During the prefill phase, the model takes in a prompt and produces its initial output token, a step that is primarily compute-bound, with GPU and HBM resources doing most of the work. The decode phase is different. Here the model stores prior tokens in what is called a KV-cache, with each token added to a running record of the context processed so far. The memory footprint this cache demands grows in proportion to both how long the context window is and how many users are being served simultaneously.
According to Bernstein, in large-scale deployments the memory occupied by KV-cache can match or surpass the memory needed to hold the model weights themselves. That pressure pushes systems toward conventional DRAM offload, making inference structurally bullish for DRAM, not just HBM.
RAG: the enterprise default
Retrieval-augmented generation (RAG) is the practical mechanism through which enterprises integrate AI with their own proprietary data. Rather than relying solely on a model’s static training, RAG retrieves relevant documents from a company’s own databases and feeds them to the model at inference time. Enterprise-focused analyses increasingly treat RAG as the default pattern for production AI deployment, not a niche technique, because static model weights alone cannot satisfy freshness, compliance, or proprietary data requirements.
RAG activates a four-layer memory and storage stack:
| Tier | Technology | RAG role |
|---|---|---|
| Hot accelerator memory | HBM / on-package SRAM | Active model inference |
| System working memory | DRAM | Active query sets, frequently accessed indexes |
| Retrieval and active context | SSD / NVMe | Indexed corpora, embeddings, active retrieval |
| Cold archive | HDD | Raw corpora, logs, infrequently accessed data |
Bernstein notes that assembling RAG databases places heavy demands on SSD or HDD capacity alongside system DRAM, with DRAM’s contribution growing more significant once those databases are under active query load. SSD and DRAM are the structural beneficiaries of RAG adoption at scale.
Agentic AI and the orchestration footprint
Agentic AI refers to autonomous AI agent systems that can take multi-step actions, call external tools, query databases, and pass results between pipeline stages without human intervention at each step. In Bernstein’s view, agentic architectures are likely to drive substantially higher memory consumption overall, as they must hold intermediate outputs across steps, coordinate with external tools, and shuttle results through chains of specialised agents.
The orchestration layer is where the memory footprint broadens. Multi-agent pipelines and tool calls lean on CPU cores and DRAM in standard servers to manage queues, state, and routing. This shifts some incremental AI capex toward richer general-purpose server and storage configurations, not just accelerator racks, broadening the beneficiary set beyond GPU vendors.
Enterprise AI adoption rates remain highly uneven, with an estimated 70-80% of enterprise AI pilots failing or stalling and agentic AI deployment sitting at just 17% among enterprises as of April 2026, figures that matter for memory demand forecasting because the RAG and agentic workloads driving the broadest memory stack consumption are still in early deployment stages.
Quick reference: four workloads and their primary memory stress tiers
- Training: HBM-led, with supporting demand across system DRAM and SSD
- Inference (decode phase): DRAM-intensive via KV-cache scaling
- RAG: SSD and DRAM for retrieval and active query serving; HDD for archives
- Agentic AI: Standard server DRAM and CPU memory for orchestration and state management
The layered picture tells you that demand for DRAM, NAND, and HDD is not a secondary consequence of AI growth but a primary one. A portfolio that only captures HBM is structurally underweight the memory opportunity.
The economic case for the full memory stack, not just HBM
The technical mechanics explain where demand sits. The supplier economics explain why the opportunity may be larger than most investors expect.
Bernstein estimates that conventional DRAM generates more than twice the revenue and nearly three times the gross profit per wafer compared to HBM.
That finding is counterintuitive. HBM captures most of the investor attention. But the financial centre of gravity for memory suppliers sits in conventional DRAM, and this creates a structural incentive to balance product mix carefully. Suppliers are not going to flood HBM capacity at the expense of the more profitable conventional line. The implication for you: HBM supply tightness is partly a deliberate economic choice, not purely a capacity constraint. That framing changes how you should read supply forecasts.
Bernstein also estimates that higher HBM, DRAM, and NAND prices could lift hyperscaler data centre capex by approximately 30%, with a caveat: higher memory prices can redistribute economics across the AI supply chain in ways that may squeeze weaker suppliers. Supply chain positioning matters as much as sector-level exposure.
The memory chip price forecast confirmed by S&P Global Ratings extends through at least 2028, with Samsung, SK Hynix, and Micron collectively controlling roughly 89% of global DRAM supply and all three redirecting capacity toward high-margin HBM, a combination that structurally tightens conventional memory availability even as AI workload demand for it accelerates.
The architectural response to the memory wall is already showing up in commercial server roadmaps. Two categories of technology are worth distinguishing:
| Technology | Category | Primary use case |
|---|---|---|
| CXL-based memory pooling | Near-term | Disaggregated, shared DRAM pools across compute nodes; KV-cache capacity relief |
| Software tiering (Nvidia Storage Next, CMX context storage) | Near-term | Dynamic tiering of context and KV-cache across HBM, DRAM, and SSD |
| High-bandwidth flash | Speculative | Flash at HBM-like bandwidth; significant technical obstacles remain (Bernstein) |
| Processing-in-memory / 3D stacking | Speculative | Bringing computation closer to data; directional signal, not near-term catalyst |
CXL (Compute Express Link) is a high-speed interconnect standard that allows memory capacity to be disaggregated from individual servers and shared across compute nodes. As KV-cache pressure intensifies, CXL-based memory expansion grows the addressable pool of DRAM that AI workloads draw on, effectively turning conventional DRAM into a larger and more flexible resource for inference and enterprise retrieval.
Mapping the memory stack to the investment opportunity
The workload mechanics and supplier economics above convert into a tiered framework you can use to assess where structural demand growth sits across the memory supply chain.
| Memory tier | Primary AI workload driver | Demand outlook | Illustrative segment types |
|---|---|---|---|
| Accelerator memory (HBM, on-package) | Training, inference prefill | Strong pricing, tight supply through 2026-2027; Samsung-Broadcom collaboration through 2030 as strategic signal | HBM producers, advanced packaging |
| System DRAM (conventional, CXL-attached) | KV-cache growth, RAG active queries, agentic orchestration | Structural beneficiary; more than twice the revenue and nearly three times the gross profit per wafer vs HBM (Bernstein) | DRAM producers, CXL ecosystem participants |
| NAND/SSD and HDD | RAG retrieval, model checkpoints, dataset staging, cold archives | Demand rises structurally with RAG adoption and training pipeline scale | Enterprise SSD suppliers, HDD manufacturers |
| Packaging, interconnect, ecosystem | Data movement optimisation as memory wall intensifies | Increasingly important as architects optimise data movement, not just raw capacity | Substrate manufacturers, CXL ecosystem, interconnect suppliers |
These segments are illustrative of the supply chain landscape, not a recommendation regarding specific securities. The framework’s value is structural: understanding which workloads are growing fastest tells you which tiers to weight most in any assessment. A reader who has absorbed the workload mechanics and supplier economics can now map them to segments in a structured way, rather than reacting to headline GPU news.
This article is for informational purposes only and should not be considered financial advice. Investors should conduct their own research and consult with financial professionals before making investment decisions.
What the memory wall means for how you assess AI infrastructure from here
AI’s memory bottleneck is workload-specific and tier-differentiated. The analytical work is not in deciding whether memory matters; Bernstein has made that case clearly. The work is in understanding which workloads are scaling fastest and which memory tiers they stress most.
Bernstein views pullbacks in memory names as attractive entry points and anchors the current upcycle through 2026-2027, with the 30% potential capex uplift figure indicating memory’s growing share of AI economics. But the firm also flags that higher memory prices can redistribute economics in ways that squeeze weaker suppliers. Supply chain positioning matters.
AI supply chain power dynamics have shifted meaningfully toward platforms and away from suppliers, a pattern visible in the July 2026 session where Apple’s evaluation of Chinese memory alternatives sent Micron and SanDisk each down more than 10% while Nasdaq 100 futures gained, illustrating that supply chain positioning carries buyer-side risk as well as the structural demand tailwinds Bernstein identifies.
Three variables to watch as the memory upcycle continues:
- AI capex trajectory: Whether hyperscaler and enterprise AI spending sustains at or above current levels, or whether macro conditions force deferrals
- Supply discipline across HBM and conventional DRAM: Whether producers maintain the product-mix balance that keeps pricing supportive, or whether capacity additions outrun demand
- Pace of enterprise RAG and agentic AI adoption: The two workloads that most broaden demand beyond HBM into conventional DRAM, SSD, and standard server configurations
The memory opportunity is not a one-time event tied to a single training cluster build-out. It is a multi-year, workload-driven demand cycle. The investors who benefit most will be those who can read the stack, not just the headline GPU numbers.
Past performance does not guarantee future results. Financial projections are subject to market conditions and various risk factors. Nothing in this analysis constitutes a recommendation regarding specific securities.
