What Is High-Bandwidth Memory, and Why Does AI Need It?

By Adam Patti, Chief Executive Officer, VistaShares | Published November 13, 2026

High-bandwidth memory (HBM) is a type of computer memory made by stacking several DRAM chips vertically and placing the stack right next to a processor, which lets data move between memory and chip far faster than with conventional memory modules. AI accelerators need that speed because training and running large models requires moving huge amounts of data every second. Only three companies produce HBM at scale, and in 2026 all of them described their output as committed or sold out, making memory one of the tightest links in the AI supply chain.

How does HBM work?

Conventional server memory sits on modules plugged into a motherboard, connected to the processor by a relatively narrow path. HBM takes a different approach. Manufacturers stack eight, twelve or more thin DRAM dies on top of one another, connect them with thousands of vertical wires called through-silicon vias, and mount the stack on the same package as the processor. The result is a very wide, very short connection.

A modern AI accelerator typically carries several HBM stacks arranged around the processor die. Each new generation, such as HBM3E and HBM4, adds layers, capacity and bandwidth. The memory and the processor are joined using advanced packaging, a specialized step that has itself been a constraint. Our article on the semiconductor supply chain explains where packaging fits.

Why does AI need so much memory bandwidth?

Large AI models contain billions or trillions of numerical parameters. During training and inference, the processor must repeatedly read those parameters and intermediate results from memory. If memory cannot deliver data fast enough, the processor waits, and expensive compute sits idle. Engineers call this the “memory wall.”

HBM narrows that gap. In many AI workloads, especially inference on large models, performance depends as much on memory bandwidth and capacity as on raw processing power. That is why each new accelerator generation tends to carry more HBM than the last, both in the number of stacks and in capacity per stack.

Why has HBM supply been tight?

Demand growth. TrendForce estimated that HBM bit demand grew more than 130% in 2025 and projected growth above 70% in 2026.¹

Wafer intensity. HBM consumes roughly three times as much silicon wafer capacity as standard DDR5 memory for the same number of bits, because the dies are larger and stacking reduces yields.² Every wafer devoted to HBM is a wafer not used for conventional memory.

Concentrated supply. Three manufacturers produce nearly all HBM. In its first-quarter 2026 earnings call, one of them said its HBM4 volume for the year was completely sold out, and another had said in October 2025 that its full 2026 supply was committed.²

Long build times. New memory fabs and packaging lines take years to build and qualify.

The squeeze has spread to other memory. TrendForce reported that conventional DRAM prices surged from the fourth quarter of 2025, and in September 2026 it forecast that blended HBM average selling prices would rise 121% in 2027.³ TrendForce also reported that some GPU and custom chip makers were considering reducing HBM capacity per device because of limited supply and rising system costs.³

How do memory makers decide how much HBM to build?

Memory makers balance HBM against conventional DRAM and NAND flash, which share fabs, equipment and engineers. HBM commands higher prices per bit, but it also uses more wafer area, requires extra packaging steps and must be qualified with each customer’s chip before volume shipments begin. Suppliers increasingly negotiate HBM volumes and prices with major customers a year or more in advance. These long-term agreements give suppliers visibility, but they also mean output for a given year can be largely committed before that year starts, leaving little room for new buyers.

How does HBM fit into the AI bill of materials?

HBM is a significant and rising share of the cost of an AI accelerator. When memory prices rise, the cost of each server rises with them, which can affect the capital budgets of cloud companies and the margins of chip designers. Several large buyers cited higher component pricing when they raised their 2026 capital spending forecasts earlier in the year.⁴ Our article on the AI data center bill of materials shows where memory sits among the other layers.

What are the risks for the memory industry?

Memory has historically been a cyclical business. Periods of shortage and high prices have encouraged heavy investment, followed by oversupply and price declines when new capacity arrived. HBM’s specialized processes and long customer qualification may change that pattern to some degree, but they do not remove it. Other risks include technology transitions between generations, yield problems, customer concentration and export controls that affect where chips can be sold.

For thematic investors, memory makers, equipment suppliers and packaging firms represent a distinct slice of AI infrastructure exposure, with economics different from those of chip designers. Our article on the AI supply chain from chips to power places memory within the wider chain.

Frequently asked questions

What is the difference between HBM and regular DRAM?

Both use DRAM cells to store data. Regular DRAM is packaged as individual chips on memory modules connected to a processor across a motherboard. HBM stacks several DRAM dies vertically and places them on the same package as the processor, creating a much wider, shorter connection. That design delivers far higher bandwidth per watt, at a higher cost per bit.

Who makes high-bandwidth memory?

Three manufacturers, based in South Korea and the United States, produce nearly all of the world’s HBM. Other memory makers, including some in China, have announced development programs. Because AI chip designers must qualify each supplier’s memory with each new chip, entering the market takes time even for experienced memory producers.

Why does HBM affect prices of other memory?

HBM and conventional DRAM are made in the same fabs, and HBM uses roughly three times the wafer capacity per bit. When manufacturers shift capacity toward HBM, less is available for standard memory used in servers, personal computers and phones. That reduced supply has contributed to higher conventional DRAM prices since late 2025, according to TrendForce.

What are HBM3E and HBM4?

They are successive generations defined under industry standards. HBM3E is an enhanced version of the third generation, widely used in AI accelerators shipping in 2025 and 2026. HBM4 doubles the width of the interface and supports higher bandwidth and capacity. Each generation requires new manufacturing steps and customer qualification, which affects which suppliers ship first.

Is the HBM shortage permanent?

Shortages in memory have historically been temporary, because high prices encourage new capacity. Suppliers are expanding HBM production, but new fabs and packaging lines take years. Industry forecasts in 2026 described tightness persisting into 2027. How long it lasts depends on AI demand, the pace of capacity additions and whether chip designers reduce memory per device.

How do thematic ETFs get exposure to memory?

Semiconductor and AI-themed ETFs may hold memory manufacturers, memory equipment suppliers and advanced packaging firms. Weightings vary widely. Some funds weight by market value, which can produce large positions in a few names. Reviewing a fund’s holdings and methodology shows how much memory exposure it carries and how concentrated that exposure is.

Cite this page: Adam Patti, "What Is High-Bandwidth Memory, and Why Does AI Need It?," VistaShares, updated November 13, 2026, https://www.vistashares.com/high-bandwidth-memory-explained/. Plain-text version.

This material is provided for educational purposes only. It is not investment, legal or tax advice, and it is not a recommendation or an offer to buy or sell any security. Information is drawn from sources believed to be reliable as of the date of publication, is not guaranteed as to accuracy or completeness, and may change without notice. Figures attributed to third parties are their estimates. Projections and forecasts are not guarantees of future results. Investors should consult their own financial, legal and tax professionals before making investment decisions. Investments involve risk, including possible loss of principal. Companies in a single industry or theme may be affected by common economic, regulatory or technological developments and may be more volatile than the broad market. References to companies, governments or programs are for illustration only.