🔍 Read the full analysis: AI Chip Design In 3D: Dies, HBM Stacks And Their Package on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 22, 2026, explainer from ThorstenMeyerAI.com maps how current AI accelerators combine compute dies, stacked HBM memory, silicon interposers and liquid cooling. It reports that packaging capacity and memory bandwidth are constraints alongside compute, while the scale and cost of future supply remain uncertain.
ThorstenMeyerAI.com published an explainer on November 22 showing how AI accelerators combine compute dies, high-bandwidth memory (HBM), silicon interposers and cooling hardware inside a package—and how those packages are assembled into racks with 72 GPUs. The report argues that AI performance and supply depend not only on compute chips, but also on memory bandwidth and the ability to manufacture advanced packages.
The explainer uses schematic 3D models to show the package layer by layer, from a multilayer package substrate at the bottom to a liquid-cooling plate above. The publisher says the models reflect layer order and component counts, but not exact dimensions: Nvidia and AMD do not publish the sizes of the dies, packages or memory stacks in millimetres, and the models do not estimate them.
At the centre of the package, a silicon interposer provides dense connections between compute dies and nearby HBM stacks. The source describes HBM as DRAM chips stacked above a base die and connected vertically by through-silicon vias, or TSVs. TSMC describes its CoWoS-S packaging as using a large silicon interposer for logic dies and HBM. The report says CoWoS-S can accommodate an interposer up to about 2,700 square millimetres, or 3.3 times a lithography reticle, according to TSMC.
The report contrasts designs from Nvidia and AMD. Nvidia says its Rubin GPU has two reticle-limited compute dies and 336 billion transistors. AMD says its MI350 Series integrates eight vertically stacked accelerator-complex dies with two I/O dies; the MI355X has 185 billion transistors and eight stacks of 12-high memory, according to the source. AMD has also said its next-generation MI450X and MI455X use CoWoS-L packaging and stacked 3D hybrid-bonded compute dies.
Packaging Sets the Pace for AI
The explainer’s central point is that a fast compute die is only one part of an AI system. Models need data delivered quickly, making memory bandwidth and the connections between memory and processors key factors in performance. The source cites Nvidia product figures showing bandwidth rising from 900 GB/s for the V100 to 19.2 TB/s for Rubin. These are figures for different products and generations, not a measure of performance for every workload.
Packaging is also a manufacturing constraint. Interposers must connect many dies and memory stacks within a compact area, while taller HBM stacks place more memory close to compute. The report cites SK hynix’s expectation that demand for memory will exceed supply through 2030; that is the company’s forecast, not a guaranteed outcome. If advanced packaging or memory supply cannot keep pace with demand, accelerator production may be limited even when compute designs are available. The source’s suggestion that this can keep compute scarce and expensive is an interpretation of those supply constraints, not a quoted price forecast.
Rack design shows the scale at which these components are deployed. Nvidia’s GB200 and GB300 NVL72 systems combine 72 GPUs and 36 CPUs per rack, alongside networking, power and cooling equipment. This makes the package a building block for a larger system whose performance depends on connections and infrastructure as well as individual chips.
Products Worth Considering

Vari ComfortEdge 48×30 Inch Adjustable Electric Standing Desk

Apple Mac mini (2026) with M6 chip

Logitech MX Mechanical Mini for Mac Wireless Keyboard, Space Gray
As an affiliate, we earn on qualifying purchases.
From Silicon Dies to Full Racks
The package substrate connects the assembly to the module and system board, carrying power and signals. Above it, the interposer acts as a dense wiring layer; it is not another compute chip. A reticle is the maximum area a lithography scanner prints in one exposure. According to TSMC, its CoWoS-L approach joins silicon bridges in a carrier layer to support larger designs than the interposer approach allows.
There are signs of continuing expansion in package size, according to statements cited by the source. It reports that TSMC had 3.5-reticle CoWoS-L designs in volume production since 2024, and that on October 15, 2026, the company’s CEO described “real world production of 5x reticle size” while work continued on 9x and 14x. These are company statements about production and development, not evidence that every planned size is already commercially available.
HBM capacity also changes with stack design. The report says JEDEC’s April 2025 HBM4 standard allows stacks of 4, 8, 12 or 16 layers and up to 64 GB per stack. It cites SK hynix on thinning DRAM layers to fit 12 layers at the thickness of a previous eight-layer product, and Samsung on a seven-micrometre gap between chips in its 12-high HBM3E. Those company-specific examples do not establish that every supplier or product has the same configuration.
At rack scale, the source cites Nvidia’s DGX GB user guide: an NVL72 rack has 18 compute trays, each with two Grace CPUs and four Blackwell GPUs, as well as nine NVLink switch trays, management switches and power shelves. The cited Open Compute Project listing gives the GB200 NVL72 rack dimensions as 600 mm wide, 1,068 mm deep and 2,236 mm high. The report also says the current Vera Rubin NVL72 listing retains 72 GPUs and 36 CPUs; system configurations and availability are subject to change.
“Where AI gets stuck today, it is often not the compute die. It is memory bandwidth, packaging, and how many of these parts the world can actually build.”
— ThorstenMeyerAI.com, describing the role of packaging and memory
Dimensions, Supply and Performance
The explainer does not provide measured dimensions for the dies, packages or HBM stacks; its 3D models are schematic. It also does not quantify how much packaging capacity is available, how large any memory shortfall may be, or how those constraints will affect accelerator prices and delivery schedules. SK hynix’s forecast is a company outlook, not an independently verified guarantee. Comparisons between bandwidth figures for different GPU generations do not, by themselves, show how much faster a particular AI workload will run. Product configurations and stated availability may also change, and the source does not supply independent testing of the systems’ performance or cooling needs.
Watch HBM and Package Capacity
The next developments to watch are supplier updates on HBM production and advanced packaging, along with official specifications and availability for Nvidia’s Vera Rubin systems and AMD’s next-generation accelerators. TSMC’s stated work on larger CoWoS-L package sizes is another milestone, but the source does not give dates for 9x- or 14x-reticle production. Future product disclosures may clarify system dimensions, memory configurations and how much package capacity can be delivered at scale.
Key Questions
What does an AI accelerator package contain?
In the designs described by the source, a package can combine compute dies, HBM stacks and a silicon interposer, mounted on a package substrate and topped by cooling hardware in the system. Exact configurations differ by product.
What is HBM, and why is it stacked?
HBM is high-bandwidth memory made from DRAM chips stacked above a base die and connected vertically using through-silicon vias. Placing memory close to compute supports high data-transfer bandwidth, though the source does not quantify the effect for individual workloads.
What does a silicon interposer do?
The interposer provides dense connections between compute dies and HBM stacks. It carries many connections in a small area, serving as a wiring layer between components rather than as a compute chip.
How many GPUs are in an NVL72 rack?
The Nvidia NVL72 configurations described in the source contain 72 GPUs and 36 CPUs, alongside compute trays, NVLink switch trays, management networking, power equipment and cooling.
Does the report establish when AI chips will become cheaper or more available?
No. It cites SK hynix’s forecast that memory demand will exceed supply through 2030, but it does not quantify future shortages, prices or delivery times. Those outcomes remain uncertain.
Source: ThorstenMeyerAI.com
