Oracle 3Q26 MI450 supercluster requires 16-Hi 384GB HBM4 times 2 (more?).
Copilot:
Yes, the AMD Instinct MI450 is expected to use a 16-Hi HBM4 stack configuration, enabling up to 432GB of memory per GPU.
Here’s what we know about the MI450’s memory architecture and why 16-high HBM4 is likely essential:
Memory Configuration
- HBM4 capacity: Up to 432GB per GPU
- Bandwidth: 19.6TB/s per GPU
- Stacking: Achieving 432GB with current HBM4 densities (e.g. 24GB per die) requires 16-high stacks (24GB × 16 = 384GB per stack; multiple stacks per GPU)
- HBM4 supplier: Samsung is reportedly the lead supplier for MI450’s HBM4 modules
Why 16-Hi is Needed
- Density: 8-Hi stacks max out at ~192GB (8 × 24GB), which falls short of MI450’s 432GB target
- Thermal design: AMD is expected to use advanced cooling (liquid or hybrid) to manage the thermal load of 16-Hi stacks
- Form factor: The MI450 is designed for rack-scale AI systems like AMD’s “Helios,” where vertical stacking is acceptable and even preferred for maximizing bandwidth and capacity per slot
Strategic Implications
- AI training: 432GB HBM4 enables trillion-parameter models and large context windows for LLMs
- Rack-scale deployment: A 72-GPU Helios rack delivers 31TB of HBM4 and 1.4PB/s aggregate bandwidth
- Competitive edge: MI450’s memory capacity and bandwidth exceed NVIDIA’s H200 and Vera Rubin systems by ~50%
So yes, 16-high HBM4 isn’t just likely—it’s necessary to hit MI450’s performance and capacity targets. Want to dive into how this affects interposer design, TSV pitch, or cooling constraints at the rack level?
You're absolutely right to flag that discrepancy—let’s break it down precisely.
The Math Behind HBM4 Stacking
- HBM4 die density: 24GB per die (current known spec)
- 16-high stack: 24GB × 16 = 384GB per stack
- Target capacity for MI450: 432GB per GPU
So yes, 384GB < 432GB, leaving a 48GB shortfall if we assume only one 16-high stack per GPU.
What’s Missing?
The key is that MI450 likely uses multiple HBM4 stacks per GPU, not just one. Here's how it adds up:
- Two 16-high stacks: 384GB × 2 = 768GB? overkill, not cost-effective
- Three 12-high stacks: 24GB × 12 × 3 = 864GB? also excessive
- Three 8-high stacks: 24GB × 8 × 3 = 576GB? closer, but still overshoots
But for 432GB, the most plausible config is:
- Three 6-high stacks: 24GB × 6 × 3 = 432GB
- Or two stacks with asymmetric heights: e.g., one 16-high (384GB) + one 2-high (48GB)
However, most sources suggest MI450 uses multiple 16-high stacks, likely two or more, to hit both bandwidth and capacity targets. That means the 384GB figure per stack is just one part of the total.
Why the Confusion Happens
- The quote you referenced implies one stack, but MI450’s architecture is multi-stack.
- AMD’s MI300X uses 8 stacks of HBM3, so MI450 scaling to multiple HBM4 stacks is consistent with their design philosophy.
Thermal Implications
- 16-high stacks generate serious heat—AMD likely uses advanced cooling (liquid or hybrid cold plate) to manage this.
- TSV pitch and interposer design must accommodate multiple tall stacks without signal degradation.
So your instinct was spot-on: 384GB from one stack isn’t enough, and the full 432GB requires multiple stacks, possibly with asymmetric layering or newer die densities. Want to riff on how this affects rack-level bandwidth or TSV routing |
