The person in the photo is Joshua Fryman — a Senior Principal Engineer at Intel — and he's holding a prototype that could completely transform the memory industry for AI servers.
-
A few days ago, at an Intel conference in Japan, an intriguing new technology was unveiled: ZAM.
To understand what makes it special, you first need to understand the technology it's designed to replace — HBM.
AI servers require enormous amounts of accessible RAM, on which they store all the billions of parameters of a language model simultaneously, enabling them to serve responses to users.
The need for exceptionally high RAM capacity makes conventional RAM unsuitable, which is why HBM is used instead.
HBM is essentially a stack of RAM dies layered on top of one another, with data passing between them through thousands of tiny holes that are carefully drilled vertically through the stack and filled with copper.
Drilling those holes is a delicate process that frequently ends in die damage and scrapped chips.
Around each drilled hole, small keep-out margins must be maintained to prevent cracking, which reduces the area available for memory cells.
On top of that, the vertical data channels generate heat that becomes trapped at the core of the HBM cube and struggles to escape.
These factors have limited the most advanced HBM generation currently in development to a maximum of just 64GB of RAM per package.
These chips are manufactured primarily by Samsung, SK Hynix, and Micron, and dependence on these suppliers has driven shortages and relentless price increases.
-
Intel has partnered with SoftBank to offer an interesting technological solution.
In the new ZAM memory, the memory layers are tightly bonded together, and the holes are drilled diagonally between layers rather than straight down from top to bottom.
It sounds like a minor change, but it enables a significant reduction in the number of channels, smaller required hole margins, more efficient heat dissipation, and a lower risk of die cracking during the drilling process.
Intel claims this approach can pack up to 512GB of RAM into a single package, with better energy efficiency and a lower defect rate during manufacturing.
-
Intel has an unsuccessful track record in memory development — its Optane project ended with billions of dollars in losses.
This time, it is contributing the design only, with SoftBank Japan funding the manufacturing and Saimemory — a SoftBank-owned company — carrying out production.
The technology was developed by Intel through a joint research project with the U.S. Department of Energy.
Intel is under significant strain right now, and its inability to shoulder the manufacturing risk means that the primary beneficiaries of this research will be the Japanese side.
--
👋 Hi, I'm Shlomo Strauss — follow me for more content on science and technology.