Artificial intelligence is driving unprecedented demand for high-performance compute clusters, which has significant implications for power delivery networks (PDNs) and data center infrastructure. As models grow progressively more complex, the components they rely on demand larger amounts of energy. Industry professionals must identify methods to address the power efficiency of AI chips at scale.
Engineers Must Optimize Power Delivery Networks
Design engineers must understand how AI chips work before they can reimagine conventional architectures. These devices comprise electronic circuits laid out on silicon wafers. They specialize in AI training or inference workloads, using parallel processing to facilitate billions of calculations within seconds.
Today’s most advanced models have billions of parameters. They have outgrown single AI chips and now require multiple chips working together. These clusters must communicate extremely fast to maximize performance. Pushing compute boundaries has necessitated tightly clustered processors with the fastest connections possible. When they can’t communicate efficiently, bottlenecks form as graphics processing units sit idle.
This problem is becoming more pressing as the gap between processing and data delivery speed widens. Since 2005, compute performance has increased by a magnitude of 90,000, while data transfer speeds have only risen by a magnitude of 30.
High-density architecture is key for accelerating data movement, so engineers must pack more transistors on chips, more chips in servers and more servers in racks. However, reliably delivering electricity is becoming increasingly challenging as form factors shrink and transistor density increases. Parasitic inductance and capacitance can cause ringing on the power supply rails, distorting the waveform and leading to timing errors.
In addition to potentially degrading signal integrity, design engineers run into the problem of introducing thermal bottlenecks. Power density directly correlates to thermal density, which can create hot spots that degrade chip reliability. Conventional heat dissipation paths are insufficient for dense boards.
Power Delivery Challenges for AI Semiconductors
Designing the components powering AI chips to be both as reliable and efficient as possible can be challenging.
Thermal Management
Unlike general-purpose chips, those designed for AI workloads are highly dense. Manufacturers pack as many transistors as possible onto the board to maximize performance. High power density creates localized hot spots that are difficult to dissipate. Without advanced heat extraction techniques, most devices require thermal throttling — reduced clock speed and power consumption — to maintain optimal performance.
High Power Consumption
AI semiconductors are power hungry, consuming 700 to 1,200 watts per chip on average due to their massive number of transistors. The average large language model requires tens of thousands of chips running simultaneously. They demand high-speed data movement between compute and memory, with rapid, dramatic swings in current depending on the workload. This characteristic complicates efficient PDN design.
Material Selection
As chip features shrink, conventional materials — namely tungsten and copper — are reaching their physical limitations regarding conductivity and scalability. Nanoscale interconnections can only become so small before these metals can no longer provide adequate low-resistance pathways for electricity. Also, vertical stacking introduces novel thermal management intricacies, complicating power delivery reliability.
High Current Densities
High current densities are standard in AI workloads. When the current density is high enough, the heat dissipated repeatedly breaks metal atoms from the structure. These move in the direction of the flowing electrons, leaving behind voids. These vacancies can grow, eventually resulting in open circuits. Over time, short circuits are almost inevitable.
Miniaturization is primarily to blame. Smaller wires have smaller cross-sectional areas, resulting in higher current densities and increased susceptibility to electromigration. As modern AI semiconductors get smaller, combating high current densities will become increasingly complex.
How to Address These Power Delivery Challenges
Design engineering professionals can address these challenges in several ways.
1. Proper Packaging
Proper packaging can make reliably powering AI chips easier. Semiconductors are particularly sensitive to electrostatic discharge, which can irreversibly damage the PDN. Packaging materials for electrostatic discharge protection and advanced thermal interfacing can effectively dissipate heat and prevent the spontaneous transfer of electrical charges between parts, protecting the sensitive circuits that receive power.
The packaging should also withstand the rigors of distribution and transportation. Such shipments often require special handling services, which provide extra care for high-value, fragile or time-sensitive products. Even with this precaution, proper protection is essential.
2. Material Innovation
Due to miniaturization, interconnects are nearing the point where they can no longer deliver low-resistance pathways. The interconnect stack already accounts for 75% of a chip’s resistive-capacity delay and consumes around 33% of device power. Design engineers must prepare for the point when copper and tungsten reach their limits.
Novel materials can replace those used in narrow interconnects. Molybdenum (Mo) is a suitable alternative metal for narrow interconnects, as it has low resistance at the nanometer scale and can help counteract voids. Unlike tungsten, it has little to no inherent diffusivity into dielectric materials and does not require a barrier layer.
This material is well-suited for emerging hybrid metallization schemes, which aim to enhance semiconductor performance by utilizing bottom-up deposition to prefill a via with barrierless metals before adding copper. Research shows that a barrierless hybrid molybdenum design can decrease overall resistance by approximately 56% relative to a conventional copper dual damascene scheme.
3. Backside Power Delivery
AI chips usually work with front-side power delivery, which requires energy to be routed through multiple layers of wiring. Instead, professionals can run it through the wafer’s underside, separating the power and data delivery networks.
This approach will improve efficiency because electricity no longer competes with signals for space on the board. Delivering electricity directly to the transistors bypasses the signal routing stack, reducing IR drop and enabling thinner power grids.
4. Vertical Power Delivery
Vertically routing high-current power traces is essential for power efficiency in AI chips. Unlike the conventional design approach, this technique is particularly suitable for modern semiconductors. Minimizing the distance from the source to the silicon enhances energy delivery performance and frees up board real estate to reduce losses.
Effectively Addressing Power Delivery Problems
Improving power delivery reliability, making voltage regulation more predictable and accelerating critical data movement hinge on semiconductor design. Optimizing power efficiency in AI chips may not be straightforward, but understanding how AI chips work provides professionals with a solid foundation. Options exist beyond changing the PDN or material selection — revolutionary innovations will come from research and development.
