The people who built Frontier - the world's first exascale supercomputer - are now designing the next generation of AI data centers. The hyperscalers should be paying attention.
Oak Ridge National Laboratory launched a new initiative in late February with a deceptively modest name: the Next-Generation Data Centers Institute. Don't let the bureaucratic branding fool you. ORNL is packaging sixty years of supercomputing facility design expertise (cooling systems, power distribution, interconnect topology, thermal management at megawatt scale) and offering it as a blueprint for the commercial AI data center buildout.
The timing is deliberate. Hyperscalers are spending $700 billion this year on AI infrastructure. Most of them are reinventing solutions to problems the national labs solved a decade ago.
Frontier, which became the world's first exascale supercomputer in 2022, wasn't just a computing milestone. It was a facility engineering achievement. Fitting 9,408 AMD MI250X GPUs and 9,408 EPYC CPUs into Oak Ridge's infrastructure required solving power, cooling, and interconnect problems at a scale the commercial data center industry is only now confronting.
Frontier consumes roughly 22.7 megawatts at peak. That's a fraction of what the new hyperscale AI data centers are planned for. Meta's facilities are targeting hundreds of megawatts each, and the Thinking Machines Lab deal with NVIDIA involves at least one gigawatt of computing. But the engineering principles scale. How you distribute power across thousands of nodes without creating hotspots. How you cool a rack dissipating 60-80 kW without the thermal gradient destroying reliability. How you design an interconnect topology that minimizes data movement, because data movement is where the energy actually goes.
The Frontier team learned hard lessons about all of this. Lessons that cost taxpayer money, produced detailed technical reports, and are now publicly available. The private sector is spending billions to learn the same lessons from scratch.
HPC and AI workloads have converged enough that national lab expertise is suddenly commercially valuable in a way it hasn't been since the early internet era.
Traditional HPC workloads (climate simulation, molecular dynamics, nuclear weapons modeling) care deeply about floating-point throughput, memory bandwidth, and inter-node communication latency. AI training workloads care about the same things, plus a few extras: massive data ingestion pipelines, checkpoint storage at terabyte scale, and the ability to tolerate node failures without losing weeks of training progress.
ORNL's institute is positioned at this intersection. The lab has operational experience running exascale workloads that stress every subsystem simultaneously: compute, memory, storage, network, power, cooling. Commercial AI data centers are about to hit the same wall, just with bigger numbers.
Consider the power efficiency challenge. ORNL's latest facilities achieve power usage effectiveness (PUE) values approaching 1.05 under optimal conditions. Industry average PUE for commercial data centers sits around 1.3-1.4, with many facilities running higher. On a 100-megawatt facility, the difference between 1.05 and 1.3 PUE is 25 megawatts of wasted power. At utility-scale electricity costs, that's millions of dollars per year in pure waste, plus an environmental footprint that's increasingly hard to justify.
ORNL isn't just looking backward at Frontier's lessons. The institute is designing for what comes next: AI data centers consuming hundreds of megawatts to gigawatts of power, running workloads that blend traditional HPC simulation with AI training and inference.
Three technical areas dominate the institute's early work.
Liquid cooling at rack-to-facility scale is the first. Air cooling hits a wall somewhere around 40-50 kW per rack. Current AI racks (NVIDIA's GB200 NVL72, for instance) already push past 100 kW per rack. The Frontier team has operational data on direct liquid cooling at scale, including failure modes, maintenance requirements, and the interactions between cooling system design and computational performance.
Then there's power distribution architecture. As facilities push past 100 MW, the electrical infrastructure becomes a design constraint that shapes the entire building. You can't just add more transformers and switchgear linearly. The topology of power distribution (how you route electricity from the utility feed to individual racks) affects reliability, efficiency, and the ability to handle the dynamic load profiles of AI training (which can swing from 40% to 100% utilization in seconds as different phases of training execute).
Finally, interconnect topology co-design. In traditional data centers, the network is designed after the compute is specified. In HPC, the network and compute are co-designed because communication patterns are known in advance. AI training workloads fall somewhere between: the communication patterns are semi-predictable (all-reduce, all-gather, pipeline parallelism), which means the network can be optimized for them. ORNL's expertise in designing interconnect topologies that match workload communication patterns could save hyperscalers significant amounts in networking hardware and, more importantly, training time.
The United States now operates three exascale systems: Frontier at Oak Ridge, Aurora at Argonne, and El Capitan at Lawrence Livermore. That gives the national lab system an unmatched concentration of operational exascale experience.
Aurora is already being used for fusion energy research simulations, running plasma physics models that would have been computationally infeasible five years ago. El Capitan supports the nuclear stockpile stewardship mission with classified simulations. Frontier continues to serve a broad portfolio of open science workloads.
Each machine represents a different architectural approach. Frontier uses AMD GPUs. Aurora uses Intel GPUs. El Capitan uses AMD GPUs in a different configuration. That diversity matters. The national lab system has operational expertise across multiple hardware ecosystems, not just the NVIDIA stack that dominates commercial AI.
Meanwhile, India announced plans for an 8-exaflop AI supercomputer through international partnerships. The EuroHPC SPACE Centre of Excellence is redesigning parallel astrophysics codes for exascale. The global expansion of exascale computing creates demand for exactly the kind of facility design expertise ORNL is now packaging.
The commercial data center industry has historically operated independently from the HPC community. Different conferences, different vendors, different design philosophies. A hyperscaler data center architect and an HPC facility engineer might never cross paths professionally.
That separation made sense when the workloads were different. Web serving and database management don't require the same facility design as running LINPACK at exascale. But AI training is functionally identical to HPC: tightly coupled parallel computation that stresses every facility subsystem simultaneously.
Specific areas where national lab expertise could accelerate the commercial buildout:
Failure analysis and resilience. Running 10,000+ GPUs for months of continuous training generates failure data that's invaluable for understanding reliability at scale. The national labs have published extensively on hardware failure patterns, mean time between failures for different components, and strategies for checkpoint/restart that minimize lost computation. Commercial AI training operations are generating their own failure data, but they're not sharing it.
Thermal modeling. ORNL has sophisticated thermal simulation capabilities that model airflow and heat distribution at facility scale. Most commercial data center designs use simpler models. As rack densities increase, the difference between a good thermal model and a rough approximation can mean the difference between a stable facility and one that throttles under full load.
Workload scheduling. National labs have decades of experience scheduling diverse workloads across shared infrastructure, balancing throughput, fairness, priority, and resource utilization. As hyperscalers begin offering multi-tenant AI infrastructure, these scheduling problems become directly relevant.
One more angle worth noting: ORNL is also a leader in quantum-classical hybrid computing. The CSIS recently published an analysis of US leadership in quantum-supercomputing integration, identifying the national labs as the natural bridging point between classical HPC and quantum computing.
The Next-Generation Data Centers Institute includes provisions for quantum-ready facility design. As quantum computers scale beyond a few hundred qubits, they'll need to be co-located with classical supercomputers for hybrid workflows. The facility requirements for quantum (extreme cooling, vibration isolation, electromagnetic shielding) add another layer of complexity that ORNL is already navigating.
The hyperscalers building AI data centers today are focused on GPUs. But the smart ones are leaving room in their facility designs for quantum hardware that might arrive in three to five years. ORNL's institute is one of the few places where that kind of forward-looking facility design is being done systematically.
The national labs built the supercomputers that trained the first generation of large AI models. They operated the facilities that proved exascale computing was possible. Now they're packaging that expertise for the largest infrastructure buildout in computing history. The question is whether the hyperscalers are humble enough to use it.
