Lab Powers Up to Plug In Next-Gen Supercomputers

Lawrence Livermore National Laboratory photo

Supercomputers help the Department of Energy’s National Nuclear Security Administration maintain confidence in the nation’s nuclear weapons.

Lawrence Livermore National Laboratory recently completed upgrades to a facility that will provide a home for next-generation supercomputers which in turn will provide greater processing power to certify and evaluate future weapons.

Lawrence Livermore — one of the three federally funded research and development labs that develop nuclear weapons — completed the $100 million “exascale computing facility modernization project” just as tensions with Russia have renewed the focus on nuclear capabilities, said Rob Neely, the lab’s advanced simulation and computing lead and program director.

Since the United States stopped underground testing in 1992, scientists have developed advanced codes and software — which continue to be improved — to ensure the weapons’ capabilities.

And that’s at the heart of the lab’s work, creating application codes that can simulate how the nuclear stockpile would work without physical testing. The lab can then certify that the weapons will perform as required, Neely explained.

“We’re trying to make our systems safer, more secure, less vulnerable to threats from adversaries,” he said.

“Every new computer we get, we make advances to our codes,” he said. They become more predictive, use higher fidelity physics and the lab learns what it is missing, he said.

Then “we get to a horizon; we learn the limits of what we can know with a given system,” he said. That pushes the need for faster and faster computers, he added.

The project kicked the lab’s power capacity up to nearly 1,800 watts per square foot, totaling about 85 megawatts, said Project Manager Anna Maria Bailey. That’s almost enough energy to power every home for the nearly 88,000 residents living in Livermore, California, where the lab is based, she said.

When construction was first completed on the facility in 2004, “Not in my wildest dreams did I think we would have that much power in one location,” she said.

The power will be used for one of the most advanced supercomputers in the world — El Capitan. Exascale systems like El Capitan — built by Hewlett Packard Enterprise and AMD — can calculate at least one exaflop, which is a quintillion calculations per second.

That kind of processing power generates a lot of heat. However, when the new computer is installed in 2023, the upgrades to the facility will allow the computer to run calculations without overheating, she said.

The modernization work more than doubled the facility’s cooling capacity. Newly installed cooling towers brought the system’s maximum from 10,000 tons to 28,000 tons of water, according to a press release.

Other upgrades included a 115 kilovolt transmission line, air switches, substation transformers and 13.8 kilovolt secondary feeders.

The additional power will also be needed to keep the current supercomputer at the facility — IBM’s Sierra — up and running during the transition. In about five years, the additional power will come in handy while the facility transitions from El Capitan to an even more advanced system.

“This kind of planning and execution in the modernization of our computing facilities enables us to move into the exascale era — and not only El Capitan — but bring a second exascale-class system online in this next decade without having to turn off El Capitan first,” the National Nuclear Security Administration’s Undersecretary for Nuclear Security Jill Hruby said in a press release.

This huge increase in computing power and capability comes just in time to meet the nation’s nuclear deterrent needs.

The Pentagon is modernizing the delivery platforms, which consist of intercontinental ballistic missiles, bombers and submarines known as the Nuclear Triad. Integrating the new weapons onto the new platforms will require new tests.

The power of supercomputing allows the government to verify new systems such as the Sentinel — which will replace the Minuteman III missile — without the use of underground testing, Neely said.

“Twenty years ago, people would laugh if you said we could field a new system in the stockpile without going to [live] testing,” he said. “They just didn’t believe it.”

The lab started the El Capitan Site Infrastructure project earlier this year, Bailey said. Now that the facility has the capacity to host the supercomputer, scientists can start to connect the power, cooling and networking to the system itself.

After the machine is on site, researchers will run tests such as multiphysics and multidomain codes, which model the effects on the weapons under different conditions such as heat and cold and scenarios such as detonation and explosion, Neely said.

Concerns about Russia’s nuclear force have grown as the United States and allies have lent support to the ongoing conflict in Ukraine, he pointed out.

“The term ‘nuclear deterrence’ is in the headlines now more than it has been in the 30 years I’ve worked at the lab because of the saber rattling of Putin and folks like that,” he said. “So it’s something that I think people are starting to pay attention to.”

Topics: Infotech, Nuclear, Training and Simulation