Making waves on Mira

Katie Jones

Facebook Twitter LinkedIn Google E-mail Printer-friendly version

In a recent study published in Science, results from Mira supercomputer simulations validate a new “wave-like” model of the van der Waals force—a weak attraction that has strong ties to function and stability in materials and biological systems. One of the study’s lead authors, Robert A. DiStasio Jr. of Cornell University, is an Argonne Leadership Computing Facility user who develops computational methods for simulating bonded and non-bonded interactions between atoms in realistic systems, notably water, the basic ingredient of many of the materials we use and biological systems we rely upon.

Many new devices for energy technologies, therapeutic drug delivery and high-performance materials are being designed at the nanoscale by manipulating systems at the atomic and molecular levels. Although scientists and engineers have a better understanding and control of materials at the nanoscale than ever before, there are still fundamental questions about how things work exactly at distances where the laws of physics toe the line between our classical understanding and the world of quantum mechanics.

That’s why a team of researchers used the Mira supercomputer to simulate the interactions between nanostructures, focusing on a lesser understood force that plays an important role in molecular assembly and function. Unlike ionic or covalent bonds, which are bonds made by attractive electric charges or shared electrons, the van der Waals force is a non-bonded interaction at the nanoscale—which spans distances of about 10 to 1,000 typical chemical bond lengths.

“They’re relatively weak forces. The atoms ‘feel’ each other through space,” said Robert A. DiStasio Jr., assistant professor in the Department of Chemistry and Chemical Biology at Cornell University. “But these non-bonded forces play a crucial role in determining the structure, stability and function in a huge number of systems throughout the fields of biology, chemistry, physics and materials science.”

By simulating nanostructures including one-dimensional nanowires and two-dimensional nanosheets, researchers validated a new “wave-like” theoretical model that predicts better than have previous models how van der Waals forces behave at the nanoscale. One of the world’s five most powerful supercomputers, Mira, an IBM Blue Gene/Q system, is operated by the Argonne Leadership Computing Facility (ALCF), a U.S. Department of Energy (DOE) Office of Science User Facility located at DOE’s Argonne National Laboratory.

“This study changes how we should think about the manner in which objects interact at the nanoscale,” DiStasio said. “The existence of van der Waals interactions has been known for more than 100 years, but our ability to actually calculate these interactions in realistic systems has exploded over the last 10 years.”

Published in Science in March, the study reveals that for low-dimensional (1-D and 2-D) structures, like the nanowire and graphene sheet, van der Waals forces are enhanced by one to two orders of magnitude at ranges of 10 to 20 nanometers—a scale that is crucially important to the design and manipulation of many technology and therapeutic drugs applications.

“The rational design and precise control over the assembly of nanostructures is one of the fundamental goals of modern science,” DiStasio said. “New batteries, thin films, 3-D printing, personalized drugs—all of these rely on manipulating properties at the nanoscale.”

Contributors to the study include lead authors DiStasio and Alexandre Tkatchenko (Fritz Haber Institute of the Max Planck Society and University of Luxembourg), Alberto Ambrosetti (Fritz Haber Institute and University of Padova) and Nicola Ferri (Fritz Haber Institute).

Historically, van der Waals forces have been calculated using a local or particle-based approach. As electrons zoom around the nucleus of an atom, they create tiny fluctuations in the charge density (the electric charge of a given area). When a fluctuation in one atom gets in sync, or “couples,” to the fluctuation in another atom, they are interacting via the van der Waals force. This is the dance that happens before a drug binds to a protein or molecules assemble. Scientists often approximate the van der Waals force in large systems by the pairwise sum of these couplings.

However, as experimental techniques to measure forces at the nanoscale have improved, DiStasio said, anomalies suggest that this pairwise sum approach can significantly underestimate van der Waals forces. In some cases, experimentalists have seen an unexpected increase in the interaction force, or “sticking,” between molecules. One explanation is that van der Waals forces are stronger than the pairwise sum approach predicts.

“I think in this paper we are able to show that those experiments are qualitatively and quantitatively on point,” DiStasio said. “We are demonstrating that this interaction can be orders of magnitudes larger than previously thought.”

A ‘driving force of all nature’

Leonardo da Vinci said ‘water is the driving force of all nature.’ DiStasio and colleagues agree, and their computational science research focuses on the structure of water and aqueous ionic solutions. In collaboration with the research group of Roberto Car at Princeton University, DiStasio has been awarded three Advanced Scientific Computing Research Leadership Computing Challenge (ALCC) allocations for computational time on Mira from 2013 to 2016. His team’s goal has been to understand the complex and disordered microscopic structure of liquid water and how it changes in the presence of solutes and under extreme temperatures and pressures.

Water is important to reactions in fuel cells and industrial catalysts, but it’s also the stuff of life on Earth, critical to protein stability and molecular assembly. And one of the driving forces of water, and a key part of its structure, is the van der Waals interaction.

“As a research community, we have always wanted to accurately describe the van der Waals interaction in complex systems of interest like water,” DiStasio said. “And the development of the many-body dispersion model—the result of many years of close collaboration with the Tkatchenko group—has been tremendously successful in this regard.”

To date, the many-body dispersion (MBD) model has been used to account for van der Waals forces in a wide variety of molecules and materials. DiStasio and Tkatchenko investigated the MBD approach to describe non-bonded interactions like the van der Waals force at the nanoscale, drawing from the sometimes head-scratching field of quantum mechanics and its signature wave-particle duality.

“When you have a small system of a few atoms, the particle-based approach makes sense because there is no larger effect to see,” DiStasio said. “But when you have a nanowire made of thousands of atoms, the coupled fluctuations move from atom to atom to atom—it’s essentially making a wave. It’s no longer local, it’s collective.”

“Our work demonstrates that, in nanoscale systems, one has to think about van der Waals forces in terms of interactions between waves instead of interactions between particles,” Tkatchenko said.

The MBD model is the first to demonstrate the importance of collective charge density fluctuations, rather than the sum of local fluctuations, in describing non-bonded interactions between nanostructures. The MBD model also uses a first-principles-based approach—meaning it relies on fundamental physics rather than being shaped by experimental data—which allows researchers to simulate systems in detail that may be difficult or impossible to observe in the laboratory.

“They applied and tested their van der Waals model on a diverse set of nanostructures with many shapes and sizes,” said Alvaro Vazquez-Mayagoitia, ALCF assistant computational scientist.

Validating the model required extremely powerful computation. Among the nanostructures simulated were proteins, as well as carbon-based nanostructures that are incredibly strong and popular for material applications, including 1-D carbyne-like nanowires, 2-D graphene sheets and 3-D carbon nanotubes comprised of a few thousand of atoms each.

A few thousand atoms may not sound like a lot for a supercomputer, but for first-principles simulations, it’s not the number of atoms that are the computational burden—it’s their electrons. The determination of the electronic structures is extremely computational intensive. Mira computes up to 10 quadrillion calculations per second, and thousands of its processors run in parallel to simulate a near-realistic electronic structure at the nanoscale.

“The theory we used can be partitioned to run many simultaneous calculations at one time,” DiStasio said. “Mira is well-designed for running these interdependent calculations very efficiently.”

Unlike nanostructures, which are static simulations, the team’s dynamic liquid water simulations take many continuous weeks on several thousand processors to model structural changes in response to different solutes and high temperatures and pressures.

By working closely with Alvaro Vazquez-Mayagoitia and other ALCF staff, DiStasio’s team modified their codes to run efficiently on Mira’s HPC environment. Vazquez-Mayagoitia assisted in porting and adapting the first-principles Quantum ESPRESSO (QE) code to Mira. The improvements to the QE software followed a twofold strategy: better use of Mira’s processors and reduction of interprocessor communication. New multi-threaded OpenMP routines and use of asynchronous communications, which reduce delays by allowing some processors to transmit data while others continue computing, enabled a speed up of QE simulations by up to 40 percent.

“For ALCF, it is a priority to support science projects by understanding what they need to get their simulations done using big computers,” Alvaro said. “By assisting projects with modernizing their open source codes, we accelerate new discoveries that can benefit people around the globe.”

Now, they want to take the MBD model and integrate it with the dynamic water simulations, which could significantly improve the accuracy and predictive ability of simulations important to aqueous ion batteries, drug delivery across cell membranes and hydrogen fuel cells, among many other systems.

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science.

The U.S. Department of Energy's Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, visit the Office of Science website.