Distribution of the energy and force components errors on the QMC validation set, before and after transfer learning. Image: Anouar Benali, Qubit Pharmaceuticals
Predicting the behavior of molecular systems is a key challenge in chemistry, biology, and materials science, with applications ranging from cancer drug discovery to the design of novel catalysts. Machine learning models can accelerate such predictions, but their accuracy often depends on computationally demanding quantum chemistry calculations. Using ALCF supercomputers, researchers have developed a scalable framework for training machine learning foundation models that learn from large datasets of lower-cost simulations while retaining the accuracy of high-level quantum results.
Challenge
High-fidelity quantum chemistry methods, such as quantum Monte Carlo (QMC), provide highly accurate molecular predictions but require significant computational resources, limiting their accessibility. Lower-cost methods, such as density functional theory (DFT), allow simulations at greater scale but with reduced precision. This tradeoff between cost and fidelity presents an obstacle for creating machine learning models that can be both accurate and broadly applicable. Bridging this gap requires new approaches that can integrate information from multiple simulation fidelities while minimizing the computational burden.
Approach
To address these challenges, the team developed a multi-fidelity pretraining framework for molecular machine learning foundation models. They began by using ALCF’s Polaris and Cineca’s Leonardo supercomputers to carry out large-scale pretraining on millions of molecular structures using DFT data to capture general molecular patterns. A high-fidelity dataset of thousands of molecular configurations was then computed on Aurora using QMC forces combined with multideterminant selected-Configuration Interaction wavefunctions, providing unprecedented accuracy for fine-tuning the model. The researchers employed graph neural networks to represent molecular structures and distributed workloads across multiple GPUs to efficiently train models on large, diverse datasets.
Results
Aurora’s exascale capabilities made it possible, for the first time, to perform full QMC force calculations and combine multideterminant QMC energies and forces at the complete basis-set limit. The team fully implemented and optimized these methods in QMCPACK, computing forces for 2,000 molecular configurations with 1.4 million determinants per molecule, achieving 52 petaflops. Models trained with this multi-fidelity approach achieved accuracy comparable to high-level quantum methods while maintaining DFT-scale efficiency. They outperformed single-fidelity models on benchmark molecular property tests and generalized effectively to new molecular systems and target properties.
Impact
This work demonstrates a practical path toward creating large-scale, accurate molecular foundation models without prohibitive computational costs. By integrating data from simulations of varying fidelity, the approach enables researchers to scale up machine learning training while retaining the precision needed for demanding scientific applications. The methodology could accelerate discovery pipelines in pharmaceuticals, energy materials, and catalysis, offering a powerful tool for scientists seeking to explore chemical space more efficiently.
Publications
Benali, A., T. Plé, O. Adjoua, V. Agarawal, T. Applencourt, M. Blazhynska, R. Clay III, K. Gasperich, K. Hossain, J. Kim, C. Knight, J. T. Krogel, Y. Maday, M. Maria, M. Montes, Y. Luo, E. Posenitskiy, C. Villot, V. Vishwanath, L. Lagardère, and J.-P. Piquemal. “Pushing the Accuracy Limit of Foundation Neural Network Models with Quantum Monte Carlo Forces and Path Integrals,” arXiv (preprint). https://doi.org/10.48550/arXiv.2504.07948
