Multiscale Blood Flow Simulations

PI Name: 
George Karniadakis
PI Email:
Brown University
Allocation Program: 
Allocation Hours at ALCF: 
Research Domain: 
Biological Sciences

Cerebral aneurysms (CAs) occur in up to 5% of the general population, leading to strokes for over 40,000 Americans each year. Sickle cell (SS) anemia – a chronic inflammatory disease – is the most common genetic diseases among African American, with an 8% incidence of the trait among this population; 50,000 individuals are suffering from sickle cell anemia in USA every year. Celebral malaria (CM) is a related red blood cell (RBC) disease, with significant mortality of up to 20% even when treatment is given. More than 1.5 million deaths each year have been reported – mostly in children. Currently, there are no quantitative tools to predict the rupture of aneurysms or the progression of RBC-related devastating pathologies such as SS and CM. Petaflop resources will allow for realistic simulations of these brain pathologies, quantifying, for first time, their biophysical characteristics. Our research team has unique modeling and computational expertise to tackle this problem of national importance. We propose multiscale simulations for modeling blood flow in the human brain vasculature, the first of its kind, consisting of 100s large 3D arteries (Macrovascular Network, MaN), 10M arterioles (Mesovascular Network, MeN), and 1B capillaries (Microvascular Network, MiN). Previous works have shown that the resolution of each of them is extremely important and must be led with unprecedented accuracy to eliminate computational artifacts and reduced confidence in the simulation results. Available clusters of workstations need hundreds of simulation years, and still not coming even close to the required resolution. Specifically, we propose image-based 3D Navier-Stokes simulations for fully resolving MaN, coupled to subpixel stochastic simulations of MeN and MiN. To this end, for MaN/MiN we will employ N"T r – aspectral element based parallel code with more than 90% parallel efficiency on 122,880 BG/P cores. For MiN, we will employ dissipative particle dynamics (DPD) simulations on representative capillary domains modeling explicitly (down to protein-level) the red blood cells. We have implemented DPD using the Sandia code LAMMPS, also leading to more than 90% parallel efficiency. Our software is ready for production computing from day one. In the first year we will focus on MaN, MiN separately as well as on coupled MaN-MeN simulations while in the second and third years we will perform fully-coupled simulations. Specifically, we propose three sets of simulations. The first set is to simulate the flow structure interaction (FSI) in patient-specific aneurysms to evaluate – via parametric simulations – the effect of FSI on sounds produced by aneurysms - a possible diagnostic breaktrough for clinical practice. The second set it to simulate the rupture of aneurysms using continuum-atomistic modeling that will shed light into this process and provide quantitative information on its genesis and hence possible clinical treatments. The third set of simulations is application to the arterioles and capillary bed for RBC-related pathologies, e.g. SS and CM. This work is suported by NSF and NIH grants, which also include validation with in vitro experiments at MIT and clinical data at Harvard Medical School. Results from preliminary simulations have already been published (cover of Clinical & Experimental Pharmacology), and recently our project was selected as one of the top nine supercomputer projects worldwide 1.