Scalable Reinforcement-Learning-Based Neural Architecture Search for Scientific and Engineering Applications

Event Sponsor: 
Mathmatics and Computer Science Division Seminar - LANS
Start Date: 
Feb 6 2019 - 10:30am
Building 240/Room 1404-1405
Argonne National Laboratory
Prasanna Balaprakash
Speaker(s) Title: 
Computer Scientist, ANL-MCS & LCF

The success of deep learning in machine learning applications has encouraged the scientific and engineering community to develop deep-learning-based predictive models for a wide a range of applications. Designing a deep neural network (DNN) architecture for a particular modeling task, however, requires significant architecture engineering by a deep learning expert. While several recent works discuss automating the process of the neural architecture search (NAS), they have focused mainly on the traditional machine learning tasks of image and text classification. In this talk, we will present a scalable NAS approach to automatically generate DNN models for predictive modeling in science and engineering applications. We will discuss a recurrent neural-network-based architecture generator that produces a multilayered perceptron with skip connections. We leverage a manager-worker-based distributed reinforcement-learning approach using proximal policy optimization method to iteratively improve the generated DNN architectures. We demonstrate the effectiveness of the proposed NAS approach for multivariate and multioutput regression problems on diverse applications. The generated architectures obtain high accuracy while maintaining significantly fewer parameters and achieve 70% to 80% node utilization on 256 to 1,024 nodes of Theta supercomputer at ALCF.

Miscellaneous Information: 

This seminar will be streamed. See details at

Click here [schedule.ics] to enter this event on your calendar.

Google Calendar:

Upcoming Seminars
January 9, 2019, "The impact of noise on Krylov method performance" Hannah Morgan, Postdoctoral Appointee, MCS/ANL
January 25, 2019, "TBA" Chiwoo Park, Associate Professor, High Performance Material Institute, Florida State University
February 6, 2019, "Scalable Reinforcement-Learning-Based Neural Architecture Search for Scientific and Engineering Applications" Prasanna Balaprakash, Computer Scientist (MCS & LCF, ANL)
February 20, 2019, "SmartKT: A Search Framework to assist Program Comprehension using Smart Knowledge Transfer" Partha Pratim Das, Professor, Department of Computer Science & Engineering, Indian Institute of Technology Kharagpu