MultiVerse-Kriging: A transfer- and multitask-learning based approach to autotuning

Event Sponsor: 
Mathmatics and Computer Science Division Seminar - LANS
Start Date: 
Aug 15 2018 - 10:30am
Building 240/Room 1404-1405
Argonne National Laboratory
Wissam M. Sid-Lakhdar
Speaker(s) Title: 
Lawrence Berkeley National Laboratory

Autotuning, or the problem of finding the optimal parameters of a software that will maximize a certain performance criteria (computation time, memory usage, energy consumption, ...), can be considered as a black-box optimization problem.
No analytical formulation of the objective function nor any information on its derivative are available.
Given a set of values for the parameters, it is possible to evaluate the objective function only through a costly run of the software.

Several algorithms exist in the literature that target this problem, among which are the surrogate-based black-box optimization algorithms. Their main idea is to build a cheap to evaluate model of the objective function and try to optimize this model instead of the original function. One such approach is Kriging.

Our work is a generalization of Kriging to the case where not only one optimization problem has to be solved, but where several and similar problems are to be. It lies within the transfer- and multitask-learning frameworks, as we use the knowledge acquired while solving one problem to help solve the other problems simultaneously.

A related approach to ours exist in the literature known as co-Kriging. In this case however, the hypothesis is that the data corresponding to one (or several) cheap to evaluate objective functions exist that are highly correlated to the main (and unique) objective function of interest.
In our case however, all objective functions ought to be of the same cost and importance, none is supposed to be under / over sampled compared to the others, and all of them are to be optimized instead of only one.

The example of the autotuning of the QR factorization routine of ScaLAPack will be the leading example in our presentation.

Miscellaneous Information: 

This seminar will be streamed. See details at

Please click here [schedule.ics] to add this event to your calendar.

Google Calendar:

Upcoming Seminars
August 1, 2018, "Ambiguous Chance-Constrained Binary Programs under Mean-Covariance Information" Siqian Shen, Associate Professor, Department of Industrial & Operations Engineering, University of Michigan
August 8, 2018, "SASSy Part II" LANS Summer Students
August 15, 2018, "MultiVerse-Kriging: A transfer- and multitask-learning based approach to autotuning" Wissam M. Sid-Lakhdar, Postdoctoral Fellow, LBNL
August 22, 2018, "SASSy Part III" LANS Summer Students
August 29, 2018, "TBA" Selin Sariaydin, Postdoctoral Appointee (XSD/ANL)
September 5, 2018, "New Polynomial-free, Variable High-order Methods using Gaussian Process Modeling for CFD" Dongwook Lee, Associate Professor, Applied Mathematics & Statistics, UC Santa Cruz
September 19, 2018, "TBA" Emil Constantinescu, Computational Mathematician (MCS/ANL)
September 21, 2018, "Solving Multi-Leader-Follower Games by Smoothing the Follower's Best Response" Anna Thuenen, Research Assistant, Institute for Geometry and Practical Mathematics, RWTH Aachen University
October 1, 2018, "Advances in bounding methods for use in global optimization" Kamil Khan, Assistant Professor, Department of Chemical Engineering, McMaster University
October 17, 2018, "TBA" Kevin Gimpel, Assistant Professor, Toyota Technological Institute at Chicago
October 24, 2018, "TBA" Dmitriy Drusvyatskiy, Assistant Professor, Department of Mathematics, University of Washington