Best Practices for Using Proxy Applications as Benchmarks

David Richards, Lawrence Livermore National Laboratory
Joe Glenski, Hewlett Packard Enterprise
ECP logo

The IDEAS Productivity project, in partnership with the DOE Computing Facilities of the ALCF, OLCF, and NERSC and the DOE Exascale Computing Project (ECP) has resumed the webinar series on Best Practices for HPC Software Developers, which we began in 2016.

As part of this series, we offer one-hour webinars on topics in scientific software development and high-performance computing, approximately once a month. The next webinar is titled Best Practices for Using Proxy Applications as Benchmarks, and will be presented by David Richards (Lawrence Livermore National Laboratory) and Joe Glenski (Hewlett Packard Enterprise). The webinar will take place on Wednesday, April 15, 2020 at 1:00 pm ET.


Proxy applications have many uses in software development and hardware/software co-design. Because most proxies are easy to build, run, and understand, they are especially appealing for use in benchmark suites and studies. This webinar will examine the role of proxy apps as benchmarks and explain why run rules and a figure of merit are essential for a proxy application to function as an effective benchmark. We will show how to evaluate the fidelity of benchmarks as a model for actual workloads and provide tips on creating problem specifications and other run rules. We will discuss what DOE facilities are looking for when they assemble benchmark suites for use in procurements. Finally, we will explain how system vendors use our benchmark suites and what practices they view as most (and least) effective.