Automating Recompute vs. Store Decisions in Adjoint Computations

Jean Utke
Seminar

The talk will first present the basic context of the recompute vs. store tradeoff as it occurs in adjoint computations enabled by algorithmic differentiation. Then, it will cover the currently implemented options deciding the tradeoff, highlight their respective weaknesses, and motivate the need for a new approach. The main part of the talk explains the tradeoff as a problem on a set of directed acyclic graphs (DAGs) and a heuristic to decide between recomputing or storing of values. The heuristic approach will be motivated in the context of a source transformation implementation of adjoint computations and compared to other currently available options. The set of DAGs does not reflect the impact of the control flow on the actual cost. A second phase of the new approach addresses control flow impacts. The talk concludes with implementation concerns and an outlook on future research on this subject.