Challenge

There are currently dozens of different drug response prediction models circulating throughout the research community. Despite their proliferation, there are no benchmarks and no standard by which to compare the models to each other. The project’s aims were twofold. First, focusing on precision medicine, the researchers sought to enable oncologists to use models that recommend treatment based on a genetic, protein, or other molecular profile that can be generated from a biopsy. Second, the researchers sought to improve the predictive performance of the deep learning models.

This means being able to quickly and accurately compare thousands of models and assess which are performing best in as fair and biologically relevant a manner as possible. Moreover, the researchers wanted the comparison of models and the comparison of the performance impacts of training and validation choice to be as fully automated as could be achieved.

Before neurons can be reconstructed in 3D, the 2D profiles of objects must be aligned between neighboring images in an image stack. Image misalignment can occur when tissue samples are cut into thin sections, or during imaging on the electron microscope. The Feabas application (developed by collaborators at Harvard) uses template-matching and feature-matching techniques to optimize image transformations and align the 2D image content between sections.

Approach

ALCF supercomputing resources were used to optimize hyperparameters, variables that control the models’ learning process; optimization was accomplished by running large ensembles of models, and the performance of each model on specific drugs and cell lines was examined. Once the researchers identified the best settings for training the models, they performed cross-study generalization analyses to determine how well models trained on a dataset from the organization where they were experimentally generated performed when applied to data from a different organization. Data included RNA and DNA sequences of cancer models, in addition to drug screening and response data continuously curated and standardized from the public domain.

Results

Runs involving the training of thousands and tens of thousands of deep learning models were carried out to assess model uncertainty as a function of specific drugs, as a function of specific cell line cancer types, and as a function of neural network architecture.

Impact

This work stands to accelerate the research and development of effective cancer treatments.