From Paraphrase Modeling to Controlled Generation

Event Sponsor: 
Mathmatics and Computer Science Division Seminar - LANS
Start Date: 
Oct 17 2018 - 10:30am
Building 240/Room 4301
Argonne National Laboratory
Kevin Gimpel
Speaker(s) Title: 
Toyota Technological Institute at Chicago

A key challenge in natural language understanding is recognizing when two sentences have the same meaning. I'll discuss our work on this problem over the past few years, including the exploration of compositional functional architectures, learning criteria, and naturally-occurring sources of training data. The result is a single sentence embedding model that outperforms all systems from the 2012-2016 SemEval semantic textual similarity competitions without training on any of the annotated data from those tasks. As a by-product, we developed a large dataset of automatically-generated paraphrase pairs by using parallel text and neural machine translation. We've since used the dataset, which we call ParaNMT-50M, to impart a notion of meaning equivalence to controlled text generation tasks, including syntactically-controlled paraphrasing and textual style transfer.

Kevin Gimpel is an assistant professor at the Toyota Technological Institute at Chicago (TTIC), a philanthropically endowed academic computer science institute on the campus of the University of Chicago. He was previously a research assistant professor at TTIC from 2012 to 2015 and he received his PhD from the Language Technologies Institute at Carnegie Mellon University in 2012. His research focuses on natural language processing and machine learning. Recent interests include paraphrase recognition, narrative modeling, commonsense knowledge representation, and structured prediction in the era of deep learning. His research has been supported by a Sandia National Laboratories Fellowship and gifts from Google and Bloomberg.

Miscellaneous Information: 

This seminar will be streamed. See details at

Please click here [schedule.ics] to add this event to your calendar.

Google Calendar:

Upcoming Seminars
September 12, 2018, "Inverse problems in new X-ray imaging techniques: Tools, algorithms and future prospects" Siddharth Maddali Vivekanand, Postdoctoral Appointee, MSD/ANL
September 21, 2018, "Solving Multi-Leader-Follower Games by Smoothing the Follower's Best Response" Anna Thuenen, Research Assistant, Institute for Geometry and Practical Mathematics, RWTH Aachen University
September 26, 2018, "Understanding graphs through spectral densities" David Bindel, Associate Professor of Computer Science, Cornell University
October 1, 2018, "Advances in bounding methods for use in global optimization" Kamil Khan, Assistant Professor, Department of Chemical Engineering, McMaster University
October 10, 2018, "TBA" Mladen Kolar, Associate Professor of Econometrics and Statistics, University of Chicago Booth School of Business
October 17, 2018, "From Paraphrase Modeling to Controlled Generation" Kevin Gimpel, Assistant Professor, Toyota Technological Institute at Chicago
October 24, 2018, "TBA" Dmitriy Drusvyatskiy, Assistant Professor, Department of Mathematics, University of Washington
October 31, 2018, "TBA" Prasanna Balaprakash, Computer Scientist, MCS & LCF, ANL
November 7, 2018, "TBA" Emil Constantinescu, Computational Mathematician, MCS/ANL