Controlling conditional language models without catastrophic forgetting

Tomasz Korbak, Hady Elsahar, German Kruszewski, Marc Dymetman

International Conference on Machine Learning, ICML2022 [paper] [slides] [code]

In this work we target an the important question of how to adapt pre-trained generative models to meet human requirements without destroying their general capabilities ("catastrophic forgetting"). We extend DPG to conditional tasks by proposing Conditional DPG (CDPG). We evaluate CDPG on four different control objectives across three tasks (translation, summarization and code generation) and two pretrained models (T5 and GPT-Neo). Our results show that fine-tuning using CDPG robustly moves these pretrained models closer towards meeting control objectives and — in contrast with baseline approaches — does not result in catastrophic forgetting.