I had the opportunity to interview with Sean Welleck on his podcast The Thesis Review. We touched on many topics including my personal journey into AI research, and some of my recent research interests such as low-resourcedness and adapting large language models without catastrophic forgetting.

Follow his podcast on his personal website , soundcloud or apple podcasts

This episode of the NAVER LABS Europe podcast series is about Energy-Based Models (EBMs).
These models, based on machine learning, have been around for many years but have been gaining a lot of interest very recently. Podcast guests, research scientist Hady Elsahar and Principal Scientist Marc Dymetman are working on EBMs in the field of natural language. They also co-organized the EBM workshop at the 2021 International Conference on Learning Representations (ICLR).

Large Language Models such as GPT3 are trained on large uncurated text from the internet. Despite their huge success and emergent properties such as in-context learning, they suffer a lot from inherent biases and toxicities that could lead to the generation of harmful content. In this blog post, we talk about a novel framework for controlled natural language generation that has been recently published in ICLR2021. Generation with Distributional Control, which achieves great generality on the types of constraints that can be imposed and has a large potential to remedy the problem of bias in language models.

Performance drop due to domain-shift is an endemic problem for NLP models in production. This problem creates an urge to continuously annotate evaluation datasets to measure the expected drop in the model performance which can be prohibitively expensive and slow. In this paper, we study the problem of predicting the performance drop of modern NLP models under domain-shift, in the absence of any target domain labels. We investigate three families of methods (H-divergence, reverse classification accuracy and confidence measures), show how they can be used to predict the performance drop and study their robustness to adversarial domain-shifts.

We are happy to announce that the development of Scribe officially started! Scribe is a project funded by the Wikimedia Foundation during the years 2019/2020 to build a tool that aims at helping new editors of under-resourced languages to start writing high-quality articles that confirm with their language standards. Reach out to us, if you are interested in a research collaboration on any of the topics described below.

This blog post aims to raise discussion within the ML research community and propose solutions to find natural, effective & easy to stick to solutions for managing and reproducing your research experiments.

#EMNLP2018 tutorial by Joel Grus, Matt Gardner and Mark Neumann from Allen Institute for Artificial Intelligence