Conversational AI Paper List

Paper List Format Publication Year: Paper Title & Paper Link: First Author (First Author’s Affiliation/Institution), Code Link

Overview

Existing Models of Dialog System

Task-Oriented Dialog

• 13: POMDP-Based Statistical Spoken Dialog Systems: A Review: Steve Young(Cambridge University)

• 11: Spoken Language Understanding: Systems for Extracting Semantic Information from Speech: Book!

• 11:Data-Driven Response Generation in Social Media: Alan Ritter(University of Washington Seattle)

• 15: A Neural Network Approach to Context-Sensitive Generation of Conversational Responses: Alessandro Sordoni(Universite de Montreal)

• 15: A Neural Conversational Model: Oriol Vinyals(Google), code via tensorflow

• 15: Neural Responding Machine for Short-Text Conversation: Lifeng Shang(Noah’s Ark Lab), code via theano and tensorflow

Traditional NLP component stack

Challenge of NLP

• 09: SPEECH and LANGUAGE PROCESSING An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition Second Edition: book

Deep Semantic Similarity Model(DSSM)

application scenarios

• Web search

• 13: Learning deep structured semantic models for web search using clickthrough data: Po-Sen Huang(University of Illinois at Urbana-Champaign), code via tensorflow

• 14: A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval: Yelong Shen(Microsoft Research)

• 16: Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval: Hamid Palangi, code

• Entity linking

• 14: Modeling Interestingness with Deep Neural Networks: Jianfeng Gao(Microsoft Research)

• Image captioning

• 15: From Captions to Visual Concepts and Back: Hao Fang&Li Deng(Microsoft Research)

• Machine Translation

• Learning Continuous Phrase Representations for Translation Modeling: Jianfeng Gao(Microsoft Research)

• Online recommendation

• [duplicate] 14: Modeling Interestingness with Deep Neural Networks: Jianfneg Gao(Microsoft Research)

Framework of Model

• [duplicate] 13: Learning deep structured semantic models for web search using clickthrough data: Po-Sen Huang(University of Illinois at Urbana-Champaign), [code]

• [duplicate] 14: A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval: Yelong Shen(Microsoft Research)

• 16: Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval: Hamid Palangi, code

• Sent2Vec: software by microsoft

Go beyound DSSM

• [duplicate] 15: From Captions to Visual Concepts and Back: Hao Fang&Li Deng(Microsoft Research)

Question answering(QA) and Machine Reading Comprehension(MRC)

Open-Domain Question Answering

Knowledge Base-QA

• Symbolic approach via Large-scale knowledge graphs

• [oral] 98: MindNet: acquiring and structuring semantic information from text: Stephen D.Richardson(Microsoft Research)

• [oral] 13: Semantic Parsing on Freebase from Question-Answer Pairs: Jonathan Berant(Stanford University)

• 15: Attention with Intention for a Neural Network Conversation Model: Kaisheng Yao(Microsoft Research)

• 14: Knowledge-Based Question Answering as Machine Translation: Junwei Bao(Harbin Institute of Technology)

• 15: Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base:Wen-tau Yih(Microsoft Research)

• ReasoNet with Shared Memory

• [oral][duplicate] 16: Link Prediction using Embedded Knowledge Graphs: Yulong Shen（Microsoft&Google Research）

• 17: ReasoNet: Learning to Stop Reading in Machine Comprehension:Yelong Shen(Microsoft Research)

• Search Controller in ReasoNet

• [duplicate] 16: Link Prediction using Embedded Knowledge Graphs: Yulong Shen（Microsoft&Google Research）

• ReasoNet in symbolic vs neural space

• Symbolic is comprehensible but not robust

• 11: Random Walk Inference and Learning in A Large Scale Knowledge Base:Ni Lao(Carnegie Mellon University)

• 98: MindNet: acquiring and structuring semantic information from text:Stephen D.Richardson(Microsoft Research)

• Neural is robust but not comprehensible

• [duplicate] 16: Link Prediction using Embedded Knowledge Graphs: Yulong Shen（Microsoft&Google Research）

• [oral] 15: EMBEDDING ENTITIES AND RELATIONS FOR LEARNING AND INFERENCE IN KNOWLEDGE BASES:Bishan Yang(Cornell University), TensorFlow code, PyTorch code

• Hybrid is robust and comprehensible

• 18: M-Walk: Learning to Walk in Graph with Monte Carlo Tree Search:Yelong Shen(Microsoft Research&Tecent AI Lab)

• 18: [oral] DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning:Wenhan Xiong(University of California,Santa Barbara), code1 code2

• 18: GO FOR A WALK AND ARRIVE AT THE ANSWER: REASONING OVER PATHS IN KNOWLEDGE BASES USING REINFORCEMENT LEARNING:Rajarshi Das(University of Massachusetts,Amherst),

• Multi-turn KB-QA

• Programmed Dialogue policy

• 15: A Probabilistic Framework for Representing Dialog Systems and Entropy-Based Dialog Management through Dynamic Stochastic State Evolution:Ji Wu(IEEE)

• Trained via RL Dialogue policy

• 16: Neural Generative Question Answering :Jun Yin(Noah’s Ark Lab, Huawei) corpus

• [oral] 16: A Network-based End-to-End Trainable Task-oriented Dialogue System:Tsung-Hsien Wen(Cambridge University), Theano code

• [oral] 17: Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access:Bhuwan Dhingra(Carnegie Mellon University), Theano code

Text-QA

• MS MARCO

• 16: MS MARCO: A Human Generated MAchine Reading COmprehension Dataset:Tri Nguyan(Microsoft AI&Research)

• SQuAD

• 16: SQuAD: 100,000+ Questions for Machine Comprehension of Text:Pranav Rajpurkar(Stanford University)

Neural MRC Models

BiDAF

• 16: BI-DIRECTIONAL ATTENTION FLOW FOR MACHINE COMPREHENSION:Minjoon Seo(University of Washington)

• code1

• code2

• code3

• code4

SAN

• 18: Stochastic Answer Networks for Machine Reading Comprehension: Xiaodong Liu(Microsoft Research,Redmond), code

Neural MRC Models on SQuAD

• Encoding: map each text span to a semantic vector

• Word Embedding

• 14: GloVe: Global Vectors for Word Representation:Jeffrey Pennington(Stanford University)

• code:midi-glove

• code:wiki-glove

• 13: Distributed Representations of Words and Phrases and their Compositionality:Tomas Mikolov(Google Inc.)

• code1

• code2

• Context Embedding

• capture context info for each word

• 16: context2vec: Learning Generic Context Embedding with Bidirectional LSTM:Oren Melamud(Bar-Ilan University)

• 18: Deep contextualized word representations:Matthew E.Peters(Allen Institute for Artificial Intelligence), code

• 18: QANET: COMBINING LOCAL CONVOLUTION WITH GLOBAL SELF-ATTENTION FOR READING COMPREHENSION:Adams Wei Yu(CMU&Google Brain)

• code1

• code2

• Context Embedding via BiLSTM/ELmo

• [duplicate] 18: Deep contextualized word representations:Matthew E.Peters(Allen Institute for Artificial Intelligence), code

• 17: Learned in Translation: Contextualized Word Vectors:Bryan McCann(SalesForce)

• 16: [duplicate]context2vec: Learning Generic Context Embedding with Bidirectional LSTM:Oren Melamud(Bar-Ilan University)

• Context Embedding

• [duplicate] 18: QANET: COMBINING LOCAL CONVOLUTION WITH GLOBAL SELF-ATTENTION FOR READING COMPREHENSION:Adams Wei Yu(CMU&Google Brain)

• code1

• code2

• Query-context/Content-query attention

• Reasoning: rank and re-rank semantic vectors

• Multi-step reasoning for Text-QA

• [duplicate] 17: ReasoNet: Learning to Stop Reading in Machine Comprehension:Yelong Shen(Microsoft Research)

• Stochastic Answer Net

• [duplicate] 18: Stochastic Answer Networks for Machine Reading Comprehension: Xiaodong Liu(Microsoft Research,Redmond), code

Task-oriented dialogues

overview

A Example Dialogue with Movie-Bot

• source code

Conversation as Reinforcement Learning

• 00: A Stochastic Model of Human-Machine Interaction for Learning Dialog Strategies: Esther Levin(IEEE)

• 00: Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System:Satinder Singh(AT&T Labs)

• 07: Partially observable Markov decision processes for spoken dialog systems:Jason D.Williams(AT&T Labs)

Dialogue System Evaluation(Simulated Users)

• Agenda based

• 09: The Hidden Agenda User Simulation Model:Jost Schatzmann(IEEE)

• source code

• Model based

• 16: A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems: Layla El Asri(Maluuba Research)

• 17: End-to-End Task-Completion Neural Dialogue Systems:Xiujun Li(Microsoft Research&National Taiwan University)

traditional approache

Decision-theoretic View of Dialogue Management

• [duplicate] 00: Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System:Satinder Singh(AT&T Labs)

• 00: A Stochastic Model of Human-Machine Interaction for Learning Dialog Strategies: Esther Levin(IEEE)

• 00: Learning Optimal Dialogue Strategies: A Case Study of a Spoken Dialogue Agent for Email: Marilyn A.Walker(ATT Labs Research)

• 02: Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning:Konrad Scheffler(Cambridge University)

Language Understanding Uncertainty: POMDP as a principled framework

• 00: Spoken Dialogue Management Using Probabilistic Reasoning: Nicholas Roy(Carnegie Mellon University)

• 01: Spoken Dialogue Management as Planning and Acting under Uncertainty:Bo Zhang(Tech. of China)

• 07: Partially observable Markov decision processes for spoken dialog systems:Jason D.Williams(AT&T Labs)

scaling up Dialogue Optimization

• Use approximate POMDP algorithms leveraging problem-specific structure

• 00: Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning:Konrad Scheffler(Cambridge University)

• 07: Partially observable Markov decision processes for spoken dialog systems:Jason D.Williams(AT&T Labs)

• Use Reinforcement Learning algorithms with function approximation

• 08: Hybrid Reinforcement/Supervised Learning of Dialogue Policies from Fixed Data Sets: James Henderson

• 09: Reinforcement Learning for Dialog Management using Least-Squares Policy Iteration and Fast Feature Selection: Lihong Li(Rutgers University)

• 14: Incremental on-line adaptation of POMDP-based dialogue managers to extended domains:M.Gasic[Cambridge University]

Natural language understanding and dialogue state tracking

Language Understanding

• DNN for Domain/Intent Classification

• 15: Recurrent Neural Network and LSTM Models for Lexical Utterance Classification: Suman Raviuri(University of California,Berkeley)

• Slot filling

• 16: Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM: Dilek Hakkani-Tur(Microsoft Research)

• Further details on NLU

• ppt

• E2E MemNN for Contextual LU: End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding: Yun-Nung Chen(National Taiwan University )

• [duplicate] LU Importance: 17: End-to-End Task-Completion Neural Dialogue Systems:Xiujun Li(Microsoft Research&National Taiwan University)

Dialogue State Tracking(DST)

• DSTC(Dialog State Tracking Challenge)

• DSTC1 official website

• DSTC2&3 official website

• DSTC4 official website

• DSTC5 official website

• Neural Belief Tracker

• 16: Neural Belief Tracker: Data-Driven Dialogue State Tracking: Nikola Mrksic(University of Cambridge)

• NN-Based DST

• 13: Deep Neural Network Approach for the Dialog State Tracking Challenge: Matthew Henderson(University of Cambridge)

• 15: Multi-domain Dialog State Tracking using Recurrent Neural Networks: Nikola Mrksic(University of Cambridge)

• [duplicate] 16: Neural Belief Tracker: Data-Driven Dialogue State Tracking: Nikola Mrksic(University of Cambridge)

Deep RL for dialogue policy learning

Two main classes of RL algorithms

• Value function based:

• 15: Human-level control through deep reinforcement learning: Volodymyr Minh

• code1 by tensorflow

• code2 by pytorch

• 16: Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning: Tiancheng Zhao(Carnegie Mellon University)

• Policy based:

• 92: Simple statistical gradient-following algorithms for connectionist reinforcement learning: Ronald J.Williams

• 17: On-line Active Reward Learning for Policy Optimisation in Spoken Dialogue Systems: Pei-Hao Su(University of Cambridge)

Domain Extension and Exploration(BBQ network)

• 18: BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems: Zachary Lipton(Carnegie Mellon University)

Composite-task Dialogues

• A Hierarchical Policy Learner

• 98: Reinforcement Learning with Hierarchies of Machines: Ronald Parr(UC Berkeley)

• 17: Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning: Baolin Peng(Microsoft Research)

• Integrating Planning for Dialogue Policy Learning

• 18: Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning: Baolin Peng(Microsoft Research) , code

Decision-theoretic View of Dialogue Management

Hybrid Code Networks

• 17: Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning: Jason D. Williams(Microsoft Research)

Differentiating KB Accesses

• [duplicate] 17: Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access:Bhuwan Dhingra(Carnegie Mellon University)

An E2E Neural Dialogue System

• [duplicate] 17: End-to-End Task-Completion Neural Dialogue Systems:Xiujun Li(Microsoft Research&National Taiwan University)

Fully data-driven conversation models and chatbots

Historical overview

Response retrieval system

• 10: Filter, Rank, and Transfer the Knowledge: Learning to Chat: Alan Ritter(University of Washington)

Response generation using Statistical Machine Translation

• 11: Data-Driven Response Generation in Social Media: Alan Ritter(University of Washington)

First neural response generation systems

• Neural Models for Response Generation

• 15: A Neural Network Approach to Context-Sensitive Generation of Conversational Responses: Alessandro Sordoni(University de Montreal)

• 15: A Neural Conversational Model: Oriol Vinyals(Google .Inc)

• 15: Neural Responding Machine for Short-Text Conversation: Lifeng Shang(Noah’s Ark Lab), code

• Neural conversation engine:

• 16: A Diversity-Promoting Objective Function for Neural Conversation Models: Jiwei Li(Stanford University)

challenges and remedies

Challenge: The blandness problem

• [duplicate] 16: A Diversity-Promoting Objective Function for Neural Conversation Models: Jiwei Li(Stanford University)

Challenge: The consistency problem

• Solution: Personalized Response Generation

• Microsoft Personality chat:speaker embedding LSTM: A Persona-Based Neural Conversation Model: Jiwei Li(Stanford University), code via Pytorch

• Personal modeling as multi-task learning

• 17: Multi-Task Learning for Speaker-Role Adaptation in Neural Conversation Models: Yi Luan(University of Washington)

• Improving personalization with multiple losses

• 16: Conversational Contextual Cues: The Case of Personalization and History for Response Ranking: Rami Al-Rfou(Google .Inc)

Challenge: Long conversational context

• It can be challenging for LSTM/GRU to encode very long context

• 18: Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context: Urvashi Khadelwal(Stanford University)

• Hierarchical Encoder-Decoder(HRED), code

• 16: Building End-to-End Dialogue Systems Using Generative Hierarchical Neural Network Models: Iulian V.Serban(University de Montreal), code

• Hierarchical Latent Variable Encoder-Decoder(VHRED)

• 17: A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues: Iulian V. Serban

Grounded conversation models

A Knowledge-Grounded Neural Conversation Model

• 15: End-To-End Memory Networks: Sainbayar Sukhbaatar(New York University)

• code1 via Tensorflow

• code2 via Tensorflow

• code3 for bAbI QA tasks

• 17: A Knowledge-Grounded Neural Conversation Model: Marjan Gahzvininejad(USC)

Grounded E2E Dialogue Systems

• 16: Visual Dialog: Abhishek Das(Georgia Institute of Technology)

• code1 via Lua

• code2 via Pytorch

• 17: Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation: Nasrin Mostafazadeh(University of Rochester)

• 18: Emotional Dialogue Generation using Image-Grounded Language Models:Bernd Huber(Harvard University)

Beyond supervised learning(Deep Reinforcement Learning for E2E Dialogue)

• 16: Deep Reinforcement Learning for Dialogue Generation:Jiwei Li(Stanford University)

• code1 via Tensorflow

• code2 via keras

• code3 by Jiwei Li

Data and evaluation

Conversational datasets(for social bots, E2E dialogue research)

• 15: A Survey of Available Corpora for Building Data-Driven Dialogue Systems: Iulian Vlad Serban(Universite de Montreal)

Evaluating E2E Dialogue Systems via Automatic evaluation

• Machine-Translation-Based Metric

• 02: BLEU: a Method for Automatic Evaluation of Machine Translation: Kishore Papineni(IBM), code

• 02: Automatic Evaluation of Machine Translation Quality Using N-gram Co-Occurrence Statistics: George Doddington

• Sentence-level correlation of MT metrics:

• 16: How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation: Chia-Wei Liu(McGill University)

• 15: Accurate Evaluation of Segment-level Machine Translation Metrics: Yvette Graham(The University of Melbourne)

The importance of sample size

• [duplicate] 02: BLEU: a Method for Automatic Evaluation of Machine Translation: Kishore Papineni(IBM), code

• 06: Statistical Significance Tests for Machine Translation Evaluation: Philipp Koehn(MIT)

Corpus-level Correlation

• [duplicate] 02: BLEU: a Method for Automatic Evaluation of Machine Translation: Kishore Papineni(IBM), code

• [duplicate] 06: Statistical Significance Tests for Machine Translation Evaluation:

Chatbot in public

Social Bots: commercial systems

• For end users

• Replika.ai system description: replika_ai: Slides

• XiaoIce: 15:Chatting With Xiaoice: News

• For bot developers

• [duplicate] Microsoft Personality chat:speaker embedding LSTM: A Persona-Based Neural Conversation Model: Jiwei Li(Stanford University), code via Pytorch

• Microsoft Personality chat’s API: Project Personality Chat’s url

Open Benchmarks

• Alexa Challenge

• website: Alexa Prize Proceedings

• Dialogue System Technology Challenge(DSTC)

• DSTC7

• Visual-Scene: DSTC7-Audio-Visual-Scene-Aware-Dialog-AVSD-Challenge 2018

• background article: DSTC7-End-to-End-Conversation-Modeling 2018

• Registration Link: DSTC7 Registration