[IJCNN 2020] TERG: Bridging the Gap Between Emotional Expression and Topic Relevance in Chatbots

TL;DR

While modern chatbots have become proficient at mimicking human language, they often struggle with two conflicting goals: expressing specific emotions and staying on topic. Most "emotional" models fall into the trap of generating generic emotive phrases like "I am sad" regardless of the context. TERG (Topic-aware Emotional Response Generation) solves this by fusing a CVAE-based emotion module with a Topic Commonsense-Aware (TCA) module, ensuring that the bot doesn't just feel—it talks about what matters.

The Motivation: Why Your Chatbot is Emotional but "Dumb"

The "Safe Response" problem has long plagued Seq2Seq models. When you add an emotion constraint, the problem worsens; a bot told to be "Happy" might reply "I am so happy!" to every input.

The authors identify two critical failures in prior work (like ECM):

1. Semantic Vacuum: Emotion factors dilute the actual meaning of the response.
2. Topic Drift: The model loses the "thread" of the conversation, ignoring context-specific entities (e.g., if you talk about basketball, the bot should mention the game, not just say "I love it").

Methodology: The Architecture of Feeling and Logic

The TERG model is built on an encoder-decoder backbone but introduces three sophisticated components to handle the dual constraints of emotion and topic.

1. Topic Commonsense-Aware (TCA) Module

Instead of relying solely on the input hidden states, TERG uses the Biterm Topic Model (BTM)—an unsupervised model optimized for short texts—to extract a set of topic-related words for every message. These are passed through a multi-attention fusion mechanism.

• Context Attention: Focuses on the sequence of the input message.
• Topic Attention: Focuses on the external topic lexicon (e.g., if the topic is "Basketball", words like "NBA" or "Kobe" get higher weights).

2. Emotion-Aware (EA) Module via CVAE

Using a Conditional Variational Autoencoder (CVAE), the model projects different emotional responses into a latent space. During training, an "Emotion Supervisor" predicts the emotion label from the latent variable $z$, forcing $z$ to capture the nuances of how a specific emotion is phrased.

3. Topic-aware Emotion (TE) Decoder

The real magic happens at the output stage. The decoder doesn't just pick the next word; it uses a word type selector to decide if the next token should be:

• An Emotional Word (e.g., "Darling", "Disgusting")
• A Keyword (e.g., "Eggs", "Ultraman")
• An Ordinary Word (e.g., "is", "the")

Fig 1: The TERG architecture showing the interaction between the CVAE latent space and the Topic Commonsense-Aware module.

Experiments & Results: Real Talk, Not Safe Talk

The researchers evaluated TERG against standard Seq2Seq and the Emotional Chatting Machine (ECM).

Key Findings:

• Topic Relevance: TERG achieved a score of 0.608, nearly doubling the performance of basic Seq2Seq (0.271).
• Fluency: Perplexity dropped to 63.95, indicating that the responses are more natural and less "stuttery."
• Informative Content: Case studies show that while ECM says "Haha, food," TERG says "Darling, do you want to eat eggs?"—maintaining both the 'Like' emotion and the 'Egg' topic.

Fig 2: Line chart showcasing how TERG's topic relevance scores closely mirror (and sometimes exceed) real human Weibo conversations.

Critical Analysis & Conclusion

The value of TERG lies in its explicit control mechanism. By splitting the vocabulary and using an external topic model, the researchers provide a roadmap for building bots that are both empathetic and knowledgeable.

Takeaway: An intelligent chatbot needs an "Internal World" (Emotion) and "External Context" (Topic). TERG shows that these two can be fused using attention mechanisms without losing the benefits of latent variable modeling.

Limitations: Since the data is sourced from Weibo, it inherits the "noise" and colloquialisms of social media. Future work could improve the logical consistency of responses over multiple turns of dialogue, as TERG currently focuses primarily on single-turn responses.