Beyond Logic: Infusing Personality into How AI Describes the World

TL;DR

Standard AI systems describe objects with clinical precision, but humans are rarely that efficient. We over-describe, and we do so in patterns consistent with our personalities. This paper introduces the first Personality-Dependent Referring Expression Generation (REG) framework, which uses Big Five personality traits to predict how different people describe faces, leading to more natural and human-like AI communication.

Background: The "Robotic" Problem in Language Generation

In the field of Natural Language Generation (NLG), a sub-task called Referring Expression Generation (REG) is responsible for identifying a target object in a scene. If you are looking at a group of people, the REG system might say, "the man on the left."

While logically sound, this lacks the "human touch." Humans exhibit vast variation—one person might say "the smiling guy," while another says "the young Asian man with short hair." Existing solutions to this variation are either:

1. Generic: Producing the same fixed output for everyone.
2. Speaker-Dependent: Requiring a massive library of previous descriptions from every single user to learn their style (highly impractical).

The authors suggest a third way: Personality Profiles.

The Insight: Personality as a Linguistic Fingerprint

Why do some people provide more detail than others? The researchers hypothesized that the Big Five personality traits (Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness) influence our "referential behavior"—specifically overspecification.

Overspecification is the act of adding more info than is strictly necessary to identify someone. For example, if there is only one man in a photo, "the man" is enough. Adding "the smiling man" is overspecification.

Data Collection and Methodology

The team built a corpus of 1,810 descriptions from 152 speakers, linking each description to the speaker's Big Five scores. They focused on facial attributes like gender, race, hair length, and even subjective traits like "is young."

The proposed architecture, DT-b5, uses a two-stage process:

1. Initial Selection: Use a standard Incremental Algorithm to get a baseline description that identifies the target.
2. Personality Refinement: Use Decision Tree classifiers (one for each attribute) to decide if extra details should be added based on the person's Big Five scores.

Experimental Results: Personality Wins

The researchers compared their DT-b5 model against a context-only model (DT-scene) and two classic baselines (Greedy and Incremental).

Deep Insight: The DT-b5 model succeeded because it learned when to include "redundant" attributes (like isyoung) that aren't strictly necessary for identification but are typical for certain personality types. Standard algorithms ignore these as "noise," but DT-b5 recognizes them as "style."

Critical Analysis & Future Outlook

Takeaway: This work proves that personality is a viable and efficient "shortcut" to personalizing AI. Instead of needing 1,000 examples of a user's speech, we might only need their personality profile to generate human-like descriptions.

Limitations:

• Attribute Frequency: The study focused on high-frequency physical traits (Gender, Race). Personality might play an even bigger role in "affective" traits (how "friendly" or "aggressive" a face looks), which were too rare in this dataset to be analyzed.
• Cultural Bias: The study was conducted on Brazilian Portuguese speakers; whether personality influences language the same way across different cultures remains an open question.

Future Work: The next frontier involves moving beyond "Attributes" to "Values"—deciding not just whether to mention emotion, but how to describe it (e.g., "radiant" vs. "grinning") based on the speaker's neuroticism or openness.