PIG: Decoding Personality through the Lens of Social Interaction Graphs

TL;DR

Personality isn't just about who you are; it's about how you change when others are around. This research moves Automatic Personality Assessment (APA) from the sterile lab to the "wild" world of TV sitcoms. By introducing Personality Interaction Graphs (PIG), the framework maps the Big Five traits (OCEAN) onto dynamic social networks, capturing how group structures influence individual behavior.

Background & Positioning

Most existing personality recognition systems treat the subject as an isolated entity in front of a camera. This Ph.D. project by Amanjot Kaur (IIT Ropar) argues that such an approach ignores the Inductive Bias of social context. It positions itself as a bridge between computer vision and social psychology, moving beyond simple trait prediction to "context-aware" behavioral understanding.

The Problem: The "In the Wild" Challenge

Moving from controlled datasets to real-world scenarios (like the Big Bang Theory dataset used here) introduces three major bottlenecks:

1. Technical Noise: Frequent occlusions, head gestures, and lighting changes make face tracking and Gaze Analysis unreliable.
2. Temporal Dynamics: Personality isn't a single frame; it's a pattern over varying time segments.
3. Contextual Blindness: A person may appear "Neurotic" in a high-stress meeting but "Extroverted" in a park. Current models fail to distinguish these states without group information.

Methodology: Building the Personality Interaction Graph (PIG)

The core innovation lies in the Personality Interaction Graph (PIG). Here is how the system moves from raw pixels to social insights:

1. Subject Specific Face Tracks (SSFT)

The system performs subject re-identification. Using LBP (Local Binary Pattern) and HOG (Histogram of Oriented Gradients) features fed into a one-vs-all SVM, the model tracks actors even as they exit and re-enter frames.

2. Feature Extraction & OCEAN Mapping

The model extracts multimodal cues:

• Facial Action Units (AUs): Using OpenFace to detect muscle activations.
• Gaze & Head Pose: To determine attention within the group.
• Visual Context: GIST and CENTRIST descriptors to identify the environment (e.g., office vs. cafe).
• OCEAN Prediction: An SVM regressor, pre-trained on the ChaLearn dataset, maps these features to the five personality dimensions.

3. Graph Evolution

The PIG connects individuals. The edge weights are defined by the difference between an individual’s score and the group's "mean OCEAN score."

Figure 1: Visualizing how the social graph changes across different frames as actors interact.

Experiments and Results

The framework was tested on data from The Big Bang Theory. A key observation was the "Penny Effect": the personality scores of characters like Sheldon and Leonard shifted significantly when the character Penny entered the graph.

The study found that:

• Maximum deviation from the mean OCEAN score often correlates with highly influential social topics or "outlier" personalities in the group.
• When character scores converge toward the mean, it indicates high group cohesion or a shared emotional situation.

Figure 2: The final output shows how group relationships shape and evolve over time.

Critical Insight & Conclusion

Takeaway

The shift from "Individual APA" to "Group-based APA" is essential for building truly empathic machines. The use of graphs allows us to treat personality as a dynamic relationship rather than a static label.

Limitations & Future Work

The current reliance on "weak labels" from subtitles is a bottleneck. The author plans to incorporate temporal deep learning (e.g., LSTMs or GRUs) and explore Indian-specific datasets to handle cultural nuances in personality expression. Furthermore, the integration of body pose and clothing segmentation (which signals social status or mood) remains a promising future direction.

In summary, this work provides a robust roadmap for making AI understand not just who we are, but how we relate to the world around us.