Does Repeating Social Network Questions Ruin Data? Insights from a Longitudinal Web Study

TL;DR

When we ask people about their friends in a survey, and then ask dozens of follow-up questions about those friends, do they learn to lie the next time to save time? This study finds that for most people, the answer is no. While web-based social network modules maintain high data quality over time, respondents with very large social circles might start cutting corners in later survey waves—a phenomenon rooted in response burden.

The "Burdensome" Nature of Social Networks

Ego-centered social network modules are notoriously repetitive. They usually follow a specific flow:

1. Name Generator: "List your closest friends."
2. Name Interpreter: "How old is Peter? Where does Clara live?"
3. Relationship Ties: "Do Peter and Clara know each other?"

In a longitudinal study (a "panel"), a respondent who realizes that naming 10 people triggers 50 more questions might strategically name only 2 people in the second wave. This is known as Motivated Underreporting.

The Experiment: Testing the Impact of Repetition

The researchers conducted a three-wave online panel with a randomized control trial to isolate the effect of "experience."

• Treatment Group: Answered the network module in Wave 1, Wave 2, and Wave 3.
• Control Group: Answered filler questions in Wave 1 and only saw the network module in Wave 2.

By comparing these groups in Wave 2, the researchers could tell if having "done it before" changed how people answered.

Results: Quality Remains Surprisingly High

Contrary to the fears of many methodologists, the data quality didn't plummet.

• Low Non-response: Less than 5% of items were left blank.
• Consistent Size: The average network size remained stable (around 3.3 names) across waves.
• Validation: The results matched sociological expectations (e.g., extraverts had larger networks).

The "Regression to the Mean" Twist

While the average stayed the same, the individuals changed.

• People with small networks in Wave 1 tended to report more friends later.
• People with large networks (4+ friends) reported significantly fewer friends in Wave 2 (dropping from 5.56 to 4.72).

This suggests that while the "general" respondent is honest, those who feel the "burden" of a large network may indeed start satisficing to reduce their workload.

Methodological Deep Dive: Why it Matters

This study challenges the assumption that self-administered web surveys are more prone to "slacking" than interviewer-led ones. Without an interviewer present to "nudge" for more names, respondents were still diligent.

The lack of moderation by Ability (education) or Motivation (Need for Cognition) suggests that the tendency to underreport is less about personality and more about the structural burden of the questionnaire itself.

Conclusion & Key Takeaways

1. Web Surveys are Viable: You can safely collect complex social network data online without massive quality loss.
2. Watch the "Heavy Hitters": Be cautious when analyzing structural changes in large networks over time; some "shrinking" might just be respondent fatigue.
3. Future Research: The researchers suggest exploring if even longer modules (e.g., involving political views or complex demographics of friends) might eventually trigger the panel conditioning effects that this shorter version avoided.

Reference: Silber, H., et al. (2018). Does panel conditioning affect data quality in ego-centered social network questions? Social Networks.