Ranking Educational Videos: Why "Social Presence" Outperforms Simple Keywords

TL;DR

Searching for quality educational content is often a hit-or-miss experience. This paper bridges the gap between what a video says (content) and how much users trust it (social presence). By augmenting standard Cosine Similarity with a "Social Weight" parameter ($Likes/[Likes+Dislikes]$), the authors created a ranking system that aligns 8.11% more closely with real human preferences than traditional keyword searches.

Context & Motivation: The "View Count" Trap

In the realm of educational videos, not all relevance is created equal. A video might mention "Machine Learning" 100 times but be pedagogically useless. Previous systems failed because they either focused purely on text (TF-IDF) or relied on raw view counts.

The authors argue that view counts are a noisy metric—they don't tell you if the viewer learned anything. Instead, they pivot to "Social Presence"—the explicit, registered actions of "Likes" and "Dislikes," which represent a higher-intent qualitative judgment.

Methodology: Fusing Math with Sentiment

The researchers developed a two-step approach to re-rank YouTube lectures:

1. The Content Foundation

First, they convert video transcripts into high-dimensional vectors using TF-IDF. Similarity ($s$) is calculated via Cosine Similarity: $s=\cos (\vec{q},\vec{d})=\frac{\vec{q}\bullet \vec{d}}{|\vec{q}\Vert \vec{d}|}$ This provides the baseline "relevance" based on the spoken words in the lecture.

2. The Social Catalyst

To refine this, they introduce the Social Weight ($sw$): $sw=\frac{likes}{likes+dislikes}$ The final Social-Content Similarity ($scs$) is: $scs=simes(1+sw)$ This formula ensures that social weight acts as a "bonus." If a video has perfect social feedback ($sw=1$), its score doubles. If it has no feedback or only dislikes ($sw=0$), it retains its original content score but gains no advantage.

Fig 1: Distribution showing that while likes generally outweigh dislikes in educational content, social weight helps distinguish the "top tier" videos.

Real-World Experiments

The authors tested this on 1,116 English-transcribed YouTube videos across 40 scientific keywords.

• The Stability of Relevance: 43% of rankings remained unchanged, while 57% shifted.
• Gradual Improvement: Most shifts (81.7%) were within 1-4 positions. This is crucial—it means the social weight "nudges" better videos to the top rather than causing chaotic re-ordering.
• The "Floor" Effect: A video with low relevance cannot be "saved" by high social weight, but a highly relevant video can be significantly penalized by high dislikes.

Fig 2: Most videos saw a positive "boost" in ranking, indicating that social signals generally identify high-quality educational content.

Human Evaluation: The Ultimate Litmus Test

Does this actually matter to learners? The authors conducted a user study where 15 rankers evaluated "Database" lectures.

Using Mean Average Precision (MAP), they found:

• Content-Only Prediction: 19.23% accuracy.
• Social-Content Prediction: 27.39% accuracy.

Qualitative interviews revealed a fascinating insight: Users are "lazy" with dislikes but "generous" with likes. People stop watching bad videos without disliking them, but actively hit the "like" button when they find a direct, well-spoken, and simple explanation. This justifies the authors' decision to focus on the ratio of likes rather than raw volume.

Critical Insights & Conclusion

While this paper was written in a specific era of YouTube's API (v2), its core insight remains timeless for the AI age: Human feedback (RLHF-style) is the strongest filter for quality.

Limitations: The model heavily relies on the availability of transcripts. In the current era of Auto-ASR (Automatic Speech Recognition), this is less of a hurdle, but noise in ASR could impact the cosine similarity accuracy.

Future Outlook: Integrating advanced NLP to analyze comment sentiment instead of just "Like/Dislike" buttons would likely bridge the MAP gap even further. As we move toward AI-curated education, blending vector-space semantics with social consensus is the only way to ensure the most "useful"—not just the most "relevant"—content rises to the top.