Engineering the Truth: Does Gender Actually Bias Student Ratings in Computer Science?

Executive Summary

TL;DR: After analyzing over 13,000 student evaluations in Sweden over a decade, researchers found that while gender interactions exist, their actual impact on teaching scores is practically zero. In a surprising twist, female teachers in male-dominated fields like Computer Science often receive higher marks for clarity than their male colleagues.

Positioning: This work serves as a rigorous, data-driven "myth-buster" in the ongoing debate over whether Student Evaluations of Teaching (SET) are a valid tool for personnel decisions or merely a vehicle for gender bias in STEM.

The "Boring" Contradiction: Motivation for the Study

For years, the academic community has been divided. On one side, large-scale studies suggest SETs are biased against women. Specifically, a 2015 study by Boring argued that male students systematically undervalue female instructors. On the other side, experts like Feldman have long maintained that these effects are "substantively negligible."

The authors of this paper noticed a gap: most research focused on social sciences. They chose to look at the "hard" sciences—specifically Computer Science (7% female students) and Environmental Engineering (60% female students)—to see if the "maleness" of the discipline acted as a catalyst for bias.

Methodology: The CEQ Framework

The team utilized the 23-item Course Experience Questionnaire (CEQ), a psychometrically validated instrument. Unlike "in-house" surveys, the CEQ assesses specific dimensions of the learning environment rather than just "likability."

Dimensions Measured:

• Good Teaching: Feedback quality and instructor support.
• Clear Goals: Transparency of expectations.
• Appropriate Workload: Volume of material.
• Appropriate Assessment: Testing for understanding vs. memory.
• Generic Skills: Problem-solving and communication.

Key Findings: The "Atypicality" Boost

The most striking discovery was that gender bias didn't follow a simple "pro-male" path. Instead, it followed a pattern of Subject Atypicality:

1. In Computer Science: Female teachers were rated higher on clarity and workload management than men.
2. In Environmental Engineering: Male teachers were rated higher on teaching quality and clarity than women.

Essentially, instructors received a "bonus" when teaching in a field where their gender was the minority.

The Interaction Effect

The data showed that male students tend to give higher ratings to male teachers for "Good Teaching," while female students give higher ratings to female teachers for "Clear Goals." However, the authors are quick to point out a critical statistical reality: Effect Size.

Results & Critical Analysis: Magnitude vs. Significance

While the p-values were "highly significant" (p < 0.001), the Partial η² (measure of effect size) was consistently below 0.005.

• Gender Effect: < 0.5% of variance.
• Course Content Effect: ~20% of variance.

This means that what you teach and how the course is designed is roughly 40 times more important than your gender or the gender of your students. If a teacher gets a low score, it is almost certainly because the course is poorly structured, not because of their gender.

Final Takeaway: Stop Blaming the Students

The study concludes that using gender bias as an excuse for the lack of female career progression in engineering is "implausible." If women are not advancing to senior faculty roles, the cause lies in institutional hiring and promotion policies, not in the feedback forms filled out by 20-year-old undergraduates.

Limitations: The study is confined to a single Swedish institution, which may have higher levels of gender egalitarianism than other regions. Future research should replicate this "atypicality" check in different cultural contexts.