Why haven't learning analytics systems been tested in actual classrooms?
The biggest gap is that most learning analytics systems are built and studied in labs or short-term pilots, not in the messy reality of a real school term. A two-year longitudinal study with 399 students and 17 teachers found that even a well-designed multimodal learning analytics (MMLA) system—which tracks physical and physiological signals during group work—was only evaluated in one specific healthcare education context [1]. The researchers themselves noted that the complexity of the system and the need for better qualitative measures of communication were areas for improvement [1]. This means we have very little evidence about how these systems perform when teachers are juggling 30 students, varying tech literacy, and unpredictable schedules.
A systematic review of 27 articles published between 2012 and 2023 confirmed that the field is still in early stages, with most studies focusing on design rather than long-term impact on teaching and learning [3]. Without real-world testing, it is impossible to know whether a dashboard that works in a controlled study will actually help a teacher spot a struggling student before they fail.
Why does the data from learning analytics often tell the wrong story?
A second major gap is that the data collected by learning management systems (LMS) and other platforms is often shallow and misleading. A worked example in health professions education analyzed data from 141 registered users of an online video-based training site and found that generic website analytics—like page views and time on page—did not correspond to meaningful learner engagement [2]. The researchers called these 'data mysteries' rather than 'data stories' because high traffic could mean students were confused and re-watching videos, not that they were learning effectively [2]. This gap is critical because if the raw data is flawed, any insights or dashboards built on it will be unreliable.
Another review focusing on secondary education found that many learning analytics dashboards lack empirical validation—meaning they have not been tested to see if the data they present actually helps teachers detect problems early [4]. The review also noted that systems designed without user-centered approaches and active teacher participation are less likely to be adopted or useful [4]. In short, the field needs better data—data that captures actual learning behaviors, not just clicks.
Why aren't teachers actually using learning analytics insights?
Even when good data is available, teachers often do not act on it. The learning analytics lifecycle framework—which includes planning, data collection, analysis, and action—shows that educators' investment of effort and resources drops off sharply after the data collection stage [2]. In the health professions example, the researchers found a 'relative absence of educators' data-informed actions,' meaning teachers looked at the dashboards but did not change their teaching based on what they saw [2]. This is a fundamental gap: the entire point of learning analytics is to close the loop between data and improved instruction, but that loop is rarely completed.
The systematic review of LMS-based interventions from 2012–2023 also found that while many systems provide dashboards and alerts, there is limited evidence that these tools actually lead to changes in teaching practices or student outcomes [3]. The MMLA study with 399 students and 17 teachers did find that teachers had positive perceptions of the system and found it helpful for facilitating feedback and reflection [1]. However, the study also highlighted the need for clear explanations and guidance on how to interpret analytics, as well as addressing concerns about data completeness and representation [1]. Without training and support, teachers cannot bridge the gap from insight to action.
Why do learning analytics systems fail in secondary schools?
A fourth gap is that most learning analytics research and development has focused on higher education, leaving secondary schools underserved. A systematic review specifically on learning analytics dashboards in secondary education found that there are still 'limitations in terms of accessibility, empirical validation, and adaptation to the specific context of secondary education' [4]. Secondary schools have different constraints—shorter class periods, less tech support, and a greater need for early detection of problems like disengagement or learning difficulties. The review concluded that systems designed with user-centered approaches and active teacher participation foster greater pedagogical appropriation and usefulness [4], but such designs are rare.
The COVID-19 pandemic dramatically increased the role of LMSs in secondary education, as they became the only interface between students and instructors [5]. Yet a study from 2021 noted that the existing body of literature had not analyzed learning in this pandemic context, where an LMS serves as the sole point of contact [5]. This means that the models and dashboards being used in secondary schools today were largely designed for a different era and a different user base. Without context-specific research, learning analytics systems risk being irrelevant or even counterproductive in K-12 settings.
About These Sources
This answer is built on 5 peer-reviewed studies — published from 2021 to 2026, 4 from 2024 or later, 3 in Q1 journals, collectively cited 81 times — selected as the most relevant from 6 studies that passed quality screening, drawn from 50 papers retrieved from a database of over 500 million.
Sources used in this answer
Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning
In a two-year longitudinal study with 399 students and 17 teachers, a multimodal learning analytics system was perceived positively for facilitating feedback and reflection, but the study highlighted gaps in design complexity, interpretability for users with disabilities, and the need for qualitative measures of communication.
Applying a Learning Analytics Lifecycle Framework to Identify Gaps in Digital Learning Design: A Worked Example in Health Professions Education.
Analyzing data from 141 users of an online video-based training site, this worked example found that generic website analytics produced 'data mysteries' rather than meaningful stories, and that educators' investment of effort drops off sharply after data collection, leaving a gap in data-informed action.
A Systematic Review of Learning Analytics
A systematic review of 27 articles from 2012–2023 found that learning analytics interventions within LMSs are still in early stages, with most studies focusing on design rather than long-term impact on teaching and learning practices.
Systematic review on learning analytics to detect problems in secondary education
A systematic review on learning analytics dashboards in secondary education found limitations in accessibility, empirical validation, and adaptation to the secondary school context, and concluded that user-centered design with active teacher participation is key to adoption.
Developing Engagement in the Learning Management System Supported by Learning Analytics
This 2021 study noted that the existing literature on LMSs had not analyzed learning in the pandemic context where the LMS serves as the only interface, and proposed a student engagement model validated by experts and student discussions.