A self-report tool for four transversal competencies
The instrument does not test whether students learned physiology content. It asks students to rate themselves on four cross-disciplinary skills that immersive environments may foster: conceptual understanding, evidence-informed decision-making, ICT operation, and application to practice. Items were written in positive declarative form and rated on a five-point Likert scale from 1 (strongly disagree) to 5 (strongly agree), administered online and taking roughly 10 to 12 minutes.
That distinction matters for interpreting every number below. A high score is a student's self-assessment of a transversal skill, not a performance measure, and the paper frames validity as a property of the inferences drawn from scores in a specific context rather than a fixed attribute of the questionnaire.
From 34 pilot items to a 26-item instrument
Development ran in phases. An initial pool of 34 items spanned seven preliminary dimensions and was reviewed by seven experts in educational technology, pedagogy, and psychometrics, who rated relevance and clarity; content validity was quantified with Aiken's V, with items below 0.70 revised and items below 0.50 candidates for elimination. The revised instrument then went to a pilot sample: of 347 enrolled students, 322 returned complete responses, a 92.8% response rate.
Exploratory factor analysis on those 34 items used maximum likelihood extraction with Oblimin oblique rotation, chosen because the constructs were expected to correlate. Sampling adequacy was high (KMO = 0.966) and Bartlett's test was significant (chi-square = 10163.824, df = 561, p < 0.001). Four factors with eigenvalues above 1 emerged, together explaining 76.7% of total variance.
The empirical structure did not match the seven theoretical dimensions. Items from Interpersonal Skills, Permanent Learning, and Adaptability were redistributed across the four retained factors, and eight items were dropped for loadings below 0.40 or cross-loadings differing by less than 0.10. The final instrument has 26 items: 5 for Conceptual Understanding, 4 for Evidence-Informed Decision-Making, 5 for ICT Operation, and 12 for Application to Practice, which absorbed items originally labelled Interpersonal, Practice, and Permanent Learning.
Very high reliability alongside a dominant general factor
Internal consistency was unusually strong: Cronbach's alpha of 0.993, with corrected item-total correlations from 0.752 to 0.943. Before rotation, the first factor alone accounted for 62.4% of variance, which the authors read as a strong general competency dimension underlying the instrument rather than four fully independent skills. After rotation, variance spread across the four factors with sums of squared loadings of 19.761, 12.515, 15.502, and 10.933.
That reading is the authors' interpretation, and it shapes how the scores should be used. If the four constructs share a large common core, a single total score and four subscale scores are not interchangeable claims about distinct skills; the paper's own framing treats them as interrelated facets of a broader transversal competency framework.
Confirmatory fit lands at the acceptance boundary
The four-factor model was tested in a separate validation sample of 58 students (92.1% response rate). Fit was acceptable but not comfortable: CFI = 0.903 meets the conventional 0.90 threshold, TLI = 0.891 falls marginally below it, and RMSEA = 0.086 exceeds the preferred 0.08. The authors attribute part of this to multivariate non-normality (Mardia's multivariate kurtosis = 714.400), which is known to inflate chi-square-based fit statistics.
Convergent and discriminant evidence was stronger. Standardized loadings ranged from 0.669 to 0.925, composite reliability was 0.929 for Conceptual Understanding, 0.898 for Decision-Making, 0.951 for ICT Operation, and 0.980 for Application to Practice, and average variance extracted exceeded 0.50 for all four constructs. Every HTMT ratio between construct pairs stayed below 0.85.
The sample size is the constraint to keep in view here. The 58-student validation sample gives a response-to-item ratio of 2.23:1, below conventional recommendations; the authors argue that the high loadings partially mitigate this, but it limits how firmly the structural model can be judged.
Self-report scores without external benchmarks
Two validity strands are thin. Response process validity is preliminary: high response rates in both phases suggest engagement, but no cognitive interview or think-aloud protocol checked whether respondents interpreted items as intended. Criterion, concurrent, and predictive validity are absent, so the paper cannot show that questionnaire scores track performance assessments or instructor ratings of the same competencies.
The setting is equally specific. Validation drew on one undergraduate physiology course at the Universidad Católica de Cuenca in Ecuador, using Sharecare YOU VR sessions covering gas transport, cardiovascular function, muscle contraction, and renal function and micturition. The authors state that the evidence supports score-based inferences only in that physiology lab context and that use in other disciplines or institutions requires independent psychometric validation.
A context-bound instrument, not a transferable benchmark
The paper's most decision-relevant contribution is the separation it enforces. Because the questionnaire targets transversal competencies rather than disciplinary mastery, it can sit alongside content assessments instead of replacing them, and its strongest evidence is internal structure and reliability rather than prediction of performance.
For readers weighing whether to adopt it, the practical split follows the evidence: within a VR-supported undergraduate physiology lab, the 26-item instrument has documented content validity, a four-factor structure, and high composite reliability, with fit indices at the edge of acceptance and a small confirmatory sample. Beyond that setting, or wherever scores would be used to predict observed competence rather than describe self-reported development, the paper provides no support and explicitly asks for new validation.
Sources used in this answer
Design and validation of a questionnaire to measure competency development in VR‑supported undergraduate physiology labs
A 26-item self-report questionnaire on four transversal competencies showed Cronbach's alpha of 0.993, a four-factor structure explaining 76.7% of variance, and acceptable but marginal confirmatory fit (CFI = 0.903; TLI = 0.891; RMSEA = 0.086) in a physiology lab context only.