Psychometric Validity
Evidence
Rigorous psychometric evidence is the foundation of any trustworthy assessment platform. This page presents b4skills' ongoing concurrent validity research — how our adaptive CEFR scores correlate with gold-standard external tests.
Concurrent Validity
Pearson r and Spearman ρ between b4skills θ estimates and criterion scores. Industry benchmark for high-stakes certification: r ≥ 0.85 (IELTS/TOEFL).
Annual Item Bank Health Report
Aggregate item statistics for the 2026 assessment year. Only items currently marked ACTIVE are counted. Active status alone does not establish empirical calibration or a minimum response count. Session statistics cover completions in the last 12 months.
Psychometric Framework
Item Response Theory (3PL)
Objective-item scoring uses a three-parameter logistic model with discrimination (a), difficulty (b), and pseudo-guessing (c). Parameters may be provisional; empirical calibration must be demonstrated with response-level evidence rather than inferred from active status.
CAT — EAP θ Estimation
The adaptive engine uses Expected A Posteriori (EAP) θ estimation with a normal N(0,1) prior. Item selection maximises Fisher information at the current θ estimate. Stopping rules use product-specific precision targets and item limits; reaching an item limit does not guarantee the precision target.
CEFR Alignment
CEFR levels are derived using the platform’s configured thresholds. They are provisional alignment estimates. No completed independent standard-setting or external validation study is presented here.
DIF & Fairness Analysis
The platform includes Mantel-Haenszel DIF screening tools. Meaningful fairness conclusions require sufficient responses in comparison groups; the availability of these tools does not establish that the item bank is bias-free.
AI Scoring Governance
Full assessments support AI-assisted writing and speaking scoring using OpenAI. Review requirements depend on assessment configuration. No independently validated human–AI agreement coefficient is claimed. The free Quick Check does not evaluate open writing or speaking responses.
Concurrent Validity Design
Concurrent validity pairs require the external test to be taken within ±30 days of the b4skills assessment. Both self-reported and verified (official document) data sources are tracked separately in all analyses.
Key References
- Baker, F. B., & Kim, S. H. (2004). Item Response Theory: Parameter Estimation Techniques (2nd ed.). Marcel Dekker.
- Bland, J. M., & Altman, D. G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327, 307–310.
- Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge University Press.
- Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73.
- Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum.
- Reckase, M. D. (2009). Multidimensional Item Response Theory. Springer.
- Wainer, H. (Ed.). (2000). Computerized Adaptive Testing: A Primer (2nd ed.). Lawrence Erlbaum.
Collaborate with Us
We are actively seeking university partners for concurrent validity studies. If your institution can provide IELTS/TOEFL/Cambridge scores alongside b4skills assessments, we would like to hear from you.
hello@b4skills.com