Psychometrics – it’s a science – not wizardry

Psychometrics is concerned with psychological (psycho) measurement (metrics).

The practitioners of psychometrics are known as psychometricians – not to be confused with the homophonous label of “psycho-magicians” – even if measuring psychology may be considered opaque or even mystical by some.

A “metric” is a means of quantifying a characteristic, quality or phenomenon. Psychometrics in this sense is often referred to as “quantitative psychology”: the science of measuring or approximating latent (unobserved) psychological traits. In the context of educational assessment, psychometrics may be thought of as the approximation of the latent traits of e.g. intelligence, ability or skills using measurement instruments such as tests, exams and other assessment tools.

In the UK, the discipline of psychometrics is gaining traction in educational measurement: the Solicitors Qualifying Examination (SQE) involved significant statistical and psychometric modelling from the design stage onwards. Nevertheless, in the UK fully fledged psychometricians are relatively few and far between and it is perhaps safe to say that the term psychometrics is most often associated with aptitude or personality testing used in screening processes for selection and recruitment activities. However, in other parts of Europe and North America in particular, psychometricians are far more prevalent and the science (or art) of psychometrics is called upon throughout the life-cycle of educational assessment and underpins the design, standard-setting, delivery, evaluation and accountability of assessment.

Psychometrics has applications in the development and validation of both low-stakes and formative assessments (e.g. frequent look, snapshot knowledge testing in schools) and in high-stakes educational assessment (e.g. professional licensure examinations). Working alongside subject matter experts and other assessment specialists, the goal of the psychometrician is to ensure that educational measurement instruments are valid, reliable, fair and fit-for-purpose.

Psychometrics is by no means a new discipline; in an 1879 essay simply entitled “Psychometric Experiments” psychometrics was elegantly described as “…the art of imposing measurement and number upon operations of the mind…”*. As with most scientific measurement – psychometric measurement instruments such as tests and exams are subject to measurement error – any test is an approximation of the test-takers ability at a snapshot in time, and has inherent imprecision. The idea is to develop tests and exams that minimise measurement error, and more so, to quantify and account for it insofar as it is possible to do so.

A personal favourite definition of psychometrics is from Dutch psychometrician Denny Borsboom: "Psychometrics is a scientific discipline concerned with the construction of assessment tools, measurement instruments, and formalised models that may serve to connect observable phenomena (e.g., responses to items in an IQ-test) to theoretical attributes (e.g., intelligence).**

Broadly then, the science of psychometrics is useful in all stages of the assessment life-cycle. In the context of high-stakes assessment, for example where an agency creates an assessment for licensure (such as our own UK Solicitors Qualifying Examination or the US Medical Licensing Examination for example), psychometrics plays a crucial role in the areas of evaluating assessment reliability and validity and in standard-setting. In such cases where an assessment standard is called for through a statement of policy, psychometric analysis of the assessment translates the intended standard implied by the policy to a reporting score scale for the test in a fair and equitable way.

Interested in bespoke assessments for your organisation?

Get in touch with our expert team who will work with you to get the right solutions for you and your employees.

*Galton, F. (1879). Psychometric experiments. Brain: A Journal of Neurology, 11, 149-162.
**The Psychometric Society (psychometricsociety.org)