Skip to content
Cognability

Method

This page sets out how the evaluation is built, how it computes, and where the norms come from. Anyone who wants to make sense of a test result needs to be able to see that.

01

Theoretical framework

Cattell–Horn–Carroll (CHC) — the model the established clinical instruments rest on as well.

The CHC model describes cognitive ability on three levels: a general factor, several broad ability domains, and a great many narrow skills. This evaluation measures five broad domains and forms an overall score from them.

The division is not cosmetic. The five domains correlate only moderately with one another, so they cannot stand in for each other. An overall score on its own conceals the fact that the same number can arise from very different profiles — which is why the report always gives both.

Verbal Comprehension

VC

Measures acquired verbal knowledge and the ability to recognise relations of meaning between concepts and to reason in language. This area keeps growing across the lifespan with education and reading.

  • Verbal Analogies — Grasping relations of meaning between concepts. Crystallised intelligence (Gc) — language comprehension
  • Vocabulary — The breadth and precision of active word knowledge. Crystallised intelligence (Gc) — lexical knowledge

Fluid Reasoning

FR

Measures the solving of unfamiliar problems without recourse to learned knowledge: spotting rules in new material, testing them and carrying them over. This area is held to be the least culturally bound.

  • Figural Matrices — Detecting rules in abstract patterns and applying them to a new case. Fluid intelligence (Gf) — induction
  • Number Series — Detecting the rules behind numerical sequences and reasoning quantitatively. Fluid intelligence (Gf) — quantitative reasoning

Visual-Spatial Processing

VS

Measures the ability to generate, hold and transform visual information in the mind — rotating objects, anticipating folds, judging spatial relations.

  • Mental Rotation — Turning two-dimensional figures in the mind. Visual processing (Gv) — visualisation
  • Paper Folding — Picturing spatial change across several steps. Visual processing (Gv) — spatial relations

Working Memory

WM

Measures how much information you can hold in mind and work on at the same time. Working memory sets the limit on how complex a train of thought may become before it breaks down.

  • Digit Span — Holding and reproducing information over the short term, forwards and backwards. Short-term memory (Gwm) — memory span
  • Letter–Number Sequencing — Holding and reordering information in working memory at the same time. Short-term memory (Gwm) — working memory capacity

Processing Speed

PS

Measures the rate at which simple visual information is processed correctly. Speed is not quality of thought, but it acts as a limit on throughput: whatever is processed more slowly occupies working memory for longer.

  • Symbol Search — Rapid visual comparison and decision under time pressure. Processing speed (Gs) — perceptual speed
  • Digit–Symbol Coding — Speed in applying a learned mapping rule. Processing speed (Gs) — transcription speed
02

How the evaluation is built

Ten subtests in a fixed order, taking roughly 58 minutes in total including instructions.

No.SubtestDomainItemsTimeScoring
01Figural MatricesFR28untimedIRT (3PL)
02VocabularyVC30untimedIRT (3PL)
03Digit SpanWMadaptiveuntimedRaw score
04Mental RotationVS32untimedIRT (3PL)
05Number SeriesFR28untimedIRT (3PL)
06Verbal AnalogiesVC30untimedIRT (3PL)
07Symbol SearchPSadaptive2 min fixedRaw score
08Paper FoldingVS32untimedIRT (3PL)
09Letter–Number SequencingWMadaptiveuntimedRaw score
10Digit–Symbol CodingPSadaptive2 min fixedRaw score

The order follows two rules: subtests from the same domain never sit next to each other, so that fatigue does not fall disproportionately on one domain; and the evaluation opens with figural material, because a verbal task at the start unsettles people who are unsure of their language more than it does anyone else.

03

The scoring model

From an answer to an index score, in four steps.

  1. 01

    Ability estimation per subtest

    The six untimed subtests are scored through a three-parameter logistic item response model. Every item has parameters for difficulty, discrimination and guessing; the guessing parameter equals the reciprocal of the number of options. What is estimated is the expected value of the posterior distribution (EAP) over a quadrature grid of 81 nodes with a standard normal prior. This has two advantages over maximum-likelihood estimation: the estimate stays finite when every answer is right or every answer is wrong, and the spread of the posterior yields the standard error of measurement directly, instead of having to derive it from an assumed reliability.

  2. 02

    Raw-score transformation for spans and timed subtests

    For the two memory spans and the two timed subtests, difficulty is not a property of the individual item but of the sequence length or the time allowed. These four subtests are therefore brought onto the same metric through a declared linear transformation of the raw score. In Symbol Search, wrong answers are subtracted from right ones, because on a yes/no task fast guessing would otherwise be rewarded.

  3. 03

    Scaled scores

    Each ability estimate is converted into a scaled score with a mean of 10 and a standard deviation of 3, bounded to the range 1 to 19 — the metric the Wechsler instruments use.

  4. 04

    Index and overall scores

    An index is the sum of the scaled scores of its two subtests, standardised against the modelled distribution of that sum. The spread of that distribution follows from a declared correlation matrix; the reliability of the sum follows from Mosier's formula for composite scores. Index and overall scores have a mean of 100 and a standard deviation of 15. Every score is reported with its 95 per cent confidence interval, computed from the standard error of measurement.

Classification bands

RangeLabelShare of the population
130 and aboveVery superior2.2 %
120–129Superior6.7 %
110–119High average16.1 %
90–109Average50.0 %
80–89Low average16.1 %
70–79Below average6.7 %
below 70Well below average2.2 %
04

Where the norms come from

The point at which this instrument differs from a clinical test.

Assumed measurement precision per subtest

SubtestDomainReliabilityScoring
Figural MatricesFR0.84IRT, estimated as you go
VocabularyVC0.82IRT, estimated as you go
Digit SpanWM0.84Raw-score transformation
Mental RotationVS0.78IRT, estimated as you go
Number SeriesFR0.80IRT, estimated as you go
Verbal AnalogiesVC0.80IRT, estimated as you go
Symbol SearchPS0.80Raw-score transformation
Paper FoldingVS0.76IRT, estimated as you go
Letter–Number SequencingWM0.82Raw-score transformation
Digit–Symbol CodingPS0.82Raw-score transformation

For the IRT subtests the figure given is a guide value; the report instead shows the measurement precision actually achieved in your session. It depends on how well the item difficulties match your ability level, and is lower at the edges of the distribution than in the middle. Because index scores combine two subtests, their reliability is higher than that of the individual scores, and that of the overall score higher still.

05

Checking how soundly the test was taken

Four kinds of irregularity are recorded and reported in the results.

  • Rapid guessing

    Answers given in under two seconds on untimed items. An item cannot be taken in within that time; such answers are guesses, and they lower the result.

  • Interruptions

    Switching to another window during the session, with the duration recorded. On the timed subtests this distorts the result directly.

  • Omissions

    Items left unanswered. They are scored as wrong.

  • Response times

    The median across all untimed items, as an indication of how carefully the test was worked through.

None of these findings makes a result invalid by itself. They determine how confidently it may be read, and they are reported in a section of their own rather than disappearing into the small print.

06

Limits

In full, and without softening.

  • Not clinical assessment. Giftedness, intellectual disability, learning disorders, ADHD and dementia processes can neither be established nor ruled out with this.
  • No standardisation sample. See section 04. The absolute level of the scores is to be read with reservation.
  • No age or education norms. Clinical instruments norm by age group, because fluid abilities decline slowly from roughly the third decade of life while crystallised abilities keep rising into old age. This instrument uses a single reference norm and therefore understates older adults and tends to overstate younger ones.
  • Dependence on language. Vocabulary and verbal analogies rely on vocabulary in the test language and systematically disadvantage people who did not grow up speaking it.
  • Unsupervised administration. Whether aids were used, whether someone helped, or whether the person was concentrating cannot be established with certainty — only partly, through the soundness checks.
  • Practice effect. A repeat sitting usually comes out better, because the item types are familiar. That is not a gain in ability.
  • No measure of personality, creativity or practical judgement. What is measured is a selected part of cognitive ability, not a person's capability as a whole.