Adaptive measurement of cognitive function based on multidimensional item response theory
Abstract INTRODUCTION Up to 20% of older adults in the United States have mild cognitive impairment (MCI), and about one‐third of people with MCI are predicted to transition to Alzheimer's disease (AD) within 5 years. Standard cognitive assessments are long and require a trained technician to a...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-10-01
|
| Series: | Alzheimer’s & Dementia: Translational Research & Clinical Interventions |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/trc2.70018 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850239968943800320 |
|---|---|
| author | Robert D. Gibbons Diane S. Lauderdale Robert S. Wilson David A. Bennett Tesnim Arar David A. Gallo |
| author_facet | Robert D. Gibbons Diane S. Lauderdale Robert S. Wilson David A. Bennett Tesnim Arar David A. Gallo |
| author_sort | Robert D. Gibbons |
| collection | DOAJ |
| description | Abstract INTRODUCTION Up to 20% of older adults in the United States have mild cognitive impairment (MCI), and about one‐third of people with MCI are predicted to transition to Alzheimer's disease (AD) within 5 years. Standard cognitive assessments are long and require a trained technician to administer. We developed the first computerized adaptive test (CAT) based on multidimensional item response theory (MIRT) to more precisely, rapidly, and repeatedly assesses cognitive abilities across the adult lifespan. We present results for a prototype CAT (pCAT‐COG) for assessment of global cognitive function. METHODS We sampled items across five cognitive domains central to neuropsychological testing (episodic memory [EM], semantic memory/language [SM], working memory [WM], executive function/flexible thinking, and processing speed [PS]). The item bank consists of 54 items, with 9 items of varying difficulty drawn from six different cognitive tasks. Each of the 54 items has 3 response trials, yielding an ordinal score (0–3 trials correct). We also include three long‐term memory items not designed for adaptive administration, for a total bank of 57 items. Calibration data were collected in‐person and online, calibrated using a bifactor MIRT model, and pCAT‐COG scores validated against a technician‐administered neuropsychological battery. RESULTS The bifactor MIRT model improved fit over a unidimensional IRT model (p < 0.0001). The global pCAT‐COG scores were inversely correlated with age (r = –0.44, p < 0.0001). Simulated adaptive administration of 11 items maintained a correlation of r = 0.94 with the total item bank scores. Significant differences between mild and no cognitive impairment (NCI) were found (effect size of 1.08 SD units). The pCAT‐COG correlated with clinician‐based global measure (r = 0.64). DISCUSSION MIRT‐based CAT is feasible and valid for the assessment of global cognitive impairment, laying the foundation for the development of a full CAT‐COG that will draw from a much larger item bank with both global and domain specific measures of cognitive impairment. Highlights As Americans age, numbers at risk for developing cognitive impairment are increasing. Aging‐related declines in cognition begins decades prior to the onset of obvious cognitive impairment. Traditional assessment is burdensome and requires trained clinicians. We developed an adaptive testing framework using multidimensional item response theory. It is comparable to lengthier in‐person assessments that require trained psychometrists. |
| format | Article |
| id | doaj-art-e576b6788eea4fcdab248cb149fb2619 |
| institution | OA Journals |
| issn | 2352-8737 |
| language | English |
| publishDate | 2024-10-01 |
| publisher | Wiley |
| record_format | Article |
| series | Alzheimer’s & Dementia: Translational Research & Clinical Interventions |
| spelling | doaj-art-e576b6788eea4fcdab248cb149fb26192025-08-20T02:01:00ZengWileyAlzheimer’s & Dementia: Translational Research & Clinical Interventions2352-87372024-10-01104n/an/a10.1002/trc2.70018Adaptive measurement of cognitive function based on multidimensional item response theoryRobert D. Gibbons0Diane S. Lauderdale1Robert S. Wilson2David A. Bennett3Tesnim Arar4David A. Gallo5Departments of Medicine and Public Health Sciences and Center for Health Statistics University of Chicago Chicago Illinois USADepartment of Public Health Sciences University of Chicago Chicago Illinois USADepartment of Neurological Sciences and Rush Alzheimer's Disease Center Rush University Medical Center Chicago Illinois USADepartment of Neurological Sciences and Rush Alzheimer's Disease Center Rush University Medical Center Chicago Illinois USADepartment of Psychology University of Chicago Chicago Illinois USADepartment of Psychology University of Chicago Chicago Illinois USAAbstract INTRODUCTION Up to 20% of older adults in the United States have mild cognitive impairment (MCI), and about one‐third of people with MCI are predicted to transition to Alzheimer's disease (AD) within 5 years. Standard cognitive assessments are long and require a trained technician to administer. We developed the first computerized adaptive test (CAT) based on multidimensional item response theory (MIRT) to more precisely, rapidly, and repeatedly assesses cognitive abilities across the adult lifespan. We present results for a prototype CAT (pCAT‐COG) for assessment of global cognitive function. METHODS We sampled items across five cognitive domains central to neuropsychological testing (episodic memory [EM], semantic memory/language [SM], working memory [WM], executive function/flexible thinking, and processing speed [PS]). The item bank consists of 54 items, with 9 items of varying difficulty drawn from six different cognitive tasks. Each of the 54 items has 3 response trials, yielding an ordinal score (0–3 trials correct). We also include three long‐term memory items not designed for adaptive administration, for a total bank of 57 items. Calibration data were collected in‐person and online, calibrated using a bifactor MIRT model, and pCAT‐COG scores validated against a technician‐administered neuropsychological battery. RESULTS The bifactor MIRT model improved fit over a unidimensional IRT model (p < 0.0001). The global pCAT‐COG scores were inversely correlated with age (r = –0.44, p < 0.0001). Simulated adaptive administration of 11 items maintained a correlation of r = 0.94 with the total item bank scores. Significant differences between mild and no cognitive impairment (NCI) were found (effect size of 1.08 SD units). The pCAT‐COG correlated with clinician‐based global measure (r = 0.64). DISCUSSION MIRT‐based CAT is feasible and valid for the assessment of global cognitive impairment, laying the foundation for the development of a full CAT‐COG that will draw from a much larger item bank with both global and domain specific measures of cognitive impairment. Highlights As Americans age, numbers at risk for developing cognitive impairment are increasing. Aging‐related declines in cognition begins decades prior to the onset of obvious cognitive impairment. Traditional assessment is burdensome and requires trained clinicians. We developed an adaptive testing framework using multidimensional item response theory. It is comparable to lengthier in‐person assessments that require trained psychometrists.https://doi.org/10.1002/trc2.70018Alzheimer's diseasebifactor modelcognitive impairmentcomputerized adaptive testingmultidimensional item response theoryneuropsychological assessment |
| spellingShingle | Robert D. Gibbons Diane S. Lauderdale Robert S. Wilson David A. Bennett Tesnim Arar David A. Gallo Adaptive measurement of cognitive function based on multidimensional item response theory Alzheimer’s & Dementia: Translational Research & Clinical Interventions Alzheimer's disease bifactor model cognitive impairment computerized adaptive testing multidimensional item response theory neuropsychological assessment |
| title | Adaptive measurement of cognitive function based on multidimensional item response theory |
| title_full | Adaptive measurement of cognitive function based on multidimensional item response theory |
| title_fullStr | Adaptive measurement of cognitive function based on multidimensional item response theory |
| title_full_unstemmed | Adaptive measurement of cognitive function based on multidimensional item response theory |
| title_short | Adaptive measurement of cognitive function based on multidimensional item response theory |
| title_sort | adaptive measurement of cognitive function based on multidimensional item response theory |
| topic | Alzheimer's disease bifactor model cognitive impairment computerized adaptive testing multidimensional item response theory neuropsychological assessment |
| url | https://doi.org/10.1002/trc2.70018 |
| work_keys_str_mv | AT robertdgibbons adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory AT dianeslauderdale adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory AT robertswilson adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory AT davidabennett adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory AT tesnimarar adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory AT davidagallo adaptivemeasurementofcognitivefunctionbasedonmultidimensionalitemresponsetheory |