How measurement noise limits the accuracy of brain-behaviour predictions
Abstract Major efforts in human neuroimaging strive to understand individual differences and find biomarkers for clinical applications by predicting behavioural phenotypes from brain imaging data. To identify generalisable and replicable brain-behaviour prediction models, sufficient measurement reli...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-12-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-024-54022-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850119002715586560 |
|---|---|
| author | Martin Gell Simon B. Eickhoff Amir Omidvarnia Vincent Küppers Kaustubh R. Patil Theodore D. Satterthwaite Veronika I. Müller Robert Langner |
| author_facet | Martin Gell Simon B. Eickhoff Amir Omidvarnia Vincent Küppers Kaustubh R. Patil Theodore D. Satterthwaite Veronika I. Müller Robert Langner |
| author_sort | Martin Gell |
| collection | DOAJ |
| description | Abstract Major efforts in human neuroimaging strive to understand individual differences and find biomarkers for clinical applications by predicting behavioural phenotypes from brain imaging data. To identify generalisable and replicable brain-behaviour prediction models, sufficient measurement reliability is essential. However, the selection of prediction targets is predominantly guided by scientific interest or data availability rather than psychometric considerations. Here, we demonstrate the impact of low reliability in behavioural phenotypes on out-of-sample prediction performance. Using simulated and empirical data from four large-scale datasets, we find that reliability levels common across many phenotypes can markedly limit the ability to link brain and behaviour. Next, using 5000 participants from the UK Biobank, we show that only highly reliable data can fully benefit from increasing sample sizes from hundreds to thousands of participants. Our findings highlight the importance of measurement reliability for identifying meaningful brain–behaviour associations from individual differences and underscore the need for greater emphasis on psychometrics in future research. |
| format | Article |
| id | doaj-art-5a2f9505de814dae81bea403199aac78 |
| institution | OA Journals |
| issn | 2041-1723 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-5a2f9505de814dae81bea403199aac782025-08-20T02:35:44ZengNature PortfolioNature Communications2041-17232024-12-0115111210.1038/s41467-024-54022-6How measurement noise limits the accuracy of brain-behaviour predictionsMartin Gell0Simon B. Eickhoff1Amir Omidvarnia2Vincent Küppers3Kaustubh R. Patil4Theodore D. Satterthwaite5Veronika I. Müller6Robert Langner7Department of Psychiatry, Psychotherapy and Psychosomatics, Medical Faculty, RWTH Aachen UniversityInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichDepartment of Psychiatry, Perelman School of Medicine, Penn Lifespan Informatics and Neuroimaging Center, University of PennsylvaniaInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichInstitute of Neuroscience and Medicine (INM-7: Brain & Behaviour), Research Centre JülichAbstract Major efforts in human neuroimaging strive to understand individual differences and find biomarkers for clinical applications by predicting behavioural phenotypes from brain imaging data. To identify generalisable and replicable brain-behaviour prediction models, sufficient measurement reliability is essential. However, the selection of prediction targets is predominantly guided by scientific interest or data availability rather than psychometric considerations. Here, we demonstrate the impact of low reliability in behavioural phenotypes on out-of-sample prediction performance. Using simulated and empirical data from four large-scale datasets, we find that reliability levels common across many phenotypes can markedly limit the ability to link brain and behaviour. Next, using 5000 participants from the UK Biobank, we show that only highly reliable data can fully benefit from increasing sample sizes from hundreds to thousands of participants. Our findings highlight the importance of measurement reliability for identifying meaningful brain–behaviour associations from individual differences and underscore the need for greater emphasis on psychometrics in future research.https://doi.org/10.1038/s41467-024-54022-6 |
| spellingShingle | Martin Gell Simon B. Eickhoff Amir Omidvarnia Vincent Küppers Kaustubh R. Patil Theodore D. Satterthwaite Veronika I. Müller Robert Langner How measurement noise limits the accuracy of brain-behaviour predictions Nature Communications |
| title | How measurement noise limits the accuracy of brain-behaviour predictions |
| title_full | How measurement noise limits the accuracy of brain-behaviour predictions |
| title_fullStr | How measurement noise limits the accuracy of brain-behaviour predictions |
| title_full_unstemmed | How measurement noise limits the accuracy of brain-behaviour predictions |
| title_short | How measurement noise limits the accuracy of brain-behaviour predictions |
| title_sort | how measurement noise limits the accuracy of brain behaviour predictions |
| url | https://doi.org/10.1038/s41467-024-54022-6 |
| work_keys_str_mv | AT martingell howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT simonbeickhoff howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT amiromidvarnia howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT vincentkuppers howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT kaustubhrpatil howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT theodoredsatterthwaite howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT veronikaimuller howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions AT robertlangner howmeasurementnoiselimitstheaccuracyofbrainbehaviourpredictions |