Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study
BackgroundFloodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study populati...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-04-01
|
| Series: | Journal of Medical Internet Research |
| Online Access: | https://www.jmir.org/2025/1/e63090 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850273184430948352 |
|---|---|
| author | Lito Kriara Frank Dondelinger Luca Capezzuto Corrado Bernasconi Florian Lipsmeier Adriano Galati Michael Lindemann |
| author_facet | Lito Kriara Frank Dondelinger Luca Capezzuto Corrado Bernasconi Florian Lipsmeier Adriano Galati Michael Lindemann |
| author_sort | Lito Kriara |
| collection | DOAJ |
| description |
BackgroundFloodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population.
ObjectiveThis study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models.
MethodsFloodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a “bring-your-own-device” (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor–based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status.
ResultsOverall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03).
ConclusionsConsistent with the hypothesis that smartphone sensor–based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed. |
| format | Article |
| id | doaj-art-450f2db4968d4bc29e6ed63426984d0c |
| institution | OA Journals |
| issn | 1438-8871 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | Journal of Medical Internet Research |
| spelling | doaj-art-450f2db4968d4bc29e6ed63426984d0c2025-08-20T01:51:35ZengJMIR PublicationsJournal of Medical Internet Research1438-88712025-04-0127e6309010.2196/63090Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device StudyLito Kriarahttps://orcid.org/0000-0002-9159-0469Frank Dondelingerhttps://orcid.org/0000-0003-1816-6300Luca Capezzutohttps://orcid.org/0000-0002-2186-2406Corrado Bernasconihttps://orcid.org/0000-0002-1019-5468Florian Lipsmeierhttps://orcid.org/0000-0002-8663-819XAdriano Galatihttps://orcid.org/0009-0006-9661-899XMichael Lindemannhttps://orcid.org/0000-0003-1010-3890 BackgroundFloodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population. ObjectiveThis study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models. MethodsFloodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a “bring-your-own-device” (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor–based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status. ResultsOverall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03). ConclusionsConsistent with the hypothesis that smartphone sensor–based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.https://www.jmir.org/2025/1/e63090 |
| spellingShingle | Lito Kriara Frank Dondelinger Luca Capezzuto Corrado Bernasconi Florian Lipsmeier Adriano Galati Michael Lindemann Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study Journal of Medical Internet Research |
| title | Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study |
| title_full | Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study |
| title_fullStr | Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study |
| title_full_unstemmed | Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study |
| title_short | Investigating Measurement Equivalence of Smartphone Sensor–Based Assessments: Remote, Digital, Bring-Your-Own-Device Study |
| title_sort | investigating measurement equivalence of smartphone sensor based assessments remote digital bring your own device study |
| url | https://www.jmir.org/2025/1/e63090 |
| work_keys_str_mv | AT litokriara investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT frankdondelinger investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT lucacapezzuto investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT corradobernasconi investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT florianlipsmeier investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT adrianogalati investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy AT michaellindemann investigatingmeasurementequivalenceofsmartphonesensorbasedassessmentsremotedigitalbringyourowndevicestudy |