Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study
Functional electrical stimulation (FES) can be used to restore motor function to people with paralysis caused by spinal cord injuries (SCIs). However, chronically-paralyzed FES-stimulated muscles can fatigue quickly, which may decrease FES controller performance. In this work, we explored the feasib...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9875337/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849763199043239936 |
|---|---|
| author | Jessica Abreu Douglas C. Crowder Robert F. Kirsch |
| author_facet | Jessica Abreu Douglas C. Crowder Robert F. Kirsch |
| author_sort | Jessica Abreu |
| collection | DOAJ |
| description | Functional electrical stimulation (FES) can be used to restore motor function to people with paralysis caused by spinal cord injuries (SCIs). However, chronically-paralyzed FES-stimulated muscles can fatigue quickly, which may decrease FES controller performance. In this work, we explored the feasibility of using deep neural network (DNN) controllers trained with reinforcement learning (RL) to control FES of upper-limb muscles after SCI. We developed upper-limb biomechanical models that exhibited increased muscle fatigability, decreased muscle recovery, and decreased muscle strength, as observed in people with chronic SCIs. Simulations confirmed that controller training time and controller performance are impaired to varying degrees by muscle fatigability. Also, the simulations showed that large muscle strength asymmetries between opposing muscles can substantially impair controller performance. However, the results of this study suggest that controller performance for highly-fatigable musculoskeletal systems can be preserved by allowing for rest between movements. Overall, the results suggest that RL can be used to successfully train FES controllers, even for highly-fatigable musculoskeletal systems. |
| format | Article |
| id | doaj-art-7960bcae7a1d4509a0fdf330e43a1d7a |
| institution | DOAJ |
| issn | 1534-4320 1558-0210 |
| language | English |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Neural Systems and Rehabilitation Engineering |
| spelling | doaj-art-7960bcae7a1d4509a0fdf330e43a1d7a2025-08-20T03:05:29ZengIEEEIEEE Transactions on Neural Systems and Rehabilitation Engineering1534-43201558-02102022-01-01302613262210.1109/TNSRE.2022.32039709875337Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility StudyJessica Abreu0https://orcid.org/0000-0002-5743-4489Douglas C. Crowder1https://orcid.org/0000-0002-4435-1799Robert F. Kirsch2https://orcid.org/0000-0003-2564-1800Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USACognitive and Emerging Computing Department, Sandia National Laboratories, Albuquerque, NM, USADepartment of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, USAFunctional electrical stimulation (FES) can be used to restore motor function to people with paralysis caused by spinal cord injuries (SCIs). However, chronically-paralyzed FES-stimulated muscles can fatigue quickly, which may decrease FES controller performance. In this work, we explored the feasibility of using deep neural network (DNN) controllers trained with reinforcement learning (RL) to control FES of upper-limb muscles after SCI. We developed upper-limb biomechanical models that exhibited increased muscle fatigability, decreased muscle recovery, and decreased muscle strength, as observed in people with chronic SCIs. Simulations confirmed that controller training time and controller performance are impaired to varying degrees by muscle fatigability. Also, the simulations showed that large muscle strength asymmetries between opposing muscles can substantially impair controller performance. However, the results of this study suggest that controller performance for highly-fatigable musculoskeletal systems can be preserved by allowing for rest between movements. Overall, the results suggest that RL can be used to successfully train FES controllers, even for highly-fatigable musculoskeletal systems.https://ieeexplore.ieee.org/document/9875337/Reinforcement learningfunctional electrical stimulationmotor controlspinal cord injurybiomechanical model |
| spellingShingle | Jessica Abreu Douglas C. Crowder Robert F. Kirsch Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study IEEE Transactions on Neural Systems and Rehabilitation Engineering Reinforcement learning functional electrical stimulation motor control spinal cord injury biomechanical model |
| title | Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study |
| title_full | Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study |
| title_fullStr | Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study |
| title_full_unstemmed | Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study |
| title_short | Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study |
| title_sort | deep reinforcement learning for control of time varying musculoskeletal systems with high fatigability a feasibility study |
| topic | Reinforcement learning functional electrical stimulation motor control spinal cord injury biomechanical model |
| url | https://ieeexplore.ieee.org/document/9875337/ |
| work_keys_str_mv | AT jessicaabreu deepreinforcementlearningforcontroloftimevaryingmusculoskeletalsystemswithhighfatigabilityafeasibilitystudy AT douglasccrowder deepreinforcementlearningforcontroloftimevaryingmusculoskeletalsystemswithhighfatigabilityafeasibilitystudy AT robertfkirsch deepreinforcementlearningforcontroloftimevaryingmusculoskeletalsystemswithhighfatigabilityafeasibilitystudy |