Physics-guided actor-critic reinforcement learning for swimming in turbulence
Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning st...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
American Physical Society
2025-01-01
|
| Series: | Physical Review Research |
| Online Access: | http://doi.org/10.1103/PhysRevResearch.7.013121 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832575978185949184 |
|---|---|
| author | Christopher Koh Laurent Pagnier Michael Chertkov |
| author_facet | Christopher Koh Laurent Pagnier Michael Chertkov |
| author_sort | Christopher Koh |
| collection | DOAJ |
| description | Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning strategy and comparing it with prescribed control and physics-agnostic reinforcement learning strategies. Our scheme, coined the actor-physicist, is an adaptation of the actor-critic algorithm in which the neural network parameterized critic is replaced with an analytically derived physical heuristic function, the physicist. We validate the proposed physics-informed reinforcement learning approach through extensive numerical experiments in both synthetic Batchelor-Kraichnan and more realistic Arnold-Beltrami-Childress flow environments, demonstrating its superiority in controlling particle dynamics when compared to standard reinforcement learning methods. |
| format | Article |
| id | doaj-art-15d03b44df924f7dad647ab53184d7d9 |
| institution | Kabale University |
| issn | 2643-1564 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | American Physical Society |
| record_format | Article |
| series | Physical Review Research |
| spelling | doaj-art-15d03b44df924f7dad647ab53184d7d92025-01-31T15:22:04ZengAmerican Physical SocietyPhysical Review Research2643-15642025-01-017101312110.1103/PhysRevResearch.7.013121Physics-guided actor-critic reinforcement learning for swimming in turbulenceChristopher KohLaurent PagnierMichael ChertkovTurbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning strategy and comparing it with prescribed control and physics-agnostic reinforcement learning strategies. Our scheme, coined the actor-physicist, is an adaptation of the actor-critic algorithm in which the neural network parameterized critic is replaced with an analytically derived physical heuristic function, the physicist. We validate the proposed physics-informed reinforcement learning approach through extensive numerical experiments in both synthetic Batchelor-Kraichnan and more realistic Arnold-Beltrami-Childress flow environments, demonstrating its superiority in controlling particle dynamics when compared to standard reinforcement learning methods.http://doi.org/10.1103/PhysRevResearch.7.013121 |
| spellingShingle | Christopher Koh Laurent Pagnier Michael Chertkov Physics-guided actor-critic reinforcement learning for swimming in turbulence Physical Review Research |
| title | Physics-guided actor-critic reinforcement learning for swimming in turbulence |
| title_full | Physics-guided actor-critic reinforcement learning for swimming in turbulence |
| title_fullStr | Physics-guided actor-critic reinforcement learning for swimming in turbulence |
| title_full_unstemmed | Physics-guided actor-critic reinforcement learning for swimming in turbulence |
| title_short | Physics-guided actor-critic reinforcement learning for swimming in turbulence |
| title_sort | physics guided actor critic reinforcement learning for swimming in turbulence |
| url | http://doi.org/10.1103/PhysRevResearch.7.013121 |
| work_keys_str_mv | AT christopherkoh physicsguidedactorcriticreinforcementlearningforswimminginturbulence AT laurentpagnier physicsguidedactorcriticreinforcementlearningforswimminginturbulence AT michaelchertkov physicsguidedactorcriticreinforcementlearningforswimminginturbulence |