Modeling Collective Behavior for Fish School With Deep Q-Networks
Modeling collective behavior is a way to better understand the mechanisms that govern collective animal behaviors. Traditional rule-based modeling methods rely heavily on human prior knowledge and may not provide a proper explanation of the phenomenon of collective behaviors. This paper proposes a D...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10087222/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849334749377593344 |
|---|---|
| author | Pengyu Chen Fang Wang Shuo Liu Yifan Yu Shengzhi Yue Yanan Song Yuanshan Lin |
| author_facet | Pengyu Chen Fang Wang Shuo Liu Yifan Yu Shengzhi Yue Yanan Song Yuanshan Lin |
| author_sort | Pengyu Chen |
| collection | DOAJ |
| description | Modeling collective behavior is a way to better understand the mechanisms that govern collective animal behaviors. Traditional rule-based modeling methods rely heavily on human prior knowledge and may not provide a proper explanation of the phenomenon of collective behaviors. This paper proposes a Deep Q-Networks (DQN)-based modeling method for fish school. Firstly, an individual’s state (continuous value) is expressed by the angle between its direction and the average direction of its perceived neighbors. An individual’s action is represented with discretized turning angle. Then, the reward function is constructed with the change in the number of neighbors. And finally, the neural network structure is constructed to represent the Q-value function and is trained by the DQN algorithm. The proposed approach is tested in two scenarios: single-learner and multi-learner. Results show that in both scenarios the proposed method can gradually converge and finally obtain a model that can produce collective behavior. On this basis, this paper also deeply analyzes the learned model from the perspectives of average order parameter and collective behavior patterns. It verifies that the behavior pattern generated by the learned model is a highly ordered collective behavior. In addition, we make a comparison between our proposed approach and the Q-Learning algorithm. The results show that our approach not only has a stronger ability to express policy and is better at handling continuous states but also has a more stable learning performance in training. |
| format | Article |
| id | doaj-art-402aff8758434d8dbab8444281c3c3bb |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-402aff8758434d8dbab8444281c3c3bb2025-08-20T03:45:28ZengIEEEIEEE Access2169-35362023-01-0111366303664110.1109/ACCESS.2023.326323710087222Modeling Collective Behavior for Fish School With Deep Q-NetworksPengyu Chen0https://orcid.org/0000-0002-9323-8936Fang Wang1Shuo Liu2Yifan Yu3Shengzhi Yue4Yanan Song5Yuanshan Lin6https://orcid.org/0000-0001-9053-8004College of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaCollege of Information Engineering, Dalian Ocean University, Dalian, ChinaModeling collective behavior is a way to better understand the mechanisms that govern collective animal behaviors. Traditional rule-based modeling methods rely heavily on human prior knowledge and may not provide a proper explanation of the phenomenon of collective behaviors. This paper proposes a Deep Q-Networks (DQN)-based modeling method for fish school. Firstly, an individual’s state (continuous value) is expressed by the angle between its direction and the average direction of its perceived neighbors. An individual’s action is represented with discretized turning angle. Then, the reward function is constructed with the change in the number of neighbors. And finally, the neural network structure is constructed to represent the Q-value function and is trained by the DQN algorithm. The proposed approach is tested in two scenarios: single-learner and multi-learner. Results show that in both scenarios the proposed method can gradually converge and finally obtain a model that can produce collective behavior. On this basis, this paper also deeply analyzes the learned model from the perspectives of average order parameter and collective behavior patterns. It verifies that the behavior pattern generated by the learned model is a highly ordered collective behavior. In addition, we make a comparison between our proposed approach and the Q-Learning algorithm. The results show that our approach not only has a stronger ability to express policy and is better at handling continuous states but also has a more stable learning performance in training.https://ieeexplore.ieee.org/document/10087222/Collective behaviorcollective behavior modelDeep Q-Networks (DQN)fish school |
| spellingShingle | Pengyu Chen Fang Wang Shuo Liu Yifan Yu Shengzhi Yue Yanan Song Yuanshan Lin Modeling Collective Behavior for Fish School With Deep Q-Networks IEEE Access Collective behavior collective behavior model Deep Q-Networks (DQN) fish school |
| title | Modeling Collective Behavior for Fish School With Deep Q-Networks |
| title_full | Modeling Collective Behavior for Fish School With Deep Q-Networks |
| title_fullStr | Modeling Collective Behavior for Fish School With Deep Q-Networks |
| title_full_unstemmed | Modeling Collective Behavior for Fish School With Deep Q-Networks |
| title_short | Modeling Collective Behavior for Fish School With Deep Q-Networks |
| title_sort | modeling collective behavior for fish school with deep q networks |
| topic | Collective behavior collective behavior model Deep Q-Networks (DQN) fish school |
| url | https://ieeexplore.ieee.org/document/10087222/ |
| work_keys_str_mv | AT pengyuchen modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT fangwang modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT shuoliu modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT yifanyu modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT shengzhiyue modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT yanansong modelingcollectivebehaviorforfishschoolwithdeepqnetworks AT yuanshanlin modelingcollectivebehaviorforfishschoolwithdeepqnetworks |