Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning
Abstract The increasing scale of unconventional oil and gas resource development has driven the demand for complex fracturing technologies. Due to their high value and energy consumption, the operational efficiency of fracturing units directly impacts the overall fracturing production efficiency. Cu...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2025-06-01
|
| Series: | Journal of Petroleum Exploration and Production Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s13202-025-02014-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849333697152548864 |
|---|---|
| author | RuYi Wang HuiShen Jiao YingCheng Tian Yi Zhao SiQi Wang Ke Zhang Bo Huang QinRui Sun DanDan Zhu |
| author_facet | RuYi Wang HuiShen Jiao YingCheng Tian Yi Zhao SiQi Wang Ke Zhang Bo Huang QinRui Sun DanDan Zhu |
| author_sort | RuYi Wang |
| collection | DOAJ |
| description | Abstract The increasing scale of unconventional oil and gas resource development has driven the demand for complex fracturing technologies. Due to their high value and energy consumption, the operational efficiency of fracturing units directly impacts the overall fracturing production efficiency. Currently, the control of fracturing units mainly relies on manual experience, where experts allocate displacement for each fracturing truck based on total displacement requirements, which is then directly converted into pump speeds according to pre-set rules. However, this manual decision-making pattern often exhibits insufficient collaborative capability when confronted with complex construction scenarios. Furthermore, due to the lack of autonomous decision-making ability in concrete fracturing trucks, timely responses to unexpected failures are challenging. To address these issues, we proposes a multi-agent reinforcement learning based pump injection coordination control method to autonomously compute the optimal pump speeds for each fracturing truck in real time. The core of this method is an improved algorithm based on the TD3, which is enhanced by the incorporation of the CQL algorithm to improve the stability of the collaborative control strategy. A standardized pump injection control sequence dataset was constructed to verify the proposed method. And the experimental results illustrates that the average deviation between the calculated optimal pump speeds from the proposed method and the historical data is less than 8%, indicating that the proposed method can effectively mine the historical pump data and achieve expert-level control capability. Furthermore, the proposed method has been experimentally compared with both classical and cutting-edge models of machine learning and reinforcement learning, resulting in 37.5% to 48.6% reductions in pump speed deviation. It evidences the superiority of the proposed method within the realm of collaborative sequential decision making, showcasing its capability to achieve autonomous decision-making and coordinated control among multiple fracturing trucks. However, due to the limited data, the agents may have limited generalizability to different fracturing truck fleet configurations. Additionally, while the method shows promise in specific contexts, further online learning will be necessary for real-world deployment to adapt the agents to site-specific conditions. |
| format | Article |
| id | doaj-art-76559a07cb844943a4bb613ed76d184e |
| institution | Kabale University |
| issn | 2190-0558 2190-0566 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Petroleum Exploration and Production Technology |
| spelling | doaj-art-76559a07cb844943a4bb613ed76d184e2025-08-20T03:45:47ZengSpringerOpenJournal of Petroleum Exploration and Production Technology2190-05582190-05662025-06-0115711510.1007/s13202-025-02014-7Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learningRuYi Wang0HuiShen Jiao1YingCheng Tian2Yi Zhao3SiQi Wang4Ke Zhang5Bo Huang6QinRui Sun7DanDan Zhu8Downhole Operation Research Institute, CNPC Engineering Technology R&D Company LimitedCollege of Artificial Intelligence, China University of Petroleum-BeijingDownhole Operation Research Institute, CNPC Engineering Technology R&D Company LimitedCollege of Artificial Intelligence, China University of Petroleum-BeijingDownhole Operation Research Institute, CNPC Engineering Technology R&D Company LimitedCollege of Artificial Intelligence, China University of Petroleum-BeijingDownhole Operation Research Institute, CNPC Engineering Technology R&D Company LimitedGWDC Engineering Technology Research InstituteCollege of Artificial Intelligence, China University of Petroleum-BeijingAbstract The increasing scale of unconventional oil and gas resource development has driven the demand for complex fracturing technologies. Due to their high value and energy consumption, the operational efficiency of fracturing units directly impacts the overall fracturing production efficiency. Currently, the control of fracturing units mainly relies on manual experience, where experts allocate displacement for each fracturing truck based on total displacement requirements, which is then directly converted into pump speeds according to pre-set rules. However, this manual decision-making pattern often exhibits insufficient collaborative capability when confronted with complex construction scenarios. Furthermore, due to the lack of autonomous decision-making ability in concrete fracturing trucks, timely responses to unexpected failures are challenging. To address these issues, we proposes a multi-agent reinforcement learning based pump injection coordination control method to autonomously compute the optimal pump speeds for each fracturing truck in real time. The core of this method is an improved algorithm based on the TD3, which is enhanced by the incorporation of the CQL algorithm to improve the stability of the collaborative control strategy. A standardized pump injection control sequence dataset was constructed to verify the proposed method. And the experimental results illustrates that the average deviation between the calculated optimal pump speeds from the proposed method and the historical data is less than 8%, indicating that the proposed method can effectively mine the historical pump data and achieve expert-level control capability. Furthermore, the proposed method has been experimentally compared with both classical and cutting-edge models of machine learning and reinforcement learning, resulting in 37.5% to 48.6% reductions in pump speed deviation. It evidences the superiority of the proposed method within the realm of collaborative sequential decision making, showcasing its capability to achieve autonomous decision-making and coordinated control among multiple fracturing trucks. However, due to the limited data, the agents may have limited generalizability to different fracturing truck fleet configurations. Additionally, while the method shows promise in specific contexts, further online learning will be necessary for real-world deployment to adapt the agents to site-specific conditions.https://doi.org/10.1007/s13202-025-02014-7Offline reinforcement learningHydraulic fracturingMulti-agent reinforcement learningMachine learning |
| spellingShingle | RuYi Wang HuiShen Jiao YingCheng Tian Yi Zhao SiQi Wang Ke Zhang Bo Huang QinRui Sun DanDan Zhu Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning Journal of Petroleum Exploration and Production Technology Offline reinforcement learning Hydraulic fracturing Multi-agent reinforcement learning Machine learning |
| title | Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning |
| title_full | Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning |
| title_fullStr | Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning |
| title_full_unstemmed | Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning |
| title_short | Cooperative control method for multi-agent ground fracturing truck group based on offline reinforcement learning |
| title_sort | cooperative control method for multi agent ground fracturing truck group based on offline reinforcement learning |
| topic | Offline reinforcement learning Hydraulic fracturing Multi-agent reinforcement learning Machine learning |
| url | https://doi.org/10.1007/s13202-025-02014-7 |
| work_keys_str_mv | AT ruyiwang cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT huishenjiao cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT yingchengtian cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT yizhao cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT siqiwang cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT kezhang cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT bohuang cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT qinruisun cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning AT dandanzhu cooperativecontrolmethodformultiagentgroundfracturingtruckgroupbasedonofflinereinforcementlearning |