A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process
Unpredictable sudden disturbances such as machine failure, processing time lag, and order changes increase the deviation between actual production and the planned schedule, seriously affecting production efficiency. This phenomenon is particularly severe in flexible manufacturing. In this paper, a d...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/1/4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549149297704960 |
|---|---|
| author | Jun Yan Tianzuo Zhao Tao Zhang Hongyan Chu Congbin Yang Yueze Zhang |
| author_facet | Jun Yan Tianzuo Zhao Tao Zhang Hongyan Chu Congbin Yang Yueze Zhang |
| author_sort | Jun Yan |
| collection | DOAJ |
| description | Unpredictable sudden disturbances such as machine failure, processing time lag, and order changes increase the deviation between actual production and the planned schedule, seriously affecting production efficiency. This phenomenon is particularly severe in flexible manufacturing. In this paper, a dynamic scheduling method combining iterative optimization and deep reinforcement learning (DRL) is proposed to address the impact of uncertain disturbances. A real-time DRL production environment model is established for the flexible job scheduling problem. Based on the DRL model, an agent training strategy and an autonomous decision-making method are proposed. An event-driven and period-driven hybrid dynamic rescheduling trigger strategy (HDRS) with four judgment mechanisms has been developed. The decision-making method and rescheduling trigger strategy solve the problem of how and when to reschedule for the dynamic scheduling problem. The data experiment results show that the trained DRL decision-making model can provide timely feedback on the adjusted scheduling arrangements for different-scale order problems. The proposed dynamic-scheduling decision-making method and rescheduling trigger strategy can achieve high responsiveness, quick feedback, high quality, and high stability for flexible manufacturing process scheduling decision making under sudden disturbance. |
| format | Article |
| id | doaj-art-7469d38f79434e0eb6319fb3aa62d1af |
| institution | Kabale University |
| issn | 2227-7390 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-7469d38f79434e0eb6319fb3aa62d1af2025-01-10T13:17:56ZengMDPI AGMathematics2227-73902024-12-01131410.3390/math13010004A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing ProcessJun Yan0Tianzuo Zhao1Tao Zhang2Hongyan Chu3Congbin Yang4Yueze Zhang5Mechanical Industry Key Laboratory of Heavy Machine Tool Digital Design and Testing, Beijing University of Technology, Beijing 100124, ChinaMechanical Industry Key Laboratory of Heavy Machine Tool Digital Design and Testing, Beijing University of Technology, Beijing 100124, ChinaBeijing Key Laboratory of Advanced Manufacturing Technology, Beijing University of Technology, Beijing 100124, ChinaMechanical Industry Key Laboratory of Heavy Machine Tool Digital Design and Testing, Beijing University of Technology, Beijing 100124, ChinaMechanical Industry Key Laboratory of Heavy Machine Tool Digital Design and Testing, Beijing University of Technology, Beijing 100124, ChinaMechanical Industry Key Laboratory of Heavy Machine Tool Digital Design and Testing, Beijing University of Technology, Beijing 100124, ChinaUnpredictable sudden disturbances such as machine failure, processing time lag, and order changes increase the deviation between actual production and the planned schedule, seriously affecting production efficiency. This phenomenon is particularly severe in flexible manufacturing. In this paper, a dynamic scheduling method combining iterative optimization and deep reinforcement learning (DRL) is proposed to address the impact of uncertain disturbances. A real-time DRL production environment model is established for the flexible job scheduling problem. Based on the DRL model, an agent training strategy and an autonomous decision-making method are proposed. An event-driven and period-driven hybrid dynamic rescheduling trigger strategy (HDRS) with four judgment mechanisms has been developed. The decision-making method and rescheduling trigger strategy solve the problem of how and when to reschedule for the dynamic scheduling problem. The data experiment results show that the trained DRL decision-making model can provide timely feedback on the adjusted scheduling arrangements for different-scale order problems. The proposed dynamic-scheduling decision-making method and rescheduling trigger strategy can achieve high responsiveness, quick feedback, high quality, and high stability for flexible manufacturing process scheduling decision making under sudden disturbance.https://www.mdpi.com/2227-7390/13/1/4dynamic schedulingdeep reinforcement learningflexible job shopdouble deep Q-networkrescheduling |
| spellingShingle | Jun Yan Tianzuo Zhao Tao Zhang Hongyan Chu Congbin Yang Yueze Zhang A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process Mathematics dynamic scheduling deep reinforcement learning flexible job shop double deep Q-network rescheduling |
| title | A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process |
| title_full | A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process |
| title_fullStr | A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process |
| title_full_unstemmed | A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process |
| title_short | A Dynamic Scheduling Method Combining Iterative Optimization and Deep Reinforcement Learning to Solve Sudden Disturbance Events in a Flexible Manufacturing Process |
| title_sort | dynamic scheduling method combining iterative optimization and deep reinforcement learning to solve sudden disturbance events in a flexible manufacturing process |
| topic | dynamic scheduling deep reinforcement learning flexible job shop double deep Q-network rescheduling |
| url | https://www.mdpi.com/2227-7390/13/1/4 |
| work_keys_str_mv | AT junyan adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT tianzuozhao adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT taozhang adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT hongyanchu adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT congbinyang adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT yuezezhang adynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT junyan dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT tianzuozhao dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT taozhang dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT hongyanchu dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT congbinyang dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess AT yuezezhang dynamicschedulingmethodcombiningiterativeoptimizationanddeepreinforcementlearningtosolvesuddendisturbanceeventsinaflexiblemanufacturingprocess |