Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum
As an increasing number of Distributed Machine Learning (DML) tasks are hosted on cloud platforms in the edge-cloud continuum, Data Centers (DCs) with massive data and computational requirements have become one of the world’s largest energy consumers, leading to significant carbon emissions. Reducin...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2024-01-01
|
| Series: | Science and Technology for Energy Transition |
| Subjects: | |
| Online Access: | https://www.stet-review.org/articles/stet/full_html/2024/01/stet20240197/stet20240197.html |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850062473483255808 |
|---|---|
| author | Miao Zicong Liu Lei Nan Haijing Li Weize Pan Xiaodong Yang Xin Yu Mi Chen Hui Zhao Yiming |
| author_facet | Miao Zicong Liu Lei Nan Haijing Li Weize Pan Xiaodong Yang Xin Yu Mi Chen Hui Zhao Yiming |
| author_sort | Miao Zicong |
| collection | DOAJ |
| description | As an increasing number of Distributed Machine Learning (DML) tasks are hosted on cloud platforms in the edge-cloud continuum, Data Centers (DCs) with massive data and computational requirements have become one of the world’s largest energy consumers, leading to significant carbon emissions. Reducing energy consumption and carbon emissions is an extremely crucial and challenging issue for the sustainable development of cloud service providers. While utilizing renewable energy can help reduce the carbon emissions of DCs, the intermittent and unstable nature still causes DCs to rely heavily on high-carbon brown energy. For the resource-intensive and delay-tolerant DML tasks, this paper introduces multi-renewable energy in the geo-distributed continuum to address this issue, the spatiotemporal complementarity maximizes the renewable energy utilization and compensates for time-dependent energy differences with geographic advantages. Additionally, considering the dynamic differences in carbon intensity and electricity prices across distributed DCs in the continuum, we propose an energy and carbon-aware algorithm called ECMR for scheduling heterogeneous virtual machine creation tasks of DML among multi-clouds in different time zones. It is demonstrated that compared with the baseline methods, the ECMR significantly reduces the total power consumption, energy cost, and carbon emission of data centers while maintaining an acceptable service quality. The utilization of renewable energy in data centers has been significantly improved to 90.8% by flexibly leveraging the spatiotemporal complementarity of multi-renewable energy. Compared with existing competing algorithms, the proposed method exhibits significant improvements with an achieved average response time of 12.6 ms, and a task failure rate of 1.25%. |
| format | Article |
| id | doaj-art-d81f5b03daa447d8b2c6dff307ea4de8 |
| institution | DOAJ |
| issn | 2804-7699 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | Science and Technology for Energy Transition |
| spelling | doaj-art-d81f5b03daa447d8b2c6dff307ea4de82025-08-20T02:49:54ZengEDP SciencesScience and Technology for Energy Transition2804-76992024-01-01798210.2516/stet/2024076stet20240197Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuumMiao Zicong0Liu Lei1Nan Haijing2Li Weize3Pan Xiaodong4Yang Xin5Yu Mi6Chen Hui7Zhao Yiming8Hybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationSchool of Energy Science and Engineering, Central South UniversityHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationHybrid Cloud Intelligent Innovation Laboratory, China Telecom Cloud Computing CorporationSchool of Business, Renmin University of ChinaAs an increasing number of Distributed Machine Learning (DML) tasks are hosted on cloud platforms in the edge-cloud continuum, Data Centers (DCs) with massive data and computational requirements have become one of the world’s largest energy consumers, leading to significant carbon emissions. Reducing energy consumption and carbon emissions is an extremely crucial and challenging issue for the sustainable development of cloud service providers. While utilizing renewable energy can help reduce the carbon emissions of DCs, the intermittent and unstable nature still causes DCs to rely heavily on high-carbon brown energy. For the resource-intensive and delay-tolerant DML tasks, this paper introduces multi-renewable energy in the geo-distributed continuum to address this issue, the spatiotemporal complementarity maximizes the renewable energy utilization and compensates for time-dependent energy differences with geographic advantages. Additionally, considering the dynamic differences in carbon intensity and electricity prices across distributed DCs in the continuum, we propose an energy and carbon-aware algorithm called ECMR for scheduling heterogeneous virtual machine creation tasks of DML among multi-clouds in different time zones. It is demonstrated that compared with the baseline methods, the ECMR significantly reduces the total power consumption, energy cost, and carbon emission of data centers while maintaining an acceptable service quality. The utilization of renewable energy in data centers has been significantly improved to 90.8% by flexibly leveraging the spatiotemporal complementarity of multi-renewable energy. Compared with existing competing algorithms, the proposed method exhibits significant improvements with an achieved average response time of 12.6 ms, and a task failure rate of 1.25%.https://www.stet-review.org/articles/stet/full_html/2024/01/stet20240197/stet20240197.htmledge-cloud continuumdml task schedulilngdistributed data centersmulti-renewable energyenergy consumptioncarbon emissionenergy complementarity |
| spellingShingle | Miao Zicong Liu Lei Nan Haijing Li Weize Pan Xiaodong Yang Xin Yu Mi Chen Hui Zhao Yiming Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum Science and Technology for Energy Transition edge-cloud continuum dml task schedulilng distributed data centers multi-renewable energy energy consumption carbon emission energy complementarity |
| title | Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum |
| title_full | Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum |
| title_fullStr | Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum |
| title_full_unstemmed | Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum |
| title_short | Energy and carbon-aware distributed machine learning tasks scheduling scheme for the multi-renewable energy-based edge-cloud continuum |
| title_sort | energy and carbon aware distributed machine learning tasks scheduling scheme for the multi renewable energy based edge cloud continuum |
| topic | edge-cloud continuum dml task schedulilng distributed data centers multi-renewable energy energy consumption carbon emission energy complementarity |
| url | https://www.stet-review.org/articles/stet/full_html/2024/01/stet20240197/stet20240197.html |
| work_keys_str_mv | AT miaozicong energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT liulei energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT nanhaijing energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT liweize energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT panxiaodong energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT yangxin energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT yumi energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT chenhui energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum AT zhaoyiming energyandcarbonawaredistributedmachinelearningtasksschedulingschemeforthemultirenewableenergybasededgecloudcontinuum |