Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency
This research investigates how Machine Learning (ML) algorithms can assist in workload allocation strategies by detecting tasks with node affinity operators (referred to as constraint operators), which constrain their execution to a limited number of nodes. Using real-world Google Cluster Data (GCD)...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10807210/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849220013047676928 |
|---|---|
| author | Leszek Sliwko |
| author_facet | Leszek Sliwko |
| author_sort | Leszek Sliwko |
| collection | DOAJ |
| description | This research investigates how Machine Learning (ML) algorithms can assist in workload allocation strategies by detecting tasks with node affinity operators (referred to as constraint operators), which constrain their execution to a limited number of nodes. Using real-world Google Cluster Data (GCD) workload traces and the AGOCS framework, the study extracts node attributes and task constraints, then analyses them to identify suitable node-task pairings. It focuses on tasks that can be executed on either a single node or fewer than a thousand out of 12.5k nodes in the analysed GCD cluster. Task constraint operators are compacted, pre-processed with one-hot encoding, and used as features in a training dataset. Various ML classifiers, including Artificial Neural Networks, K-Nearest Neighbours, Decision Trees, Naive Bayes, Ridge Regression, Adaptive Boosting, and Bagging, are fine-tuned and assessed for accuracy and F1-scores. The final ensemble voting classifier model achieved 98% accuracy and a 1.5-1.8% misclassification rate for tasks with a single suitable node. |
| format | Article |
| id | doaj-art-671282cafa7f43848bd57ab7513e7971 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-671282cafa7f43848bd57ab7513e79712024-12-25T00:01:43ZengIEEEIEEE Access2169-35362024-01-011219409119410710.1109/ACCESS.2024.352042210807210Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning EfficiencyLeszek Sliwko0https://orcid.org/0000-0002-1927-8710School of Computer Science and Engineering, University of Westminster, London, U.K.This research investigates how Machine Learning (ML) algorithms can assist in workload allocation strategies by detecting tasks with node affinity operators (referred to as constraint operators), which constrain their execution to a limited number of nodes. Using real-world Google Cluster Data (GCD) workload traces and the AGOCS framework, the study extracts node attributes and task constraints, then analyses them to identify suitable node-task pairings. It focuses on tasks that can be executed on either a single node or fewer than a thousand out of 12.5k nodes in the analysed GCD cluster. Task constraint operators are compacted, pre-processed with one-hot encoding, and used as features in a training dataset. Various ML classifiers, including Artificial Neural Networks, K-Nearest Neighbours, Decision Trees, Naive Bayes, Ridge Regression, Adaptive Boosting, and Bagging, are fine-tuned and assessed for accuracy and F1-scores. The final ensemble voting classifier model achieved 98% accuracy and a 1.5-1.8% misclassification rate for tasks with a single suitable node.https://ieeexplore.ieee.org/document/10807210/Machine learningclassification algorithmsload balancing and task assignmentGoogle cluster data |
| spellingShingle | Leszek Sliwko Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency IEEE Access Machine learning classification algorithms load balancing and task assignment Google cluster data |
| title | Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency |
| title_full | Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency |
| title_fullStr | Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency |
| title_full_unstemmed | Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency |
| title_short | Cluster Workload Allocation: A Predictive Approach Leveraging Machine Learning Efficiency |
| title_sort | cluster workload allocation a predictive approach leveraging machine learning efficiency |
| topic | Machine learning classification algorithms load balancing and task assignment Google cluster data |
| url | https://ieeexplore.ieee.org/document/10807210/ |
| work_keys_str_mv | AT leszeksliwko clusterworkloadallocationapredictiveapproachleveragingmachinelearningefficiency |