The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability
Human-on-the-loop (HOTL) evaluation enables decision-makers to provide qualitative insights that support standard performance metrics. While these metrics are designed to assess convergence and diversity of solutions using statistical measures, they often fail to grasp problem-specific characteristi...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-09-01
|
| Series: | Results in Control and Optimization |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666720725000918 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849236185670483968 |
|---|---|
| author | Isaac Okola Daniel Orwa Ochieng Evans Kirimi Miriti Gilbert Ouma Ong'isa |
| author_facet | Isaac Okola Daniel Orwa Ochieng Evans Kirimi Miriti Gilbert Ouma Ong'isa |
| author_sort | Isaac Okola |
| collection | DOAJ |
| description | Human-on-the-loop (HOTL) evaluation enables decision-makers to provide qualitative insights that support standard performance metrics. While these metrics are designed to assess convergence and diversity of solutions using statistical measures, they often fail to grasp problem-specific characteristics inherent in noisy and uncertain real-world problems, subsequently generating inaccurate or uninterpretable numerical results. Additionally, the lack of qualitative insights and decision-maker preferences in performance evaluation compromises the effectiveness of identifying the appropriate algorithm to solve a specific real-world multiobjective optimization problem (MOP). This research applies graphical analysis, examination of objectives, and decision variable values to complement statistical analysis. The HOTL concept is explored as an “Interaction after a complete run (IAR)” mechanism for evaluating optimization and performance results through human intervention. It is achieved by implementing the inter-rater reliability (IRR) evaluation facilitated by computing Krippendorff’s alpha scores. Based on the results, the Fuzzy Decision Variable (FDV) algorithm is selected as the most suitable algorithm to solve a multiobjective problem that integrates the Food-Energy-Water Nexus (FEWN) and Climate Change Vulnerability (CCV). The findings highlight the HOTL's importance in addressing the shortcomings of the existing performance metrics, such as ambiguous and inaccurate results. Based on the identified knowledge gaps, the findings, and the limitations of this research, we propose future research areas that can be undertaken to improve the performance evaluation of algorithms. Such research areas include incorporating machine learning to predict, using performance data, the most suitable algorithm to solve a specific problem, and advancing interactive learning and user adjustments of the optimization process. |
| format | Article |
| id | doaj-art-2d67d59287d748c4a383f76afd1f30df |
| institution | Kabale University |
| issn | 2666-7207 |
| language | English |
| publishDate | 2025-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Results in Control and Optimization |
| spelling | doaj-art-2d67d59287d748c4a383f76afd1f30df2025-08-20T04:02:27ZengElsevierResults in Control and Optimization2666-72072025-09-012010060610.1016/j.rico.2025.100606The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerabilityIsaac Okola0Daniel Orwa Ochieng1Evans Kirimi Miriti2Gilbert Ouma Ong'isa3Department of Applied Computing and Networks, KCA University, Ruaraka, Thika Superhighway, Kenya; Corresponding author.Department of Computing and Informatics, University of Nairobi, Chiromo, Waiyaki Way, KenyaDepartment of Computing and Informatics, University of Nairobi, Chiromo, Waiyaki Way, KenyaInstitute for Climate Change Adaptation, University of Nairobi, Chiromo, Waiyaki Way, KenyaHuman-on-the-loop (HOTL) evaluation enables decision-makers to provide qualitative insights that support standard performance metrics. While these metrics are designed to assess convergence and diversity of solutions using statistical measures, they often fail to grasp problem-specific characteristics inherent in noisy and uncertain real-world problems, subsequently generating inaccurate or uninterpretable numerical results. Additionally, the lack of qualitative insights and decision-maker preferences in performance evaluation compromises the effectiveness of identifying the appropriate algorithm to solve a specific real-world multiobjective optimization problem (MOP). This research applies graphical analysis, examination of objectives, and decision variable values to complement statistical analysis. The HOTL concept is explored as an “Interaction after a complete run (IAR)” mechanism for evaluating optimization and performance results through human intervention. It is achieved by implementing the inter-rater reliability (IRR) evaluation facilitated by computing Krippendorff’s alpha scores. Based on the results, the Fuzzy Decision Variable (FDV) algorithm is selected as the most suitable algorithm to solve a multiobjective problem that integrates the Food-Energy-Water Nexus (FEWN) and Climate Change Vulnerability (CCV). The findings highlight the HOTL's importance in addressing the shortcomings of the existing performance metrics, such as ambiguous and inaccurate results. Based on the identified knowledge gaps, the findings, and the limitations of this research, we propose future research areas that can be undertaken to improve the performance evaluation of algorithms. Such research areas include incorporating machine learning to predict, using performance data, the most suitable algorithm to solve a specific problem, and advancing interactive learning and user adjustments of the optimization process.http://www.sciencedirect.com/science/article/pii/S2666720725000918Multiobjective optimization algorithmsFood-energy-water nexusClimate change vulnerabilityPerformance metric evaluationHuman-on-the-Loop evaluationInter-rater reliability |
| spellingShingle | Isaac Okola Daniel Orwa Ochieng Evans Kirimi Miriti Gilbert Ouma Ong'isa The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability Results in Control and Optimization Multiobjective optimization algorithms Food-energy-water nexus Climate change vulnerability Performance metric evaluation Human-on-the-Loop evaluation Inter-rater reliability |
| title | The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability |
| title_full | The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability |
| title_fullStr | The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability |
| title_full_unstemmed | The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability |
| title_short | The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability |
| title_sort | human on the loop evaluation of multiobjective optimization algorithms for solving a real world problem that integrates the food energy water nexus security and climate change vulnerability |
| topic | Multiobjective optimization algorithms Food-energy-water nexus Climate change vulnerability Performance metric evaluation Human-on-the-Loop evaluation Inter-rater reliability |
| url | http://www.sciencedirect.com/science/article/pii/S2666720725000918 |
| work_keys_str_mv | AT isaacokola thehumanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT danielorwaochieng thehumanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT evanskirimimiriti thehumanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT gilbertoumaongisa thehumanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT isaacokola humanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT danielorwaochieng humanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT evanskirimimiriti humanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability AT gilbertoumaongisa humanontheloopevaluationofmultiobjectiveoptimizationalgorithmsforsolvingarealworldproblemthatintegratesthefoodenergywaternexussecurityandclimatechangevulnerability |