Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety
Improving industrial safety using NLP technologies supports the triple bottom line of environmental, social and economic sustainability. Rapid evolution of Large Language Models (LLMs) has potential to transform the industrial safety and improve disaster mitigation. In this paper, we evaluate and b...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
LibraryPress@UF
2025-05-01
|
| Series: | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| Subjects: | |
| Online Access: | https://journals.flvc.org/FLAIRS/article/view/138959 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849322597706104832 |
|---|---|
| author | Siddharth Tumre Sumit Koundanya Shubham Kumbhar Sangameshwar Patil |
| author_facet | Siddharth Tumre Sumit Koundanya Shubham Kumbhar Sangameshwar Patil |
| author_sort | Siddharth Tumre |
| collection | DOAJ |
| description |
Improving industrial safety using NLP technologies supports the triple bottom line of environmental, social and economic sustainability. Rapid evolution of Large Language Models (LLMs) has potential to transform the industrial safety and improve disaster mitigation. In this paper, we evaluate and benchmark the feasibility of using Falcon and Phi3 open-source LLMs for the task of generating preventive recommendations to improve industrial safety. Based on domain expert evaluation, we find that the standard, pre-trained LLMs have limitations concerning the quality and quantity of recommendations generated. They can be of diverse quality, such as specific, generic, or irrelevant. We find that the pre-trained version of Phi3 is better than base version of Falcon for the proposed task. We show that the quantity, output format as well as domain-awareness of the Falcon can be significantly improved using supervised fine-tuning (SFT) with a small amount of labeled data that illustrates the expected output. In spite of the quality improvement post-SFT and the high societal and economic impact of the application, there are still many areas of improvement, which we point to as part of future work. To the best of our knowledge, this is the first attempt to align LLMs for industrial safety recommendation improvement.
|
| format | Article |
| id | doaj-art-623640f8f3a540d0aeab13df06af60d8 |
| institution | Kabale University |
| issn | 2334-0754 2334-0762 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | LibraryPress@UF |
| record_format | Article |
| series | Proceedings of the International Florida Artificial Intelligence Research Society Conference |
| spelling | doaj-art-623640f8f3a540d0aeab13df06af60d82025-08-20T03:49:18ZengLibraryPress@UFProceedings of the International Florida Artificial Intelligence Research Society Conference2334-07542334-07622025-05-0138110.32473/flairs.38.1.138959Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial SafetySiddharth TumreSumit Koundanya0Shubham KumbharSangameshwar PatilTCS RESEARCH Improving industrial safety using NLP technologies supports the triple bottom line of environmental, social and economic sustainability. Rapid evolution of Large Language Models (LLMs) has potential to transform the industrial safety and improve disaster mitigation. In this paper, we evaluate and benchmark the feasibility of using Falcon and Phi3 open-source LLMs for the task of generating preventive recommendations to improve industrial safety. Based on domain expert evaluation, we find that the standard, pre-trained LLMs have limitations concerning the quality and quantity of recommendations generated. They can be of diverse quality, such as specific, generic, or irrelevant. We find that the pre-trained version of Phi3 is better than base version of Falcon for the proposed task. We show that the quantity, output format as well as domain-awareness of the Falcon can be significantly improved using supervised fine-tuning (SFT) with a small amount of labeled data that illustrates the expected output. In spite of the quality improvement post-SFT and the high societal and economic impact of the application, there are still many areas of improvement, which we point to as part of future work. To the best of our knowledge, this is the first attempt to align LLMs for industrial safety recommendation improvement. https://journals.flvc.org/FLAIRS/article/view/138959Industrial Safety ImprovementOpen-source Large Language ModelsPreventive RecommendationOccupational Safety and Health |
| spellingShingle | Siddharth Tumre Sumit Koundanya Shubham Kumbhar Sangameshwar Patil Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety Proceedings of the International Florida Artificial Intelligence Research Society Conference Industrial Safety Improvement Open-source Large Language Models Preventive Recommendation Occupational Safety and Health |
| title | Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety |
| title_full | Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety |
| title_fullStr | Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety |
| title_full_unstemmed | Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety |
| title_short | Aligning LLMs to Improve Specificity of Preventive Action Recommendations for Industrial Safety |
| title_sort | aligning llms to improve specificity of preventive action recommendations for industrial safety |
| topic | Industrial Safety Improvement Open-source Large Language Models Preventive Recommendation Occupational Safety and Health |
| url | https://journals.flvc.org/FLAIRS/article/view/138959 |
| work_keys_str_mv | AT siddharthtumre aligningllmstoimprovespecificityofpreventiveactionrecommendationsforindustrialsafety AT sumitkoundanya aligningllmstoimprovespecificityofpreventiveactionrecommendationsforindustrialsafety AT shubhamkumbhar aligningllmstoimprovespecificityofpreventiveactionrecommendationsforindustrialsafety AT sangameshwarpatil aligningllmstoimprovespecificityofpreventiveactionrecommendationsforindustrialsafety |