Dataset resulting from the user study on comprehensibility of explainable AI algorithms
Abstract This paper introduces a dataset that is the result of a user study on the comprehensibility of explainable artificial intelligence (XAI) algorithms. The study participants were recruited from 149 candidates to form three groups representing experts in the domain of mycology (DE), students w...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-06-01
|
| Series: | Scientific Data |
| Online Access: | https://doi.org/10.1038/s41597-025-05167-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850222293447344128 |
|---|---|
| author | Szymon Bobek Paloma Korycińska Monika Krakowska Maciej Mozolewski Dorota Rak Magdalena Zych Magdalena Wójcik Grzegorz J. Nalepa |
| author_facet | Szymon Bobek Paloma Korycińska Monika Krakowska Maciej Mozolewski Dorota Rak Magdalena Zych Magdalena Wójcik Grzegorz J. Nalepa |
| author_sort | Szymon Bobek |
| collection | DOAJ |
| description | Abstract This paper introduces a dataset that is the result of a user study on the comprehensibility of explainable artificial intelligence (XAI) algorithms. The study participants were recruited from 149 candidates to form three groups representing experts in the domain of mycology (DE), students with a data science and visualization background (IT) and students from social sciences and humanities (SSH). The main part of the dataset contains 39 transcripts of interviews during which participants were asked to complete a series of tasks and questions related to the interpretation of explanations of decisions of a machine learning model trained to distinguish between edible and inedible mushrooms. The transcripts were complemented with additional data that includes visualizations of explanations presented to the user, results from thematic analysis, recommendations of improvements of explanations provided by the participants, and the initial survey results that allow to determine the domain knowledge of the participant and data analysis literacy. The transcripts were manually tagged to allow for automatic matching between the text and other data related to particular fragments. In the advent of the area of rapid development of XAI techniques, the need for a multidisciplinary qualitative evaluation of explainability is one of the emerging topics in the community. Our dataset allows not only to reproduce the study we conducted, but also to open a wide range of possibilities for the analysis of the material we gathered. |
| format | Article |
| id | doaj-art-e63ab07ea3aa4c41acc518e5950e13b5 |
| institution | OA Journals |
| issn | 2052-4463 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Data |
| spelling | doaj-art-e63ab07ea3aa4c41acc518e5950e13b52025-08-20T02:06:23ZengNature PortfolioScientific Data2052-44632025-06-0112111210.1038/s41597-025-05167-6Dataset resulting from the user study on comprehensibility of explainable AI algorithmsSzymon Bobek0Paloma Korycińska1Monika Krakowska2Maciej Mozolewski3Dorota Rak4Magdalena Zych5Magdalena Wójcik6Grzegorz J. Nalepa7Jagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research, Institute of Applied Computer Science, Jagiellonian UniversityInstitute of Information Studies, Faculty of Management and Social Communication, Jagiellonian UniversityInstitute of Information Studies, Faculty of Management and Social Communication, Jagiellonian UniversityJagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research, Institute of Applied Computer Science, Jagiellonian UniversityInstitute of Information Studies, Faculty of Management and Social Communication, Jagiellonian UniversityInstitute of Information Studies, Faculty of Management and Social Communication, Jagiellonian UniversityInstitute of Information Studies, Faculty of Management and Social Communication, Jagiellonian UniversityJagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research, Institute of Applied Computer Science, Jagiellonian UniversityAbstract This paper introduces a dataset that is the result of a user study on the comprehensibility of explainable artificial intelligence (XAI) algorithms. The study participants were recruited from 149 candidates to form three groups representing experts in the domain of mycology (DE), students with a data science and visualization background (IT) and students from social sciences and humanities (SSH). The main part of the dataset contains 39 transcripts of interviews during which participants were asked to complete a series of tasks and questions related to the interpretation of explanations of decisions of a machine learning model trained to distinguish between edible and inedible mushrooms. The transcripts were complemented with additional data that includes visualizations of explanations presented to the user, results from thematic analysis, recommendations of improvements of explanations provided by the participants, and the initial survey results that allow to determine the domain knowledge of the participant and data analysis literacy. The transcripts were manually tagged to allow for automatic matching between the text and other data related to particular fragments. In the advent of the area of rapid development of XAI techniques, the need for a multidisciplinary qualitative evaluation of explainability is one of the emerging topics in the community. Our dataset allows not only to reproduce the study we conducted, but also to open a wide range of possibilities for the analysis of the material we gathered.https://doi.org/10.1038/s41597-025-05167-6 |
| spellingShingle | Szymon Bobek Paloma Korycińska Monika Krakowska Maciej Mozolewski Dorota Rak Magdalena Zych Magdalena Wójcik Grzegorz J. Nalepa Dataset resulting from the user study on comprehensibility of explainable AI algorithms Scientific Data |
| title | Dataset resulting from the user study on comprehensibility of explainable AI algorithms |
| title_full | Dataset resulting from the user study on comprehensibility of explainable AI algorithms |
| title_fullStr | Dataset resulting from the user study on comprehensibility of explainable AI algorithms |
| title_full_unstemmed | Dataset resulting from the user study on comprehensibility of explainable AI algorithms |
| title_short | Dataset resulting from the user study on comprehensibility of explainable AI algorithms |
| title_sort | dataset resulting from the user study on comprehensibility of explainable ai algorithms |
| url | https://doi.org/10.1038/s41597-025-05167-6 |
| work_keys_str_mv | AT szymonbobek datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT palomakorycinska datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT monikakrakowska datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT maciejmozolewski datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT dorotarak datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT magdalenazych datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT magdalenawojcik datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms AT grzegorzjnalepa datasetresultingfromtheuserstudyoncomprehensibilityofexplainableaialgorithms |