Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach
The classification of clinical notes into specific diagnostic categories is critical in healthcare, especially for mental health conditions like anxiety and adjustment disorder. In this study, we compare the performance of various artificial intelligence models, including both traditional machine le...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2025-07-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-3045.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850055285400403968 |
|---|---|
| author | Sergio Rubio-Martín María Teresa García-Ordás Antonio Serrano-García Clara Margarita Franch-Pato Arturo Crespo-Álvaro José Alberto Benítez-Andrades |
| author_facet | Sergio Rubio-Martín María Teresa García-Ordás Antonio Serrano-García Clara Margarita Franch-Pato Arturo Crespo-Álvaro José Alberto Benítez-Andrades |
| author_sort | Sergio Rubio-Martín |
| collection | DOAJ |
| description | The classification of clinical notes into specific diagnostic categories is critical in healthcare, especially for mental health conditions like anxiety and adjustment disorder. In this study, we compare the performance of various artificial intelligence models, including both traditional machine learning approaches (random forest, support vector machine, K-nearest neighbors, decision tree, and eXtreme Gradient Boost) and deep learning models (DistilBERT and SciBERT), to classify clinical notes into these two diagnoses. Additionally, we implemented three oversampling strategies: No Oversampling, Random Oversampling, and Synthetic Minority Over-sampling Technique (SMOTE), to assess their impact on model performance. Hyperparameter tuning was also applied to optimize model accuracy. Our results indicate that oversampling techniques had minimal impact on model performance overall. The only exception was SMOTE, which showed a positive effect specifically with Bidirectional Encoder Representations from Transformers (BERT)-based models. However, hyperparameter optimization significantly improved accuracy across the models, enhancing their ability to generalize and perform on the dataset. The decision tree and eXtreme Gradient Boost models achieved the highest accuracy among machine learning approaches, both reaching 96%, while the DistilBERT and SciBERT models also attained 96% accuracy in the deep learning category. These findings underscore the importance of hyperparameter tuning in maximizing model performance. This study contributes to the ongoing research on AI-assisted diagnostic tools in mental health by providing insights into the efficacy of different model architectures and data balancing methods. |
| format | Article |
| id | doaj-art-9c81bfa2b7594032b6f5dd4fed1eabcd |
| institution | DOAJ |
| issn | 2376-5992 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj-art-9c81bfa2b7594032b6f5dd4fed1eabcd2025-08-20T02:52:00ZengPeerJ Inc.PeerJ Computer Science2376-59922025-07-0111e304510.7717/peerj-cs.3045Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approachSergio Rubio-Martín0María Teresa García-Ordás1Antonio Serrano-García2Clara Margarita Franch-Pato3Arturo Crespo-Álvaro4José Alberto Benítez-Andrades5ALBA Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, León, SpainALBA Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, León, SpainServicio de Psiquiatría, Complejo Asistencial Universitario de León (CAULE), León, SpainServicio de Psiquiatría, Complejo Asistencial Universitario de León (CAULE), León, SpainALBA Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, León, SpainALBA Research Group, Department of Electric, Systems and Automatics Engineering, Universidad de León, León, SpainThe classification of clinical notes into specific diagnostic categories is critical in healthcare, especially for mental health conditions like anxiety and adjustment disorder. In this study, we compare the performance of various artificial intelligence models, including both traditional machine learning approaches (random forest, support vector machine, K-nearest neighbors, decision tree, and eXtreme Gradient Boost) and deep learning models (DistilBERT and SciBERT), to classify clinical notes into these two diagnoses. Additionally, we implemented three oversampling strategies: No Oversampling, Random Oversampling, and Synthetic Minority Over-sampling Technique (SMOTE), to assess their impact on model performance. Hyperparameter tuning was also applied to optimize model accuracy. Our results indicate that oversampling techniques had minimal impact on model performance overall. The only exception was SMOTE, which showed a positive effect specifically with Bidirectional Encoder Representations from Transformers (BERT)-based models. However, hyperparameter optimization significantly improved accuracy across the models, enhancing their ability to generalize and perform on the dataset. The decision tree and eXtreme Gradient Boost models achieved the highest accuracy among machine learning approaches, both reaching 96%, while the DistilBERT and SciBERT models also attained 96% accuracy in the deep learning category. These findings underscore the importance of hyperparameter tuning in maximizing model performance. This study contributes to the ongoing research on AI-assisted diagnostic tools in mental health by providing insights into the efficacy of different model architectures and data balancing methods.https://peerj.com/articles/cs-3045.pdfArtificial intelligenceClinical notesPsychiatryElectronic health recordsAdjustment disorderAnxiety |
| spellingShingle | Sergio Rubio-Martín María Teresa García-Ordás Antonio Serrano-García Clara Margarita Franch-Pato Arturo Crespo-Álvaro José Alberto Benítez-Andrades Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach PeerJ Computer Science Artificial intelligence Clinical notes Psychiatry Electronic health records Adjustment disorder Anxiety |
| title | Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach |
| title_full | Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach |
| title_fullStr | Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach |
| title_full_unstemmed | Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach |
| title_short | Classification of psychiatry clinical notes by diagnosis: a deep learning and machine learning approach |
| title_sort | classification of psychiatry clinical notes by diagnosis a deep learning and machine learning approach |
| topic | Artificial intelligence Clinical notes Psychiatry Electronic health records Adjustment disorder Anxiety |
| url | https://peerj.com/articles/cs-3045.pdf |
| work_keys_str_mv | AT sergiorubiomartin classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach AT mariateresagarciaordas classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach AT antonioserranogarcia classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach AT claramargaritafranchpato classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach AT arturocrespoalvaro classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach AT josealbertobenitezandrades classificationofpsychiatryclinicalnotesbydiagnosisadeeplearningandmachinelearningapproach |