Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets
Abstract Small datasets are common in many fields due to factors such as limited data collection opportunities or privacy concerns. These datasets often contain high-dimensional features, yet present significant challenges of dimensionality, wherein the sparsity of data in high-dimensional spaces ma...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-07725-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849334663714177024 |
|---|---|
| author | Hongqi Niu Gabrielle B. McCallum Anne B. Chang Khalid Khan Sami Azam |
| author_facet | Hongqi Niu Gabrielle B. McCallum Anne B. Chang Khalid Khan Sami Azam |
| author_sort | Hongqi Niu |
| collection | DOAJ |
| description | Abstract Small datasets are common in many fields due to factors such as limited data collection opportunities or privacy concerns. These datasets often contain high-dimensional features, yet present significant challenges of dimensionality, wherein the sparsity of data in high-dimensional spaces makes it difficult to extract meaningful information and less accurate predictive models are produced. In this regard, feature extraction algorithms are important in addressing these challenges by reducing dimensionality while retaining essential information. These algorithms can be classified into supervised, unsupervised, and semi-supervised methods and categorized as linear or nonlinear. To overview this critical issue, this review focuses on unsupervised feature extraction algorithms (UFEAs) due to their ability to handle high-dimensional data without relying on labelled information. From this review, eight representative UFEAs were selected: principal component analysis, classical multidimensional scaling, Kernel PCA, isometric mapping, locally linear embedding, Laplacian Eigenmaps, independent component analysis and Autoencoders. The theoretical background of these algorithms has been presented, discussing their conceptual viewpoints, such as whether they are linear or nonlinear, manifold-based, probabilistic density function-based, or neural network-based. After classifying these algorithms using these taxonomies, we thoroughly and systematically reviewed each algorithm from the perspective of their working mechanisms, providing a detailed algorithmic explanation for each UFEA. We also explored how these mechanisms contribute to an effective reduction in dimensionality, particularly in small datasets with high dimensionality. Furthermore, we compared these algorithms in terms of transformation approach, goals, parameters, and computational complexity. Finally, we evaluated each algorithm against state-of-the-art research using various datasets to assess their accuracy, highlighting which algorithm is most appropriate for specific scenarios. Overall, this review provides insights into the strengths and weaknesses of various UFEAs, offering guidance on selecting appropriate algorithms for small high-dimensional datasets. |
| format | Article |
| id | doaj-art-032b558e476342358a63bb61dad9993c |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-032b558e476342358a63bb61dad9993c2025-08-20T03:45:30ZengNature PortfolioScientific Reports2045-23222025-07-0115112110.1038/s41598-025-07725-9Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasetsHongqi Niu0Gabrielle B. McCallum1Anne B. Chang2Khalid Khan3Sami Azam4Faculty of Science and Technology, Charles Darwin UniversityChild and Maternal Health Division and NHMRC Centre for Research Excellence in Paediatric Bronchiectasis (AusBREATHE), Menzies School of Health Research, Charles Darwin UniversityChild and Maternal Health Division and NHMRC Centre for Research Excellence in Paediatric Bronchiectasis (AusBREATHE), Menzies School of Health Research, Charles Darwin UniversityFaculty of Arts and Society, Charles Darwin UniversityFaculty of Science and Technology, Charles Darwin UniversityAbstract Small datasets are common in many fields due to factors such as limited data collection opportunities or privacy concerns. These datasets often contain high-dimensional features, yet present significant challenges of dimensionality, wherein the sparsity of data in high-dimensional spaces makes it difficult to extract meaningful information and less accurate predictive models are produced. In this regard, feature extraction algorithms are important in addressing these challenges by reducing dimensionality while retaining essential information. These algorithms can be classified into supervised, unsupervised, and semi-supervised methods and categorized as linear or nonlinear. To overview this critical issue, this review focuses on unsupervised feature extraction algorithms (UFEAs) due to their ability to handle high-dimensional data without relying on labelled information. From this review, eight representative UFEAs were selected: principal component analysis, classical multidimensional scaling, Kernel PCA, isometric mapping, locally linear embedding, Laplacian Eigenmaps, independent component analysis and Autoencoders. The theoretical background of these algorithms has been presented, discussing their conceptual viewpoints, such as whether they are linear or nonlinear, manifold-based, probabilistic density function-based, or neural network-based. After classifying these algorithms using these taxonomies, we thoroughly and systematically reviewed each algorithm from the perspective of their working mechanisms, providing a detailed algorithmic explanation for each UFEA. We also explored how these mechanisms contribute to an effective reduction in dimensionality, particularly in small datasets with high dimensionality. Furthermore, we compared these algorithms in terms of transformation approach, goals, parameters, and computational complexity. Finally, we evaluated each algorithm against state-of-the-art research using various datasets to assess their accuracy, highlighting which algorithm is most appropriate for specific scenarios. Overall, this review provides insights into the strengths and weaknesses of various UFEAs, offering guidance on selecting appropriate algorithms for small high-dimensional datasets.https://doi.org/10.1038/s41598-025-07725-9UnsupervisedHigh dimensionalitySmall datasetsFeature extraction |
| spellingShingle | Hongqi Niu Gabrielle B. McCallum Anne B. Chang Khalid Khan Sami Azam Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets Scientific Reports Unsupervised High dimensionality Small datasets Feature extraction |
| title | Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets |
| title_full | Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets |
| title_fullStr | Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets |
| title_full_unstemmed | Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets |
| title_short | Exploring unsupervised feature extraction algorithms: tackling high dimensionality in small datasets |
| title_sort | exploring unsupervised feature extraction algorithms tackling high dimensionality in small datasets |
| topic | Unsupervised High dimensionality Small datasets Feature extraction |
| url | https://doi.org/10.1038/s41598-025-07725-9 |
| work_keys_str_mv | AT hongqiniu exploringunsupervisedfeatureextractionalgorithmstacklinghighdimensionalityinsmalldatasets AT gabriellebmccallum exploringunsupervisedfeatureextractionalgorithmstacklinghighdimensionalityinsmalldatasets AT annebchang exploringunsupervisedfeatureextractionalgorithmstacklinghighdimensionalityinsmalldatasets AT khalidkhan exploringunsupervisedfeatureextractionalgorithmstacklinghighdimensionalityinsmalldatasets AT samiazam exploringunsupervisedfeatureextractionalgorithmstacklinghighdimensionalityinsmalldatasets |