BenchMake: turn any scientific data set into a reproducible benchmark
Benchmark data sets are a cornerstone of machine learning development and applications, ensuring new methods are robust, reliable and competitive. The relative rarity of benchmark sets in computational science, due to the uniqueness of the problems and the pace of change in the associated domains, m...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IOP Publishing
2025-01-01
|
| Series: | Machine Learning: Science and Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1088/2632-2153/adf810 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849399329192673280 |
|---|---|
| author | A S Barnard |
| author_facet | A S Barnard |
| author_sort | A S Barnard |
| collection | DOAJ |
| description | Benchmark data sets are a cornerstone of machine learning development and applications, ensuring new methods are robust, reliable and competitive. The relative rarity of benchmark sets in computational science, due to the uniqueness of the problems and the pace of change in the associated domains, makes evaluating new innovations difficult for computational scientists. In this paper a new tool is developed and tested to potentially turn any of the increasing numbers of scientific data sets made openly available into a benchmark accessible to the community. BenchMake uses non-negative matrix factorization to deterministically identify and isolate challenging edge cases on the convex hull (the smallest convex set that contains all existing data instances) and partitions a required fraction of matched data instances into a testing set that maximizes divergence and statistical significance, across tabular, graph, image, signal and textual modalities. BenchMake splits are compared to establish splits and random splits using ten publicly available benchmark sets from different areas of science, with different sizes, shapes, distributions. |
| format | Article |
| id | doaj-art-2ee7b94ca6de43708d096ffc943d049e |
| institution | Kabale University |
| issn | 2632-2153 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| series | Machine Learning: Science and Technology |
| spelling | doaj-art-2ee7b94ca6de43708d096ffc943d049e2025-08-20T03:38:22ZengIOP PublishingMachine Learning: Science and Technology2632-21532025-01-016303050210.1088/2632-2153/adf810BenchMake: turn any scientific data set into a reproducible benchmarkA S Barnard0https://orcid.org/0000-0002-4784-2382School of Computing, Australian National University , Acton ACT 2601, AustraliaBenchmark data sets are a cornerstone of machine learning development and applications, ensuring new methods are robust, reliable and competitive. The relative rarity of benchmark sets in computational science, due to the uniqueness of the problems and the pace of change in the associated domains, makes evaluating new innovations difficult for computational scientists. In this paper a new tool is developed and tested to potentially turn any of the increasing numbers of scientific data sets made openly available into a benchmark accessible to the community. BenchMake uses non-negative matrix factorization to deterministically identify and isolate challenging edge cases on the convex hull (the smallest convex set that contains all existing data instances) and partitions a required fraction of matched data instances into a testing set that maximizes divergence and statistical significance, across tabular, graph, image, signal and textual modalities. BenchMake splits are compared to establish splits and random splits using ten publicly available benchmark sets from different areas of science, with different sizes, shapes, distributions.https://doi.org/10.1088/2632-2153/adf810scientific databenchmarkarchetypal analysissamplingdata splitmodel evaluation |
| spellingShingle | A S Barnard BenchMake: turn any scientific data set into a reproducible benchmark Machine Learning: Science and Technology scientific data benchmark archetypal analysis sampling data split model evaluation |
| title | BenchMake: turn any scientific data set into a reproducible benchmark |
| title_full | BenchMake: turn any scientific data set into a reproducible benchmark |
| title_fullStr | BenchMake: turn any scientific data set into a reproducible benchmark |
| title_full_unstemmed | BenchMake: turn any scientific data set into a reproducible benchmark |
| title_short | BenchMake: turn any scientific data set into a reproducible benchmark |
| title_sort | benchmake turn any scientific data set into a reproducible benchmark |
| topic | scientific data benchmark archetypal analysis sampling data split model evaluation |
| url | https://doi.org/10.1088/2632-2153/adf810 |
| work_keys_str_mv | AT asbarnard benchmaketurnanyscientificdatasetintoareproduciblebenchmark |