SpectralEarth: Training Hyperspectral Foundation Models at Scale
Foundation models have triggered a paradigm shift in computer vision and are increasingly being adopted in remote sensing, particularly for multispectral imagery. Yet, their potential in hyperspectral imaging (HSI) remains untapped due to the absence of comprehensive and globally representative hype...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11045062/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849718136431968256 |
|---|---|
| author | Nassim Ait Ali Braham Conrad M. Albrecht Julien Mairal Jocelyn Chanussot Yi Wang Xiao Xiang Zhu |
| author_facet | Nassim Ait Ali Braham Conrad M. Albrecht Julien Mairal Jocelyn Chanussot Yi Wang Xiao Xiang Zhu |
| author_sort | Nassim Ait Ali Braham |
| collection | DOAJ |
| description | Foundation models have triggered a paradigm shift in computer vision and are increasingly being adopted in remote sensing, particularly for multispectral imagery. Yet, their potential in hyperspectral imaging (HSI) remains untapped due to the absence of comprehensive and globally representative hyperspectral datasets. To close this gap, we introduce <italic>SpectralEarth</italic>, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models leveraging data from the environmental mapping and analysis program (EnMAP). SpectralEarth comprises 538 974 image patches covering 415 153 unique locations from 11 636 globally distributed EnMAP scenes spanning two years of archive. In addition, 17.5% of these locations include multiple timestamps, enabling multitemporal HSI analysis. Utilizing state-of-the-art self-supervised learning algorithms, we pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones to accommodate the unique characteristics of HSI. In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification, providing benchmarks for model evaluation. Experimental results support the versatility of our models and their generalizability across different tasks and sensors. We also highlight computational efficiency during model fine-tuning. |
| format | Article |
| id | doaj-art-345bf1b4e3bf4ee683ebdbe53f93436f |
| institution | DOAJ |
| issn | 1939-1404 2151-1535 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj-art-345bf1b4e3bf4ee683ebdbe53f93436f2025-08-20T03:12:27ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing1939-14042151-15352025-01-0118167801679710.1109/JSTARS.2025.358145111045062SpectralEarth: Training Hyperspectral Foundation Models at ScaleNassim Ait Ali Braham0https://orcid.org/0009-0001-3346-3373Conrad M. Albrecht1https://orcid.org/0009-0009-2422-7289Julien Mairal2https://orcid.org/0000-0001-6991-2110Jocelyn Chanussot3https://orcid.org/0000-0003-4817-2875Yi Wang4https://orcid.org/0000-0002-3096-6610Xiao Xiang Zhu5https://orcid.org/0000-0001-8107-9096Chair of Data Science in Earth Observation, Technical University of Munich, Munich, GermanyRemote Sensing Technology Institute, German Aerospace Center, Wessling, GermanyUniversity Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, FranceUniversity Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, Grenoble, FranceChair of Data Science in Earth Observation, Technical University of Munich, Munich, GermanyChair of Data Science in Earth Observation, Technical University of Munich, Munich, GermanyFoundation models have triggered a paradigm shift in computer vision and are increasingly being adopted in remote sensing, particularly for multispectral imagery. Yet, their potential in hyperspectral imaging (HSI) remains untapped due to the absence of comprehensive and globally representative hyperspectral datasets. To close this gap, we introduce <italic>SpectralEarth</italic>, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models leveraging data from the environmental mapping and analysis program (EnMAP). SpectralEarth comprises 538 974 image patches covering 415 153 unique locations from 11 636 globally distributed EnMAP scenes spanning two years of archive. In addition, 17.5% of these locations include multiple timestamps, enabling multitemporal HSI analysis. Utilizing state-of-the-art self-supervised learning algorithms, we pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones to accommodate the unique characteristics of HSI. In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification, providing benchmarks for model evaluation. Experimental results support the versatility of our models and their generalizability across different tasks and sensors. We also highlight computational efficiency during model fine-tuning.https://ieeexplore.ieee.org/document/11045062/Hyperspectral imagingSelf-supervised learningFoundation models |
| spellingShingle | Nassim Ait Ali Braham Conrad M. Albrecht Julien Mairal Jocelyn Chanussot Yi Wang Xiao Xiang Zhu SpectralEarth: Training Hyperspectral Foundation Models at Scale IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Hyperspectral imaging Self-supervised learning Foundation models |
| title | SpectralEarth: Training Hyperspectral Foundation Models at Scale |
| title_full | SpectralEarth: Training Hyperspectral Foundation Models at Scale |
| title_fullStr | SpectralEarth: Training Hyperspectral Foundation Models at Scale |
| title_full_unstemmed | SpectralEarth: Training Hyperspectral Foundation Models at Scale |
| title_short | SpectralEarth: Training Hyperspectral Foundation Models at Scale |
| title_sort | spectralearth training hyperspectral foundation models at scale |
| topic | Hyperspectral imaging Self-supervised learning Foundation models |
| url | https://ieeexplore.ieee.org/document/11045062/ |
| work_keys_str_mv | AT nassimaitalibraham spectralearthtraininghyperspectralfoundationmodelsatscale AT conradmalbrecht spectralearthtraininghyperspectralfoundationmodelsatscale AT julienmairal spectralearthtraininghyperspectralfoundationmodelsatscale AT jocelynchanussot spectralearthtraininghyperspectralfoundationmodelsatscale AT yiwang spectralearthtraininghyperspectralfoundationmodelsatscale AT xiaoxiangzhu spectralearthtraininghyperspectralfoundationmodelsatscale |