Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data
The large spatial/temporal/frequency scale of geoscience and remote-sensing datasets causes memory issues when using convolutional neural networks for (sub-) surface data segmentation. Recently developed fully reversible or fully invertible networks can mostly avoid memory limitations by recomputing...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co. Ltd.
2024-12-01
|
| Series: | Artificial Intelligence in Geosciences |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666544124000285 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850180277514534912 |
|---|---|
| author | Bas Peters Eldad Haber Keegan Lensink |
| author_facet | Bas Peters Eldad Haber Keegan Lensink |
| author_sort | Bas Peters |
| collection | DOAJ |
| description | The large spatial/temporal/frequency scale of geoscience and remote-sensing datasets causes memory issues when using convolutional neural networks for (sub-) surface data segmentation. Recently developed fully reversible or fully invertible networks can mostly avoid memory limitations by recomputing the states during the backward pass through the network. This results in a low and fixed memory requirement for storing network states, as opposed to the typical linear memory growth with network depth. This work focuses on a fully invertible network based on the telegraph equation. While reversibility saves the major amount of memory used in deep networks by the data, the convolutional kernels can take up most memory if fully invertible networks contain multiple invertible pooling/coarsening layers. We address the explosion of the number of convolutional kernels by combining fully invertible networks with layers that contain the convolutional kernels in a compressed form directly. A second challenge is that invertible networks output a tensor the same size as its input. This property prevents the straightforward application of invertible networks to applications that map between different input–output dimensions, need to map to outputs with more channels than present in the input data, or desire outputs that decrease/increase the resolution compared to the input data. However, we show that by employing invertible networks in a non-standard fashion, we can still use them for these tasks. Examples in hyperspectral land-use classification, airborne geophysical surveying, and seismic imaging illustrate that we can input large data volumes in one chunk and do not need to work on small patches, use dimensionality reduction, or employ methods that classify a patch to a single central pixel. |
| format | Article |
| id | doaj-art-8c33c70e82444bcd9431b1e5e92fc40c |
| institution | OA Journals |
| issn | 2666-5441 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | KeAi Communications Co. Ltd. |
| record_format | Article |
| series | Artificial Intelligence in Geosciences |
| spelling | doaj-art-8c33c70e82444bcd9431b1e5e92fc40c2025-08-20T02:18:15ZengKeAi Communications Co. Ltd.Artificial Intelligence in Geosciences2666-54412024-12-01510008710.1016/j.aiig.2024.100087Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface dataBas Peters0Eldad Haber1Keegan Lensink2Computational Geosciences Inc., Vancouver, V6H 3Y4, BC, Canada; Corresponding author.The University of British Columbia, Department of Earth, Ocean, and Atmospheric Sciences, Vancouver, V6T 1Z4, BC, CanadaThe University of British Columbia, Department of Earth, Ocean, and Atmospheric Sciences, Vancouver, V6T 1Z4, BC, CanadaThe large spatial/temporal/frequency scale of geoscience and remote-sensing datasets causes memory issues when using convolutional neural networks for (sub-) surface data segmentation. Recently developed fully reversible or fully invertible networks can mostly avoid memory limitations by recomputing the states during the backward pass through the network. This results in a low and fixed memory requirement for storing network states, as opposed to the typical linear memory growth with network depth. This work focuses on a fully invertible network based on the telegraph equation. While reversibility saves the major amount of memory used in deep networks by the data, the convolutional kernels can take up most memory if fully invertible networks contain multiple invertible pooling/coarsening layers. We address the explosion of the number of convolutional kernels by combining fully invertible networks with layers that contain the convolutional kernels in a compressed form directly. A second challenge is that invertible networks output a tensor the same size as its input. This property prevents the straightforward application of invertible networks to applications that map between different input–output dimensions, need to map to outputs with more channels than present in the input data, or desire outputs that decrease/increase the resolution compared to the input data. However, we show that by employing invertible networks in a non-standard fashion, we can still use them for these tasks. Examples in hyperspectral land-use classification, airborne geophysical surveying, and seismic imaging illustrate that we can input large data volumes in one chunk and do not need to work on small patches, use dimensionality reduction, or employ methods that classify a patch to a single central pixel.http://www.sciencedirect.com/science/article/pii/S2666544124000285Invertible neural networksLarge scale deep learningMemory efficient deep learning |
| spellingShingle | Bas Peters Eldad Haber Keegan Lensink Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data Artificial Intelligence in Geosciences Invertible neural networks Large scale deep learning Memory efficient deep learning |
| title | Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data |
| title_full | Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data |
| title_fullStr | Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data |
| title_full_unstemmed | Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data |
| title_short | Fully invertible hyperbolic neural networks for segmenting large-scale surface and sub-surface data |
| title_sort | fully invertible hyperbolic neural networks for segmenting large scale surface and sub surface data |
| topic | Invertible neural networks Large scale deep learning Memory efficient deep learning |
| url | http://www.sciencedirect.com/science/article/pii/S2666544124000285 |
| work_keys_str_mv | AT baspeters fullyinvertiblehyperbolicneuralnetworksforsegmentinglargescalesurfaceandsubsurfacedata AT eldadhaber fullyinvertiblehyperbolicneuralnetworksforsegmentinglargescalesurfaceandsubsurfacedata AT keeganlensink fullyinvertiblehyperbolicneuralnetworksforsegmentinglargescalesurfaceandsubsurfacedata |