Estimating rainfall intensity based on surveillance audio and deep-learning
Rainfall data with high spatial and temporal resolutions are essential for urban hydrological modeling. Ubiquitous surveillance cameras can continuously record rainfall events through video and audio, so they have been recognized as potential rain gauges to supplement professional rainfall observati...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2024-11-01
|
| Series: | Environmental Science and Ecotechnology |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666498424000644 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846166909014245376 |
|---|---|
| author | Meizhen Wang Mingzheng Chen Ziran Wang Yuxuan Guo Yong Wu Wei Zhao Xuejun Liu |
| author_facet | Meizhen Wang Mingzheng Chen Ziran Wang Yuxuan Guo Yong Wu Wei Zhao Xuejun Liu |
| author_sort | Meizhen Wang |
| collection | DOAJ |
| description | Rainfall data with high spatial and temporal resolutions are essential for urban hydrological modeling. Ubiquitous surveillance cameras can continuously record rainfall events through video and audio, so they have been recognized as potential rain gauges to supplement professional rainfall observation networks. Since video-based rainfall estimation methods can be affected by variable backgrounds and lighting conditions, audio-based approaches could be a supplement without suffering from these conditions. However, most audio-based approaches focus on rainfall-level classification rather than rainfall intensity estimation. Here, we introduce a dataset named Surveillance Audio Rainfall Intensity Dataset (SARID) and a deep learning model for estimating rainfall intensity. First, we created the dataset through audio of six real-world rainfall events. This dataset's audio recordings are segmented into 12,066 pieces and annotated with rainfall intensity and environmental information, such as underlying surfaces, temperature, humidity, and wind. Then, we developed a deep learning-based baseline using Mel-Frequency Cepstral Coefficients (MFCC) and Transformer architecture to estimate rainfall intensity from surveillance audio. Validated from ground truth data, our baseline achieves a root mean absolute error of 0.88 mm h-1 and a coefficient of correlation of 0.765. Our findings demonstrate the potential of surveillance audio-based models as practical and effective tools for rainfall observation systems, initiating a new chapter in rainfall intensity estimation. It offers a novel data source for high-resolution hydrological sensing and contributes to the broader landscape of urban sensing, emergency response, and resilience. |
| format | Article |
| id | doaj-art-8390c4d9273d47f4b3c01988108709c4 |
| institution | Kabale University |
| issn | 2666-4984 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Environmental Science and Ecotechnology |
| spelling | doaj-art-8390c4d9273d47f4b3c01988108709c42024-11-15T06:15:13ZengElsevierEnvironmental Science and Ecotechnology2666-49842024-11-0122100450Estimating rainfall intensity based on surveillance audio and deep-learningMeizhen Wang0Mingzheng Chen1Ziran Wang2Yuxuan Guo3Yong Wu4Wei Zhao5Xuejun Liu6Key Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, ChinaKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, ChinaKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, China; School of Information Engineering, Nanjing Normal University Taizhou College, Taizhou 225300, ChinaKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, ChinaInstitute of Geography, Fujian Normal University, Fuzhou, 350000, ChinaKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, ChinaKey Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China; State Key Laboratory Cultivation Base of Geographical Environment Evolution (Jiangsu Province), Nanjing, 210023, China; Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, 210023, China; Corresponding author. Key Laboratory of Virtual Geographic Environment (Nanjing Normal University), Ministry of Education, Nanjing, 210023, China.Rainfall data with high spatial and temporal resolutions are essential for urban hydrological modeling. Ubiquitous surveillance cameras can continuously record rainfall events through video and audio, so they have been recognized as potential rain gauges to supplement professional rainfall observation networks. Since video-based rainfall estimation methods can be affected by variable backgrounds and lighting conditions, audio-based approaches could be a supplement without suffering from these conditions. However, most audio-based approaches focus on rainfall-level classification rather than rainfall intensity estimation. Here, we introduce a dataset named Surveillance Audio Rainfall Intensity Dataset (SARID) and a deep learning model for estimating rainfall intensity. First, we created the dataset through audio of six real-world rainfall events. This dataset's audio recordings are segmented into 12,066 pieces and annotated with rainfall intensity and environmental information, such as underlying surfaces, temperature, humidity, and wind. Then, we developed a deep learning-based baseline using Mel-Frequency Cepstral Coefficients (MFCC) and Transformer architecture to estimate rainfall intensity from surveillance audio. Validated from ground truth data, our baseline achieves a root mean absolute error of 0.88 mm h-1 and a coefficient of correlation of 0.765. Our findings demonstrate the potential of surveillance audio-based models as practical and effective tools for rainfall observation systems, initiating a new chapter in rainfall intensity estimation. It offers a novel data source for high-resolution hydrological sensing and contributes to the broader landscape of urban sensing, emergency response, and resilience.http://www.sciencedirect.com/science/article/pii/S2666498424000644Surveillance audioRainfall intensityDatasetRegressionDeep learning |
| spellingShingle | Meizhen Wang Mingzheng Chen Ziran Wang Yuxuan Guo Yong Wu Wei Zhao Xuejun Liu Estimating rainfall intensity based on surveillance audio and deep-learning Environmental Science and Ecotechnology Surveillance audio Rainfall intensity Dataset Regression Deep learning |
| title | Estimating rainfall intensity based on surveillance audio and deep-learning |
| title_full | Estimating rainfall intensity based on surveillance audio and deep-learning |
| title_fullStr | Estimating rainfall intensity based on surveillance audio and deep-learning |
| title_full_unstemmed | Estimating rainfall intensity based on surveillance audio and deep-learning |
| title_short | Estimating rainfall intensity based on surveillance audio and deep-learning |
| title_sort | estimating rainfall intensity based on surveillance audio and deep learning |
| topic | Surveillance audio Rainfall intensity Dataset Regression Deep learning |
| url | http://www.sciencedirect.com/science/article/pii/S2666498424000644 |
| work_keys_str_mv | AT meizhenwang estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT mingzhengchen estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT ziranwang estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT yuxuanguo estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT yongwu estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT weizhao estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning AT xuejunliu estimatingrainfallintensitybasedonsurveillanceaudioanddeeplearning |