Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor
Abstract The Event‐based Vision Sensor (EVS) is a bio‐inspired sensor that captures detailed motions of objects, aiming to become the ‘eyes’ of machines like self‐driving cars. Compared to conventional frame‐based image sensors, the EVS has an extremely fast motion capture equivalent to 10,000‐fps e...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-08-01
|
| Series: | Ecology and Evolution |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/ece3.70150 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850259589110431744 |
|---|---|
| author | Susumu Takatsuka Norio Miyamoto Hidehito Sato Yoshiaki Morino Yoshihisa Kurita Akinori Yabuki Chong Chen Shinsuke Kawagucci |
| author_facet | Susumu Takatsuka Norio Miyamoto Hidehito Sato Yoshiaki Morino Yoshihisa Kurita Akinori Yabuki Chong Chen Shinsuke Kawagucci |
| author_sort | Susumu Takatsuka |
| collection | DOAJ |
| description | Abstract The Event‐based Vision Sensor (EVS) is a bio‐inspired sensor that captures detailed motions of objects, aiming to become the ‘eyes’ of machines like self‐driving cars. Compared to conventional frame‐based image sensors, the EVS has an extremely fast motion capture equivalent to 10,000‐fps even with standard optical settings, plus high dynamic ranges for brightness and also lower consumption of memory and energy. Here, we developed 22 characteristic features for analysing the motions of aquatic particles from the EVS raw data and tested the applicability of the EVS in analysing plankton behaviour. Laboratory cultures of six species of zooplankton and phytoplankton were observed, confirming species‐specific motion periodicities up to 41 Hz. We applied machine learning to automatically classify particles into four categories of zooplankton and passive particles, achieving an accuracy up to 86%. At the in situ deployment of the EVS at the bottom of Lake Biwa, several particles exhibiting distinct cumulative trajectory with periodicities in their motion (up to 16 Hz) were identified, suggesting that they were living organisms with rhythmic behaviour. We also used the EVS in the deep sea, observing particles with active motion and periodicities over 40 Hz. Our application of the EVS, especially focusing on its millisecond‐scale temporal resolution and wide dynamic range, provides a new avenue to investigate organismal behaviour characterised by rapid and periodical motions. The EVS will likely be applicable in the near future for the automated monitoring of plankton behaviour by edge computing on autonomous floats, as well as quantifying rapid cellular‐level activities under microscopy. |
| format | Article |
| id | doaj-art-91612a2be7534bfb949a23de360eaa4c |
| institution | OA Journals |
| issn | 2045-7758 |
| language | English |
| publishDate | 2024-08-01 |
| publisher | Wiley |
| record_format | Article |
| series | Ecology and Evolution |
| spelling | doaj-art-91612a2be7534bfb949a23de360eaa4c2025-08-20T01:55:49ZengWileyEcology and Evolution2045-77582024-08-01148n/an/a10.1002/ece3.70150Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision SensorSusumu Takatsuka0Norio Miyamoto1Hidehito Sato2Yoshiaki Morino3Yoshihisa Kurita4Akinori Yabuki5Chong Chen6Shinsuke Kawagucci7Sony Group Corporation Minato‐ku JapanSuper‐Cutting‐Edge Grand and Advanced Research (SUGAR) Program, Institute for Extra‐Cutting‐Edge Science and Technology Avant‐Garde Research (X‐STAR) Japan Agency for Marine‐Earth Science and Technology (JAMSTEC) Yokosuka Kanagawa JapanSony Group Corporation Minato‐ku JapanInstitute of Life and Environmental Sciences University of Tsukuba Tsukuba Ibaraki JapanFishery Research Laboratory Kyushu University Fukutsu Fukuoka JapanMarine Biodiversity and Environmental Assessment Research Center (BioEnv), Research Institute for Global Change (RIGC) Japan Agency for Marine‐Earth Science and Technology (JAMSTEC) Yokosuka Kanagawa JapanSuper‐Cutting‐Edge Grand and Advanced Research (SUGAR) Program, Institute for Extra‐Cutting‐Edge Science and Technology Avant‐Garde Research (X‐STAR) Japan Agency for Marine‐Earth Science and Technology (JAMSTEC) Yokosuka Kanagawa JapanSuper‐Cutting‐Edge Grand and Advanced Research (SUGAR) Program, Institute for Extra‐Cutting‐Edge Science and Technology Avant‐Garde Research (X‐STAR) Japan Agency for Marine‐Earth Science and Technology (JAMSTEC) Yokosuka Kanagawa JapanAbstract The Event‐based Vision Sensor (EVS) is a bio‐inspired sensor that captures detailed motions of objects, aiming to become the ‘eyes’ of machines like self‐driving cars. Compared to conventional frame‐based image sensors, the EVS has an extremely fast motion capture equivalent to 10,000‐fps even with standard optical settings, plus high dynamic ranges for brightness and also lower consumption of memory and energy. Here, we developed 22 characteristic features for analysing the motions of aquatic particles from the EVS raw data and tested the applicability of the EVS in analysing plankton behaviour. Laboratory cultures of six species of zooplankton and phytoplankton were observed, confirming species‐specific motion periodicities up to 41 Hz. We applied machine learning to automatically classify particles into four categories of zooplankton and passive particles, achieving an accuracy up to 86%. At the in situ deployment of the EVS at the bottom of Lake Biwa, several particles exhibiting distinct cumulative trajectory with periodicities in their motion (up to 16 Hz) were identified, suggesting that they were living organisms with rhythmic behaviour. We also used the EVS in the deep sea, observing particles with active motion and periodicities over 40 Hz. Our application of the EVS, especially focusing on its millisecond‐scale temporal resolution and wide dynamic range, provides a new avenue to investigate organismal behaviour characterised by rapid and periodical motions. The EVS will likely be applicable in the near future for the automated monitoring of plankton behaviour by edge computing on autonomous floats, as well as quantifying rapid cellular‐level activities under microscopy.https://doi.org/10.1002/ece3.70150computer visiondeep seadynamic vision sensorevent camerahigh‐speed cameramarine particles |
| spellingShingle | Susumu Takatsuka Norio Miyamoto Hidehito Sato Yoshiaki Morino Yoshihisa Kurita Akinori Yabuki Chong Chen Shinsuke Kawagucci Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor Ecology and Evolution computer vision deep sea dynamic vision sensor event camera high‐speed camera marine particles |
| title | Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor |
| title_full | Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor |
| title_fullStr | Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor |
| title_full_unstemmed | Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor |
| title_short | Millisecond‐scale behaviours of plankton quantified in vitro and in situ using the Event‐based Vision Sensor |
| title_sort | millisecond scale behaviours of plankton quantified in vitro and in situ using the event based vision sensor |
| topic | computer vision deep sea dynamic vision sensor event camera high‐speed camera marine particles |
| url | https://doi.org/10.1002/ece3.70150 |
| work_keys_str_mv | AT susumutakatsuka millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT noriomiyamoto millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT hidehitosato millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT yoshiakimorino millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT yoshihisakurita millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT akinoriyabuki millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT chongchen millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor AT shinsukekawagucci millisecondscalebehavioursofplanktonquantifiedinvitroandinsituusingtheeventbasedvisionsensor |