Automating multi-task learning on optical neural networks with weight sharing and physical rotation
Abstract The democratization of AI encourages multi-task learning (MTL), demanding more parameters and processing time. To achieve highly energy-efficient MTL, Diffractive Optical Neural Networks (DONNs) have garnered attention due to extremely low energy and high computation speed. However, impleme...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-97262-2 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850172903064076288 |
|---|---|
| author | Shanglin Zhou Yingjie Li Weilu Gao Cunxi Yu Caiwen Ding |
| author_facet | Shanglin Zhou Yingjie Li Weilu Gao Cunxi Yu Caiwen Ding |
| author_sort | Shanglin Zhou |
| collection | DOAJ |
| description | Abstract The democratization of AI encourages multi-task learning (MTL), demanding more parameters and processing time. To achieve highly energy-efficient MTL, Diffractive Optical Neural Networks (DONNs) have garnered attention due to extremely low energy and high computation speed. However, implementing MTL on DONNs requires manually reconfiguring & replacing layers, and rebuilding & duplicating the physical optical systems. To overcome the challenges, we propose LUMEN-PRO, an automated MTL framework using DONNs. We first propose to automate MTL utilizing an arbitrary backbone DONN and a set of tasks, resulting in a high-accuracy multi-task DONN model with small memory footprint that surpasses existing MTL. Second, we leverage the rotability of the physical optical system and replace task-specific layers with rotation of the corresponding shared layers. This replacement eliminates the storage requirement of task-specific layers, further optimizing the memory footprint. LUMEN-PRO provides flexibility in identifying optimal sharing patterns across diverse datasets, facilitating the search for highly energy-efficient DONNs. Experiments show that LUMEN-PRO provides up to 49.58% higher accuracy and 4× better cost efficiency than single-task and existing DONN approaches. It achieves memory lower bound of MTL, with memory efficiency matching single-task models. Compared to IBM-TrueNorth, LUMEN-PRO achieves an $$8.78\times$$ energy efficiency gain, while it matches Nanophotonic in efficiency but surpasses it in per-operator efficiency due to its larger system. |
| format | Article |
| id | doaj-art-2a0d3f6ffd524d02acd49ed7832d325f |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-2a0d3f6ffd524d02acd49ed7832d325f2025-08-20T02:19:58ZengNature PortfolioScientific Reports2045-23222025-04-0115111210.1038/s41598-025-97262-2Automating multi-task learning on optical neural networks with weight sharing and physical rotationShanglin Zhou0Yingjie Li1Weilu Gao2Cunxi Yu3Caiwen Ding4School of Computing, University of ConnecticutA. James Clark School of Engineering, University of MarylandElectrical and Computer Engineering, University of UtahA. James Clark School of Engineering, University of MarylandDepartment of Computer Science & Engineering, University of Minnesota Twin CitiesAbstract The democratization of AI encourages multi-task learning (MTL), demanding more parameters and processing time. To achieve highly energy-efficient MTL, Diffractive Optical Neural Networks (DONNs) have garnered attention due to extremely low energy and high computation speed. However, implementing MTL on DONNs requires manually reconfiguring & replacing layers, and rebuilding & duplicating the physical optical systems. To overcome the challenges, we propose LUMEN-PRO, an automated MTL framework using DONNs. We first propose to automate MTL utilizing an arbitrary backbone DONN and a set of tasks, resulting in a high-accuracy multi-task DONN model with small memory footprint that surpasses existing MTL. Second, we leverage the rotability of the physical optical system and replace task-specific layers with rotation of the corresponding shared layers. This replacement eliminates the storage requirement of task-specific layers, further optimizing the memory footprint. LUMEN-PRO provides flexibility in identifying optimal sharing patterns across diverse datasets, facilitating the search for highly energy-efficient DONNs. Experiments show that LUMEN-PRO provides up to 49.58% higher accuracy and 4× better cost efficiency than single-task and existing DONN approaches. It achieves memory lower bound of MTL, with memory efficiency matching single-task models. Compared to IBM-TrueNorth, LUMEN-PRO achieves an $$8.78\times$$ energy efficiency gain, while it matches Nanophotonic in efficiency but surpasses it in per-operator efficiency due to its larger system.https://doi.org/10.1038/s41598-025-97262-2 |
| spellingShingle | Shanglin Zhou Yingjie Li Weilu Gao Cunxi Yu Caiwen Ding Automating multi-task learning on optical neural networks with weight sharing and physical rotation Scientific Reports |
| title | Automating multi-task learning on optical neural networks with weight sharing and physical rotation |
| title_full | Automating multi-task learning on optical neural networks with weight sharing and physical rotation |
| title_fullStr | Automating multi-task learning on optical neural networks with weight sharing and physical rotation |
| title_full_unstemmed | Automating multi-task learning on optical neural networks with weight sharing and physical rotation |
| title_short | Automating multi-task learning on optical neural networks with weight sharing and physical rotation |
| title_sort | automating multi task learning on optical neural networks with weight sharing and physical rotation |
| url | https://doi.org/10.1038/s41598-025-97262-2 |
| work_keys_str_mv | AT shanglinzhou automatingmultitasklearningonopticalneuralnetworkswithweightsharingandphysicalrotation AT yingjieli automatingmultitasklearningonopticalneuralnetworkswithweightsharingandphysicalrotation AT weilugao automatingmultitasklearningonopticalneuralnetworkswithweightsharingandphysicalrotation AT cunxiyu automatingmultitasklearningonopticalneuralnetworkswithweightsharingandphysicalrotation AT caiwending automatingmultitasklearningonopticalneuralnetworkswithweightsharingandphysicalrotation |