Noise-resilient designs and analysis for optical neural networks
All analog signal processing is fundamentally subject to noise, and this is also the case in next generation implementations of optical neural networks (ONNs). Therefore, we propose the first hardware-based approach to mitigate noise in ONNs. A tree-like and an accordion-like design are constructed...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IOP Publishing
2024-01-01
|
| Series: | Neuromorphic Computing and Engineering |
| Subjects: | |
| Online Access: | https://doi.org/10.1088/2634-4386/ad836f |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832582520591351808 |
|---|---|
| author | Gianluca Kosmella Ripalta Stabile Jaron Sanders |
| author_facet | Gianluca Kosmella Ripalta Stabile Jaron Sanders |
| author_sort | Gianluca Kosmella |
| collection | DOAJ |
| description | All analog signal processing is fundamentally subject to noise, and this is also the case in next generation implementations of optical neural networks (ONNs). Therefore, we propose the first hardware-based approach to mitigate noise in ONNs. A tree-like and an accordion-like design are constructed from a given NN that one wishes to implement. Both designs have the capability that the resulting ONNs gives outputs close to the desired solution. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the tree-like design that establishes the correctness of the design, i.e. for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the tree-like design thus also inherit the universal approximation property of NNs. For the accordion-like design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs’ output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. The results indicate that adding just a few components and/or adding them only in the first (few) layers in the manner of either design can already be expected to increase the accuracy of ONNs considerably. To illustrate the effect we point to a specific simulation of a LeNet implementation, in which adding one copy of the layers components in each layer reduces the mean-squared error (MSE) by 59.1% for the tree-like design and by 51.5% for the accordion-like design. In this scenario, the gap in accuracy of prediction between the noiseless NN and the ONNs reduces even more: 93.3% for the tree-like design and 80% for the accordion-like design. |
| format | Article |
| id | doaj-art-7e714cec3a95467f92650e9279ce6f08 |
| institution | Kabale University |
| issn | 2634-4386 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| series | Neuromorphic Computing and Engineering |
| spelling | doaj-art-7e714cec3a95467f92650e9279ce6f082025-01-29T16:06:29ZengIOP PublishingNeuromorphic Computing and Engineering2634-43862024-01-014404400210.1088/2634-4386/ad836fNoise-resilient designs and analysis for optical neural networksGianluca Kosmella0https://orcid.org/0009-0008-7543-8484Ripalta Stabile1https://orcid.org/0000-0001-5197-3150Jaron Sanders2https://orcid.org/0000-0003-0187-2065Department of Electrical Engineering, Eindhoven University of Technology , Eindhoven, The Netherlands; Department of Mathematics & Computer Science, Eindhoven University of Technology , Eindhoven, The NetherlandsDepartment of Electrical Engineering, Eindhoven University of Technology , Eindhoven, The NetherlandsDepartment of Mathematics & Computer Science, Eindhoven University of Technology , Eindhoven, The NetherlandsAll analog signal processing is fundamentally subject to noise, and this is also the case in next generation implementations of optical neural networks (ONNs). Therefore, we propose the first hardware-based approach to mitigate noise in ONNs. A tree-like and an accordion-like design are constructed from a given NN that one wishes to implement. Both designs have the capability that the resulting ONNs gives outputs close to the desired solution. To establish the latter, we analyze the designs mathematically. Specifically, we investigate a probabilistic framework for the tree-like design that establishes the correctness of the design, i.e. for any feed-forward NN with Lipschitz continuous activation functions, an ONN can be constructed that produces output arbitrarily close to the original. ONNs constructed with the tree-like design thus also inherit the universal approximation property of NNs. For the accordion-like design, we restrict the analysis to NNs with linear activation functions and characterize the ONNs’ output distribution using exact formulas. Finally, we report on numerical experiments with LeNet ONNs that give insight into the number of components required in these designs for certain accuracy gains. The results indicate that adding just a few components and/or adding them only in the first (few) layers in the manner of either design can already be expected to increase the accuracy of ONNs considerably. To illustrate the effect we point to a specific simulation of a LeNet implementation, in which adding one copy of the layers components in each layer reduces the mean-squared error (MSE) by 59.1% for the tree-like design and by 51.5% for the accordion-like design. In this scenario, the gap in accuracy of prediction between the noiseless NN and the ONNs reduces even more: 93.3% for the tree-like design and 80% for the accordion-like design.https://doi.org/10.1088/2634-4386/ad836fnoise-resilient designsoptical neural networksuniversal approximationlaw of large numbers |
| spellingShingle | Gianluca Kosmella Ripalta Stabile Jaron Sanders Noise-resilient designs and analysis for optical neural networks Neuromorphic Computing and Engineering noise-resilient designs optical neural networks universal approximation law of large numbers |
| title | Noise-resilient designs and analysis for optical neural networks |
| title_full | Noise-resilient designs and analysis for optical neural networks |
| title_fullStr | Noise-resilient designs and analysis for optical neural networks |
| title_full_unstemmed | Noise-resilient designs and analysis for optical neural networks |
| title_short | Noise-resilient designs and analysis for optical neural networks |
| title_sort | noise resilient designs and analysis for optical neural networks |
| topic | noise-resilient designs optical neural networks universal approximation law of large numbers |
| url | https://doi.org/10.1088/2634-4386/ad836f |
| work_keys_str_mv | AT gianlucakosmella noiseresilientdesignsandanalysisforopticalneuralnetworks AT ripaltastabile noiseresilientdesignsandanalysisforopticalneuralnetworks AT jaronsanders noiseresilientdesignsandanalysisforopticalneuralnetworks |