Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning

Neuroevolution is the field of study that uses evolutionary computation in order to optimize certain aspect of the design of neural networks, most often its topology and hyperparameters. The field was introduced in the late-1980s, but only in the latest years the field has become mature enough to en...

Full description

Saved in:
Bibliographic Details
Main Authors: Alejandro Baldominos, Yago Saez, Pedro Isasi
Format: Article
Language:English
Published: Wiley 2019-01-01
Series:Complexity
Online Access:http://dx.doi.org/10.1155/2019/2952304
Tags: Add Tag
No Tags, Be the first to tag this record!
_version_ 1832560342959390720
author Alejandro Baldominos
Yago Saez
Pedro Isasi
author_facet Alejandro Baldominos
Yago Saez
Pedro Isasi
author_sort Alejandro Baldominos
collection DOAJ
description Neuroevolution is the field of study that uses evolutionary computation in order to optimize certain aspect of the design of neural networks, most often its topology and hyperparameters. The field was introduced in the late-1980s, but only in the latest years the field has become mature enough to enable the optimization of deep learning models, such as convolutional neural networks. In this paper, we rely on previous work to apply neuroevolution in order to optimize the topology of deep neural networks that can be used to solve the problem of handwritten character recognition. Moreover, we take advantage of the fact that evolutionary algorithms optimize a population of candidate solutions, by combining a set of the best evolved models resulting in a committee of convolutional neural networks. This process is enhanced by using specific mechanisms to preserve the diversity of the population. Additionally, in this paper, we address one of the disadvantages of neuroevolution: the process is very expensive in terms of computational time. To lessen this issue, we explore the performance of topology transfer learning: whether the best topology obtained using neuroevolution for a certain domain can be successfully applied to a different domain. By doing so, the expensive process of neuroevolution can be reused to tackle different problems, turning it into a more appealing approach for optimizing the design of neural networks topologies. After evaluating our proposal, results show that both the use of neuroevolved committees and the application of topology transfer learning are successful: committees of convolutional neural networks are able to improve classification results when compared to single models, and topologies learned for one problem can be reused for a different problem and data with a good performance. Additionally, both approaches can be combined by building committees of transferred topologies, and this combination attains results that combine the best of both approaches.
format Article
id doaj-art-bde389c794fa4346949323f67724c106
institution Kabale University
issn 1076-2787
1099-0526
language English
publishDate 2019-01-01
publisher Wiley
record_format Article
series Complexity
spelling doaj-art-bde389c794fa4346949323f67724c1062025-02-03T01:27:48ZengWileyComplexity1076-27871099-05262019-01-01201910.1155/2019/29523042952304Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer LearningAlejandro Baldominos0Yago Saez1Pedro Isasi2Computer Science Department, Universidad Carlos III de Madrid. Avenida de la Universidad, 30. 28911 Leganes, Madrid, SpainComputer Science Department, Universidad Carlos III de Madrid. Avenida de la Universidad, 30. 28911 Leganes, Madrid, SpainComputer Science Department, Universidad Carlos III de Madrid. Avenida de la Universidad, 30. 28911 Leganes, Madrid, SpainNeuroevolution is the field of study that uses evolutionary computation in order to optimize certain aspect of the design of neural networks, most often its topology and hyperparameters. The field was introduced in the late-1980s, but only in the latest years the field has become mature enough to enable the optimization of deep learning models, such as convolutional neural networks. In this paper, we rely on previous work to apply neuroevolution in order to optimize the topology of deep neural networks that can be used to solve the problem of handwritten character recognition. Moreover, we take advantage of the fact that evolutionary algorithms optimize a population of candidate solutions, by combining a set of the best evolved models resulting in a committee of convolutional neural networks. This process is enhanced by using specific mechanisms to preserve the diversity of the population. Additionally, in this paper, we address one of the disadvantages of neuroevolution: the process is very expensive in terms of computational time. To lessen this issue, we explore the performance of topology transfer learning: whether the best topology obtained using neuroevolution for a certain domain can be successfully applied to a different domain. By doing so, the expensive process of neuroevolution can be reused to tackle different problems, turning it into a more appealing approach for optimizing the design of neural networks topologies. After evaluating our proposal, results show that both the use of neuroevolved committees and the application of topology transfer learning are successful: committees of convolutional neural networks are able to improve classification results when compared to single models, and topologies learned for one problem can be reused for a different problem and data with a good performance. Additionally, both approaches can be combined by building committees of transferred topologies, and this combination attains results that combine the best of both approaches.http://dx.doi.org/10.1155/2019/2952304
spellingShingle Alejandro Baldominos
Yago Saez
Pedro Isasi
Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
Complexity
title Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
title_full Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
title_fullStr Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
title_full_unstemmed Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
title_short Hybridizing Evolutionary Computation and Deep Neural Networks: An Approach to Handwriting Recognition Using Committees and Transfer Learning
title_sort hybridizing evolutionary computation and deep neural networks an approach to handwriting recognition using committees and transfer learning
url http://dx.doi.org/10.1155/2019/2952304
work_keys_str_mv AT alejandrobaldominos hybridizingevolutionarycomputationanddeepneuralnetworksanapproachtohandwritingrecognitionusingcommitteesandtransferlearning
AT yagosaez hybridizingevolutionarycomputationanddeepneuralnetworksanapproachtohandwritingrecognitionusingcommitteesandtransferlearning
AT pedroisasi hybridizingevolutionarycomputationanddeepneuralnetworksanapproachtohandwritingrecognitionusingcommitteesandtransferlearning