Improving deep convolutional neural networks with mixed maxout units
The maxout units have the problem of not delivering non-max features， resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings,when they are applied in deep convolutional neural networks.The mixed maxout (mixout) units were proposed to de...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Editorial Department of Journal on Communications
2017-07-01
|
| Series: | Tongxin xuebao |
| Subjects: | |
| Online Access: | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2017145/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841539541718007808 |
|---|---|
| author | Hui-zhen ZHAO Fu-xian LIU Long-yue LI Chang LUO |
| author_facet | Hui-zhen ZHAO Fu-xian LIU Long-yue LI Chang LUO |
| author_sort | Hui-zhen ZHAO |
| collection | DOAJ |
| description | The maxout units have the problem of not delivering non-max features， resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings,when they are applied in deep convolutional neural networks.The mixed maxout (mixout) units were proposed to deal with this constrain.Firstly,the exponential probability of the feature mappings getting from different linear transformations was computed.Then,the averaging of a subspace of different feature mappings by the exponential probability was computed.Finally,the output was randomly sampled from the max feature and the mean value by the Bernoulli distribution,leading to the better utilizing of model averaging ability of dropout.The simple models and network in network models was built to evaluate the performance of mixout units.The results show that mixout units based models have better performance. |
| format | Article |
| id | doaj-art-de53903a587343e0b2e516fec2fdd935 |
| institution | Kabale University |
| issn | 1000-436X |
| language | zho |
| publishDate | 2017-07-01 |
| publisher | Editorial Department of Journal on Communications |
| record_format | Article |
| series | Tongxin xuebao |
| spelling | doaj-art-de53903a587343e0b2e516fec2fdd9352025-01-14T07:12:36ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2017-07-013810511459710990Improving deep convolutional neural networks with mixed maxout unitsHui-zhen ZHAOFu-xian LIULong-yue LIChang LUOThe maxout units have the problem of not delivering non-max features， resulting in the insufficient of pooling operation over a subspace that is composed of several linear feature mappings,when they are applied in deep convolutional neural networks.The mixed maxout (mixout) units were proposed to deal with this constrain.Firstly,the exponential probability of the feature mappings getting from different linear transformations was computed.Then,the averaging of a subspace of different feature mappings by the exponential probability was computed.Finally,the output was randomly sampled from the max feature and the mean value by the Bernoulli distribution,leading to the better utilizing of model averaging ability of dropout.The simple models and network in network models was built to evaluate the performance of mixout units.The results show that mixout units based models have better performance.http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2017145/deep learningconvolutional neural networkmaxout unitsactivation function |
| spellingShingle | Hui-zhen ZHAO Fu-xian LIU Long-yue LI Chang LUO Improving deep convolutional neural networks with mixed maxout units Tongxin xuebao deep learning convolutional neural network maxout units activation function |
| title | Improving deep convolutional neural networks with mixed maxout units |
| title_full | Improving deep convolutional neural networks with mixed maxout units |
| title_fullStr | Improving deep convolutional neural networks with mixed maxout units |
| title_full_unstemmed | Improving deep convolutional neural networks with mixed maxout units |
| title_short | Improving deep convolutional neural networks with mixed maxout units |
| title_sort | improving deep convolutional neural networks with mixed maxout units |
| topic | deep learning convolutional neural network maxout units activation function |
| url | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2017145/ |
| work_keys_str_mv | AT huizhenzhao improvingdeepconvolutionalneuralnetworkswithmixedmaxoutunits AT fuxianliu improvingdeepconvolutionalneuralnetworkswithmixedmaxoutunits AT longyueli improvingdeepconvolutionalneuralnetworkswithmixedmaxoutunits AT changluo improvingdeepconvolutionalneuralnetworkswithmixedmaxoutunits |