Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful compu...
Ausführliche Beschreibung
Autor*in: |
Xianya Fu [verfasserIn] Peixuan Zuo [verfasserIn] Jia Zhai [verfasserIn] Rui Wang [verfasserIn] Hailong Yang [verfasserIn] Depei Qian [verfasserIn] |
---|
Format: |
E-Artikel |
---|---|
Sprache: |
Englisch |
Erschienen: |
2018 |
---|
Schlagwörter: |
---|
Übergeordnetes Werk: |
In: IEEE Access - IEEE, 2014, 6(2018), Seite 58673-58681 |
---|---|
Übergeordnetes Werk: |
volume:6 ; year:2018 ; pages:58673-58681 |
Links: |
---|
DOI / URN: |
10.1109/ACCESS.2018.2872560 |
---|
Katalog-ID: |
DOAJ015283968 |
---|
LEADER | 01000caa a22002652 4500 | ||
---|---|---|---|
001 | DOAJ015283968 | ||
003 | DE-627 | ||
005 | 20230501192401.0 | ||
007 | cr uuu---uuuuu | ||
008 | 230226s2018 xx |||||o 00| ||eng c | ||
024 | 7 | |a 10.1109/ACCESS.2018.2872560 |2 doi | |
035 | |a (DE-627)DOAJ015283968 | ||
035 | |a (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 | ||
040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
041 | |a eng | ||
050 | 0 | |a TK1-9971 | |
100 | 0 | |a Xianya Fu |e verfasserin |4 aut | |
245 | 1 | 0 | |a Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
264 | 1 | |c 2018 | |
336 | |a Text |b txt |2 rdacontent | ||
337 | |a Computermedien |b c |2 rdamedia | ||
338 | |a Online-Ressource |b cr |2 rdacarrier | ||
520 | |a Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. | ||
650 | 4 | |a Deep neural network | |
650 | 4 | |a sparsity | |
650 | 4 | |a matrix decomposition | |
650 | 4 | |a memory footprints | |
653 | 0 | |a Electrical engineering. Electronics. Nuclear engineering | |
700 | 0 | |a Peixuan Zuo |e verfasserin |4 aut | |
700 | 0 | |a Jia Zhai |e verfasserin |4 aut | |
700 | 0 | |a Rui Wang |e verfasserin |4 aut | |
700 | 0 | |a Hailong Yang |e verfasserin |4 aut | |
700 | 0 | |a Depei Qian |e verfasserin |4 aut | |
773 | 0 | 8 | |i In |t IEEE Access |d IEEE, 2014 |g 6(2018), Seite 58673-58681 |w (DE-627)728440385 |w (DE-600)2687964-5 |x 21693536 |7 nnns |
773 | 1 | 8 | |g volume:6 |g year:2018 |g pages:58673-58681 |
856 | 4 | 0 | |u https://doi.org/10.1109/ACCESS.2018.2872560 |z kostenfrei |
856 | 4 | 0 | |u https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 |z kostenfrei |
856 | 4 | 0 | |u https://ieeexplore.ieee.org/document/8478113/ |z kostenfrei |
856 | 4 | 2 | |u https://doaj.org/toc/2169-3536 |y Journal toc |z kostenfrei |
912 | |a GBV_USEFLAG_A | ||
912 | |a SYSFLAG_A | ||
912 | |a GBV_DOAJ | ||
912 | |a SSG-OLC-PHA | ||
912 | |a GBV_ILN_11 | ||
912 | |a GBV_ILN_20 | ||
912 | |a GBV_ILN_22 | ||
912 | |a GBV_ILN_23 | ||
912 | |a GBV_ILN_24 | ||
912 | |a GBV_ILN_31 | ||
912 | |a GBV_ILN_39 | ||
912 | |a GBV_ILN_40 | ||
912 | |a GBV_ILN_60 | ||
912 | |a GBV_ILN_62 | ||
912 | |a GBV_ILN_63 | ||
912 | |a GBV_ILN_65 | ||
912 | |a GBV_ILN_69 | ||
912 | |a GBV_ILN_70 | ||
912 | |a GBV_ILN_73 | ||
912 | |a GBV_ILN_95 | ||
912 | |a GBV_ILN_105 | ||
912 | |a GBV_ILN_110 | ||
912 | |a GBV_ILN_151 | ||
912 | |a GBV_ILN_161 | ||
912 | |a GBV_ILN_170 | ||
912 | |a GBV_ILN_213 | ||
912 | |a GBV_ILN_230 | ||
912 | |a GBV_ILN_285 | ||
912 | |a GBV_ILN_293 | ||
912 | |a GBV_ILN_370 | ||
912 | |a GBV_ILN_602 | ||
912 | |a GBV_ILN_2014 | ||
912 | |a GBV_ILN_4012 | ||
912 | |a GBV_ILN_4037 | ||
912 | |a GBV_ILN_4112 | ||
912 | |a GBV_ILN_4125 | ||
912 | |a GBV_ILN_4126 | ||
912 | |a GBV_ILN_4249 | ||
912 | |a GBV_ILN_4305 | ||
912 | |a GBV_ILN_4306 | ||
912 | |a GBV_ILN_4307 | ||
912 | |a GBV_ILN_4313 | ||
912 | |a GBV_ILN_4322 | ||
912 | |a GBV_ILN_4323 | ||
912 | |a GBV_ILN_4324 | ||
912 | |a GBV_ILN_4325 | ||
912 | |a GBV_ILN_4335 | ||
912 | |a GBV_ILN_4338 | ||
912 | |a GBV_ILN_4367 | ||
912 | |a GBV_ILN_4700 | ||
951 | |a AR | ||
952 | |d 6 |j 2018 |h 58673-58681 |
author_variant |
x f xf p z pz j z jz r w rw h y hy d q dq |
---|---|
matchkey_str |
article:21693536:2018----::prigeperlewruigeiiceea |
hierarchy_sort_str |
2018 |
callnumber-subject-code |
TK |
publishDate |
2018 |
allfields |
10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681 |
spelling |
10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681 |
allfields_unstemmed |
10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681 |
allfieldsGer |
10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681 |
allfieldsSound |
10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681 |
language |
English |
source |
In IEEE Access 6(2018), Seite 58673-58681 volume:6 year:2018 pages:58673-58681 |
sourceStr |
In IEEE Access 6(2018), Seite 58673-58681 volume:6 year:2018 pages:58673-58681 |
format_phy_str_mv |
Article |
institution |
findex.gbv.de |
topic_facet |
Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering |
isfreeaccess_bool |
true |
container_title |
IEEE Access |
authorswithroles_txt_mv |
Xianya Fu @@aut@@ Peixuan Zuo @@aut@@ Jia Zhai @@aut@@ Rui Wang @@aut@@ Hailong Yang @@aut@@ Depei Qian @@aut@@ |
publishDateDaySort_date |
2018-01-01T00:00:00Z |
hierarchy_top_id |
728440385 |
id |
DOAJ015283968 |
language_de |
englisch |
fullrecord |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">DOAJ015283968</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230501192401.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">230226s2018 xx |||||o 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1109/ACCESS.2018.2872560</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)DOAJ015283968</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">TK1-9971</subfield></datafield><datafield tag="100" ind1="0" ind2=" "><subfield code="a">Xianya Fu</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2018</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural network</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">sparsity</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">matrix decomposition</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">memory footprints</subfield></datafield><datafield tag="653" ind1=" " ind2="0"><subfield code="a">Electrical engineering. Electronics. Nuclear engineering</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Peixuan Zuo</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Jia Zhai</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Rui Wang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Hailong Yang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Depei Qian</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">In</subfield><subfield code="t">IEEE Access</subfield><subfield code="d">IEEE, 2014</subfield><subfield code="g">6(2018), Seite 58673-58681</subfield><subfield code="w">(DE-627)728440385</subfield><subfield code="w">(DE-600)2687964-5</subfield><subfield code="x">21693536</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:6</subfield><subfield code="g">year:2018</subfield><subfield code="g">pages:58673-58681</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doi.org/10.1109/ACCESS.2018.2872560</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://ieeexplore.ieee.org/document/8478113/</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="u">https://doaj.org/toc/2169-3536</subfield><subfield code="y">Journal toc</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_DOAJ</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_370</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">6</subfield><subfield code="j">2018</subfield><subfield code="h">58673-58681</subfield></datafield></record></collection>
|
callnumber-first |
T - Technology |
author |
Xianya Fu |
spellingShingle |
Xianya Fu misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
authorStr |
Xianya Fu |
ppnlink_with_tag_str_mv |
@@773@@(DE-627)728440385 |
format |
electronic Article |
delete_txt_mv |
keep |
author_role |
aut aut aut aut aut aut |
collection |
DOAJ |
remote_str |
true |
callnumber-label |
TK1-9971 |
illustrated |
Not Illustrated |
issn |
21693536 |
topic_title |
TK1-9971 Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition Deep neural network sparsity matrix decomposition memory footprints |
topic |
misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering |
topic_unstemmed |
misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering |
topic_browse |
misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering |
format_facet |
Elektronische Aufsätze Aufsätze Elektronische Ressource |
format_main_str_mv |
Text Zeitschrift/Artikel |
carriertype_str_mv |
cr |
hierarchy_parent_title |
IEEE Access |
hierarchy_parent_id |
728440385 |
hierarchy_top_title |
IEEE Access |
isfreeaccess_txt |
true |
familylinks_str_mv |
(DE-627)728440385 (DE-600)2687964-5 |
title |
Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
ctrlnum |
(DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 |
title_full |
Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
author_sort |
Xianya Fu |
journal |
IEEE Access |
journalStr |
IEEE Access |
callnumber-first-code |
T |
lang_code |
eng |
isOA_bool |
true |
recordtype |
marc |
publishDateSort |
2018 |
contenttype_str_mv |
txt |
container_start_page |
58673 |
author_browse |
Xianya Fu Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian |
container_volume |
6 |
class |
TK1-9971 |
format_se |
Elektronische Aufsätze |
author-letter |
Xianya Fu |
doi_str_mv |
10.1109/ACCESS.2018.2872560 |
author2-role |
verfasserin |
title_sort |
sparsing deep neural network using semi-discrete matrix decomposition |
callnumber |
TK1-9971 |
title_auth |
Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
abstract |
Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. |
abstractGer |
Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. |
abstract_unstemmed |
Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. |
collection_details |
GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 |
title_short |
Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition |
url |
https://doi.org/10.1109/ACCESS.2018.2872560 https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 https://ieeexplore.ieee.org/document/8478113/ https://doaj.org/toc/2169-3536 |
remote_bool |
true |
author2 |
Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian |
author2Str |
Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian |
ppnlink |
728440385 |
callnumber-subject |
TK - Electrical and Nuclear Engineering |
mediatype_str_mv |
c |
isOA_txt |
true |
hochschulschrift_bool |
false |
doi_str |
10.1109/ACCESS.2018.2872560 |
callnumber-a |
TK1-9971 |
up_date |
2024-07-03T14:04:15.318Z |
_version_ |
1803566926082867200 |
fullrecord_marcxml |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">DOAJ015283968</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230501192401.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">230226s2018 xx |||||o 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1109/ACCESS.2018.2872560</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)DOAJ015283968</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">TK1-9971</subfield></datafield><datafield tag="100" ind1="0" ind2=" "><subfield code="a">Xianya Fu</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2018</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural network</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">sparsity</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">matrix decomposition</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">memory footprints</subfield></datafield><datafield tag="653" ind1=" " ind2="0"><subfield code="a">Electrical engineering. Electronics. Nuclear engineering</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Peixuan Zuo</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Jia Zhai</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Rui Wang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Hailong Yang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Depei Qian</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">In</subfield><subfield code="t">IEEE Access</subfield><subfield code="d">IEEE, 2014</subfield><subfield code="g">6(2018), Seite 58673-58681</subfield><subfield code="w">(DE-627)728440385</subfield><subfield code="w">(DE-600)2687964-5</subfield><subfield code="x">21693536</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:6</subfield><subfield code="g">year:2018</subfield><subfield code="g">pages:58673-58681</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doi.org/10.1109/ACCESS.2018.2872560</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://ieeexplore.ieee.org/document/8478113/</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="u">https://doaj.org/toc/2169-3536</subfield><subfield code="y">Journal toc</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_DOAJ</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_370</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">6</subfield><subfield code="j">2018</subfield><subfield code="h">58673-58681</subfield></datafield></record></collection>
|
score |
7.4013433 |