Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition

Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful compu...
Ausführliche Beschreibung

Gespeichert in:

Autor*in:	Xianya Fu [verfasserIn] Peixuan Zuo [verfasserIn] Jia Zhai [verfasserIn] Rui Wang [verfasserIn] Hailong Yang [verfasserIn] Depei Qian [verfasserIn]

Format:	E-Artikel
Sprache:	Englisch

Erschienen:	2018

Schlagwörter:	Deep neural network sparsity matrix decomposition memory footprints

Übergeordnetes Werk:	In: IEEE Access - IEEE, 2014, 6(2018), Seite 58673-58681
Übergeordnetes Werk:	volume:6 ; year:2018 ; pages:58673-58681

Links:	Link aufrufen Link aufrufen Link aufrufen Journal toc

DOI / URN:	10.1109/ACCESS.2018.2872560

Katalog-ID:	DOAJ015283968

Internformat


LEADER	01000caa a22002652 4500
001	DOAJ015283968
003	DE-627
005	20230501192401.0
007	cr uuu---uuuuu
008	230226s2018 xx \|\|\|\|\|o 00\| \|\|eng c
024	7		\|a 10.1109/ACCESS.2018.2872560 \|2 doi
035			\|a (DE-627)DOAJ015283968
035			\|a (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551
040			\|a DE-627 \|b ger \|c DE-627 \|e rakwb
041			\|a eng
050		0	\|a TK1-9971
100	0		\|a Xianya Fu \|e verfasserin \|4 aut
245	1	0	\|a Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
264		1	\|c 2018
336			\|a Text \|b txt \|2 rdacontent
337			\|a Computermedien \|b c \|2 rdamedia
338			\|a Online-Ressource \|b cr \|2 rdacarrier
520			\|a Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.
650		4	\|a Deep neural network
650		4	\|a sparsity
650		4	\|a matrix decomposition
650		4	\|a memory footprints
653		0	\|a Electrical engineering. Electronics. Nuclear engineering
700	0		\|a Peixuan Zuo \|e verfasserin \|4 aut
700	0		\|a Jia Zhai \|e verfasserin \|4 aut
700	0		\|a Rui Wang \|e verfasserin \|4 aut
700	0		\|a Hailong Yang \|e verfasserin \|4 aut
700	0		\|a Depei Qian \|e verfasserin \|4 aut
773	0	8	\|i In \|t IEEE Access \|d IEEE, 2014 \|g 6(2018), Seite 58673-58681 \|w (DE-627)728440385 \|w (DE-600)2687964-5 \|x 21693536 \|7 nnns
773	1	8	\|g volume:6 \|g year:2018 \|g pages:58673-58681
856	4	0	\|u https://doi.org/10.1109/ACCESS.2018.2872560 \|z kostenfrei
856	4	0	\|u https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 \|z kostenfrei
856	4	0	\|u https://ieeexplore.ieee.org/document/8478113/ \|z kostenfrei
856	4	2	\|u https://doaj.org/toc/2169-3536 \|y Journal toc \|z kostenfrei
912			\|a GBV_USEFLAG_A
912			\|a SYSFLAG_A
912			\|a GBV_DOAJ
912			\|a SSG-OLC-PHA
912			\|a GBV_ILN_11
912			\|a GBV_ILN_20
912			\|a GBV_ILN_22
912			\|a GBV_ILN_23
912			\|a GBV_ILN_24
912			\|a GBV_ILN_31
912			\|a GBV_ILN_39
912			\|a GBV_ILN_40
912			\|a GBV_ILN_60
912			\|a GBV_ILN_62
912			\|a GBV_ILN_63
912			\|a GBV_ILN_65
912			\|a GBV_ILN_69
912			\|a GBV_ILN_70
912			\|a GBV_ILN_73
912			\|a GBV_ILN_95
912			\|a GBV_ILN_105
912			\|a GBV_ILN_110
912			\|a GBV_ILN_151
912			\|a GBV_ILN_161
912			\|a GBV_ILN_170
912			\|a GBV_ILN_213
912			\|a GBV_ILN_230
912			\|a GBV_ILN_285
912			\|a GBV_ILN_293
912			\|a GBV_ILN_370
912			\|a GBV_ILN_602
912			\|a GBV_ILN_2014
912			\|a GBV_ILN_4012
912			\|a GBV_ILN_4037
912			\|a GBV_ILN_4112
912			\|a GBV_ILN_4125
912			\|a GBV_ILN_4126
912			\|a GBV_ILN_4249
912			\|a GBV_ILN_4305
912			\|a GBV_ILN_4306
912			\|a GBV_ILN_4307
912			\|a GBV_ILN_4313
912			\|a GBV_ILN_4322
912			\|a GBV_ILN_4323
912			\|a GBV_ILN_4324
912			\|a GBV_ILN_4325
912			\|a GBV_ILN_4335
912			\|a GBV_ILN_4338
912			\|a GBV_ILN_4367
912			\|a GBV_ILN_4700
951			\|a AR
952			\|d 6 \|j 2018 \|h 58673-58681

Indexfelder

author_variant	x f xf p z pz j z jz r w rw h y hy d q dq
matchkey_str	article:21693536:2018----::prigeperlewruigeiiceea
hierarchy_sort_str	2018
callnumber-subject-code	TK
publishDate	2018
allfields	10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681
spelling	10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681
allfields_unstemmed	10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681
allfieldsGer	10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681
allfieldsSound	10.1109/ACCESS.2018.2872560 doi (DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551 DE-627 ger DE-627 rakwb eng TK1-9971 Xianya Fu verfasserin aut Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition 2018 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<. Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering Peixuan Zuo verfasserin aut Jia Zhai verfasserin aut Rui Wang verfasserin aut Hailong Yang verfasserin aut Depei Qian verfasserin aut In IEEE Access IEEE, 2014 6(2018), Seite 58673-58681 (DE-627)728440385 (DE-600)2687964-5 21693536 nnns volume:6 year:2018 pages:58673-58681 https://doi.org/10.1109/ACCESS.2018.2872560 kostenfrei https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 kostenfrei https://ieeexplore.ieee.org/document/8478113/ kostenfrei https://doaj.org/toc/2169-3536 Journal toc kostenfrei GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 6 2018 58673-58681
language	English
source	In IEEE Access 6(2018), Seite 58673-58681 volume:6 year:2018 pages:58673-58681
sourceStr	In IEEE Access 6(2018), Seite 58673-58681 volume:6 year:2018 pages:58673-58681
format_phy_str_mv	Article
institution	findex.gbv.de
topic_facet	Deep neural network sparsity matrix decomposition memory footprints Electrical engineering. Electronics. Nuclear engineering
isfreeaccess_bool	true
container_title	IEEE Access
authorswithroles_txt_mv	Xianya Fu @@aut@@ Peixuan Zuo @@aut@@ Jia Zhai @@aut@@ Rui Wang @@aut@@ Hailong Yang @@aut@@ Depei Qian @@aut@@
publishDateDaySort_date	2018-01-01T00:00:00Z
hierarchy_top_id	728440385
id	DOAJ015283968
language_de	englisch
fullrecord	<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">DOAJ015283968</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230501192401.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">230226s2018 xx \|\|\|\|\|o 00\| \|\|eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1109/ACCESS.2018.2872560</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)DOAJ015283968</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">TK1-9971</subfield></datafield><datafield tag="100" ind1="0" ind2=" "><subfield code="a">Xianya Fu</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2018</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural network</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">sparsity</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">matrix decomposition</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">memory footprints</subfield></datafield><datafield tag="653" ind1=" " ind2="0"><subfield code="a">Electrical engineering. Electronics. Nuclear engineering</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Peixuan Zuo</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Jia Zhai</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Rui Wang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Hailong Yang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Depei Qian</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">In</subfield><subfield code="t">IEEE Access</subfield><subfield code="d">IEEE, 2014</subfield><subfield code="g">6(2018), Seite 58673-58681</subfield><subfield code="w">(DE-627)728440385</subfield><subfield code="w">(DE-600)2687964-5</subfield><subfield code="x">21693536</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:6</subfield><subfield code="g">year:2018</subfield><subfield code="g">pages:58673-58681</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doi.org/10.1109/ACCESS.2018.2872560</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://ieeexplore.ieee.org/document/8478113/</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="u">https://doaj.org/toc/2169-3536</subfield><subfield code="y">Journal toc</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_DOAJ</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_370</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">6</subfield><subfield code="j">2018</subfield><subfield code="h">58673-58681</subfield></datafield></record></collection>
callnumber-first	T - Technology
author	Xianya Fu
spellingShingle	Xianya Fu misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
authorStr	Xianya Fu
ppnlink_with_tag_str_mv	@@773@@(DE-627)728440385
format	electronic Article
delete_txt_mv	keep
author_role	aut aut aut aut aut aut
collection	DOAJ
remote_str	true
callnumber-label	TK1-9971
illustrated	Not Illustrated
issn	21693536
topic_title	TK1-9971 Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition Deep neural network sparsity matrix decomposition memory footprints
topic	misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering
topic_unstemmed	misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering
topic_browse	misc TK1-9971 misc Deep neural network misc sparsity misc matrix decomposition misc memory footprints misc Electrical engineering. Electronics. Nuclear engineering
format_facet	Elektronische Aufsätze Aufsätze Elektronische Ressource
format_main_str_mv	Text Zeitschrift/Artikel
carriertype_str_mv	cr
hierarchy_parent_title	IEEE Access
hierarchy_parent_id	728440385
hierarchy_top_title	IEEE Access
isfreeaccess_txt	true
familylinks_str_mv	(DE-627)728440385 (DE-600)2687964-5
title	Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
ctrlnum	(DE-627)DOAJ015283968 (DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551
title_full	Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
author_sort	Xianya Fu
journal	IEEE Access
journalStr	IEEE Access
callnumber-first-code	T
lang_code	eng
isOA_bool	true
recordtype	marc
publishDateSort	2018
contenttype_str_mv	txt
container_start_page	58673
author_browse	Xianya Fu Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian
container_volume	6
class	TK1-9971
format_se	Elektronische Aufsätze
author-letter	Xianya Fu
doi_str_mv	10.1109/ACCESS.2018.2872560
author2-role	verfasserin
title_sort	sparsing deep neural network using semi-discrete matrix decomposition
callnumber	TK1-9971
title_auth	Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
abstract	Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.
abstractGer	Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.
abstract_unstemmed	Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.
collection_details	GBV_USEFLAG_A SYSFLAG_A GBV_DOAJ SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_370 GBV_ILN_602 GBV_ILN_2014 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700
title_short	Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition
url	https://doi.org/10.1109/ACCESS.2018.2872560 https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551 https://ieeexplore.ieee.org/document/8478113/ https://doaj.org/toc/2169-3536
remote_bool	true
author2	Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian
author2Str	Peixuan Zuo Jia Zhai Rui Wang Hailong Yang Depei Qian
ppnlink	728440385
callnumber-subject	TK - Electrical and Nuclear Engineering
mediatype_str_mv	c
isOA_txt	true
hochschulschrift_bool	false
doi_str	10.1109/ACCESS.2018.2872560
callnumber-a	TK1-9971
up_date	2024-07-03T14:04:15.318Z
_version_	1803566926082867200
fullrecord_marcxml	<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">DOAJ015283968</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230501192401.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">230226s2018 xx \|\|\|\|\|o 00\| \|\|eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1109/ACCESS.2018.2872560</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)DOAJ015283968</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-599)DOAJ9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="050" ind1=" " ind2="0"><subfield code="a">TK1-9971</subfield></datafield><datafield tag="100" ind1="0" ind2=" "><subfield code="a">Xianya Fu</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2018</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Deep learning has gained a lot of successes in various areas, including computer vision, natural language process, and robot control. Convolution neural network (CNN) is the most commonly used model in deep neural networks. Despite their effectiveness on feature abstraction, CNNs need powerful computation even in the inference stage, which becomes a major obstacle in their deployment in embedded and mobile devices. In order to solve this problem, we 1) propose to make decomposition on convolution layers and full connected layers in CNNs with naïve semi-discrete matrix decomposition (SDD), which achieves the low-rank decomposition and parameters sparse at the same time; and 2) we propose a layer-merging scheme which merges two out of all the three result matrices, which can avoid the explode of the intermediate data come with the naïve semi-discrete matrix decomposition; 3) we propose a progressive training strategy to speed up the converging. We implement this optimized method in image classification and object detection networks. Under the loss of network accuracy by 1%, we achieve significant running time and model size reduction. The full-connected layer of the LeNet network achieves <inline-formula< <tex-math notation="LaTeX"<$7\times $ </tex-math<</inline-formula< speedup in the inference stage. In the Faster-Rcnn, the weight parameters are reduced by the factor of <inline-formula< <tex-math notation="LaTeX"<$5.85\times $ </tex-math<</inline-formula<, and it can have a speedup by the factor of <inline-formula< <tex-math notation="LaTeX"<$1.75\times $ </tex-math<</inline-formula<.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural network</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">sparsity</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">matrix decomposition</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">memory footprints</subfield></datafield><datafield tag="653" ind1=" " ind2="0"><subfield code="a">Electrical engineering. Electronics. Nuclear engineering</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Peixuan Zuo</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Jia Zhai</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Rui Wang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Hailong Yang</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="0" ind2=" "><subfield code="a">Depei Qian</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">In</subfield><subfield code="t">IEEE Access</subfield><subfield code="d">IEEE, 2014</subfield><subfield code="g">6(2018), Seite 58673-58681</subfield><subfield code="w">(DE-627)728440385</subfield><subfield code="w">(DE-600)2687964-5</subfield><subfield code="x">21693536</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:6</subfield><subfield code="g">year:2018</subfield><subfield code="g">pages:58673-58681</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doi.org/10.1109/ACCESS.2018.2872560</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://doaj.org/article/9a1d2b4dfd3a4d5cb1c2a1a42d806551</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://ieeexplore.ieee.org/document/8478113/</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="856" ind1="4" ind2="2"><subfield code="u">https://doaj.org/toc/2169-3536</subfield><subfield code="y">Journal toc</subfield><subfield code="z">kostenfrei</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_DOAJ</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_370</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">6</subfield><subfield code="j">2018</subfield><subfield code="h">58673-58681</subfield></datafield></record></collection>
score	7.4013433

Nicht das Richtige dabei?

Schreiben Sie uns!

Sparsing Deep Neural Network Using Semi-Discrete Matrix Decomposition

Nicht das Richtige dabei?

Zugang & Verfügbarkeit

Vorhandene Bände

Nicht das Richtige dabei?