Pruning during training by network efficacy modeling
Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early p...
Ausführliche Beschreibung
Gespeichert in:
| Autor*in: |
Rajpal, Mohit [verfasserIn] |
|---|
| Format: |
Artikel |
|---|---|
| Sprache: |
Englisch |
| Erschienen: |
2023 |
|---|
| Schlagwörter: |
|---|
| Anmerkung: |
© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
|---|
| Übergeordnetes Werk: |
Enthalten in: Machine learning - Springer US, 1986, 112(2023), 7 vom: 14. März, Seite 2653-2684 |
|---|---|
| Übergeordnetes Werk: |
volume:112 ; year:2023 ; number:7 ; day:14 ; month:03 ; pages:2653-2684 |
| Links: |
|---|
| DOI / URN: |
10.1007/s10994-023-06304-1 |
|---|
| Katalog-ID: |
OLC2144469385 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | OLC2144469385 | ||
| 003 | DE-627 | ||
| 005 | 20240118095212.0 | ||
| 007 | tu | ||
| 008 | 240118s2023 xx ||||| 00| ||eng c | ||
| 024 | 7 | |a 10.1007/s10994-023-06304-1 |2 doi | |
| 035 | |a (DE-627)OLC2144469385 | ||
| 035 | |a (DE-He213)s10994-023-06304-1-p | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 082 | 0 | 4 | |a 150 |a 004 |q VZ |
| 100 | 1 | |a Rajpal, Mohit |e verfasserin |0 (orcid)0000-0002-8928-6302 |4 aut | |
| 245 | 1 | 0 | |a Pruning during training by network efficacy modeling |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ohne Hilfsmittel zu benutzen |b n |2 rdamedia | ||
| 338 | |a Band |b nc |2 rdacarrier | ||
| 500 | |a © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. | ||
| 520 | |a Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. | ||
| 650 | 4 | |a Early pruning | |
| 650 | 4 | |a Network efficacy modeling | |
| 650 | 4 | |a Network saliency | |
| 650 | 4 | |a Multi-output Gaussian process | |
| 650 | 4 | |a Foresight pruning | |
| 700 | 1 | |a Zhang, Yehong |4 aut | |
| 700 | 1 | |a Low, Bryan Kian Hsiang |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Machine learning |d Springer US, 1986 |g 112(2023), 7 vom: 14. März, Seite 2653-2684 |w (DE-627)12920403X |w (DE-600)54638-0 |w (DE-576)014457377 |x 0885-6125 |7 nnns |
| 773 | 1 | 8 | |g volume:112 |g year:2023 |g number:7 |g day:14 |g month:03 |g pages:2653-2684 |
| 856 | 4 | 1 | |u https://doi.org/10.1007/s10994-023-06304-1 |z lizenzpflichtig |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a SYSFLAG_A | ||
| 912 | |a GBV_OLC | ||
| 912 | |a SSG-OLC-MAT | ||
| 951 | |a AR | ||
| 952 | |d 112 |j 2023 |e 7 |b 14 |c 03 |h 2653-2684 | ||
| author_variant |
m r mr y z yz b k h l bkh bkhl |
|---|---|
| matchkey_str |
article:08856125:2023----::rnndrntannbntokf |
| hierarchy_sort_str |
2023 |
| publishDate |
2023 |
| allfields |
10.1007/s10994-023-06304-1 doi (DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p DE-627 ger DE-627 rakwb eng 150 004 VZ Rajpal, Mohit verfasserin (orcid)0000-0002-8928-6302 aut Pruning during training by network efficacy modeling 2023 Text txt rdacontent ohne Hilfsmittel zu benutzen n rdamedia Band nc rdacarrier © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning Zhang, Yehong aut Low, Bryan Kian Hsiang aut Enthalten in Machine learning Springer US, 1986 112(2023), 7 vom: 14. März, Seite 2653-2684 (DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 0885-6125 nnns volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 https://doi.org/10.1007/s10994-023-06304-1 lizenzpflichtig Volltext GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT AR 112 2023 7 14 03 2653-2684 |
| spelling |
10.1007/s10994-023-06304-1 doi (DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p DE-627 ger DE-627 rakwb eng 150 004 VZ Rajpal, Mohit verfasserin (orcid)0000-0002-8928-6302 aut Pruning during training by network efficacy modeling 2023 Text txt rdacontent ohne Hilfsmittel zu benutzen n rdamedia Band nc rdacarrier © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning Zhang, Yehong aut Low, Bryan Kian Hsiang aut Enthalten in Machine learning Springer US, 1986 112(2023), 7 vom: 14. März, Seite 2653-2684 (DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 0885-6125 nnns volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 https://doi.org/10.1007/s10994-023-06304-1 lizenzpflichtig Volltext GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT AR 112 2023 7 14 03 2653-2684 |
| allfields_unstemmed |
10.1007/s10994-023-06304-1 doi (DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p DE-627 ger DE-627 rakwb eng 150 004 VZ Rajpal, Mohit verfasserin (orcid)0000-0002-8928-6302 aut Pruning during training by network efficacy modeling 2023 Text txt rdacontent ohne Hilfsmittel zu benutzen n rdamedia Band nc rdacarrier © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning Zhang, Yehong aut Low, Bryan Kian Hsiang aut Enthalten in Machine learning Springer US, 1986 112(2023), 7 vom: 14. März, Seite 2653-2684 (DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 0885-6125 nnns volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 https://doi.org/10.1007/s10994-023-06304-1 lizenzpflichtig Volltext GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT AR 112 2023 7 14 03 2653-2684 |
| allfieldsGer |
10.1007/s10994-023-06304-1 doi (DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p DE-627 ger DE-627 rakwb eng 150 004 VZ Rajpal, Mohit verfasserin (orcid)0000-0002-8928-6302 aut Pruning during training by network efficacy modeling 2023 Text txt rdacontent ohne Hilfsmittel zu benutzen n rdamedia Band nc rdacarrier © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning Zhang, Yehong aut Low, Bryan Kian Hsiang aut Enthalten in Machine learning Springer US, 1986 112(2023), 7 vom: 14. März, Seite 2653-2684 (DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 0885-6125 nnns volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 https://doi.org/10.1007/s10994-023-06304-1 lizenzpflichtig Volltext GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT AR 112 2023 7 14 03 2653-2684 |
| allfieldsSound |
10.1007/s10994-023-06304-1 doi (DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p DE-627 ger DE-627 rakwb eng 150 004 VZ Rajpal, Mohit verfasserin (orcid)0000-0002-8928-6302 aut Pruning during training by network efficacy modeling 2023 Text txt rdacontent ohne Hilfsmittel zu benutzen n rdamedia Band nc rdacarrier © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning Zhang, Yehong aut Low, Bryan Kian Hsiang aut Enthalten in Machine learning Springer US, 1986 112(2023), 7 vom: 14. März, Seite 2653-2684 (DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 0885-6125 nnns volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 https://doi.org/10.1007/s10994-023-06304-1 lizenzpflichtig Volltext GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT AR 112 2023 7 14 03 2653-2684 |
| language |
English |
| source |
Enthalten in Machine learning 112(2023), 7 vom: 14. März, Seite 2653-2684 volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 |
| sourceStr |
Enthalten in Machine learning 112(2023), 7 vom: 14. März, Seite 2653-2684 volume:112 year:2023 number:7 day:14 month:03 pages:2653-2684 |
| format_phy_str_mv |
Article |
| institution |
findex.gbv.de |
| topic_facet |
Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning |
| dewey-raw |
150 |
| isfreeaccess_bool |
false |
| container_title |
Machine learning |
| authorswithroles_txt_mv |
Rajpal, Mohit @@aut@@ Zhang, Yehong @@aut@@ Low, Bryan Kian Hsiang @@aut@@ |
| publishDateDaySort_date |
2023-03-14T00:00:00Z |
| hierarchy_top_id |
12920403X |
| dewey-sort |
3150 |
| id |
OLC2144469385 |
| language_de |
englisch |
| fullrecord |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000naa a22002652 4500</leader><controlfield tag="001">OLC2144469385</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20240118095212.0</controlfield><controlfield tag="007">tu</controlfield><controlfield tag="008">240118s2023 xx ||||| 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1007/s10994-023-06304-1</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)OLC2144469385</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-He213)s10994-023-06304-1-p</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="082" ind1="0" ind2="4"><subfield code="a">150</subfield><subfield code="a">004</subfield><subfield code="q">VZ</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Rajpal, Mohit</subfield><subfield code="e">verfasserin</subfield><subfield code="0">(orcid)0000-0002-8928-6302</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Pruning during training by network efficacy modeling</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2023</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">ohne Hilfsmittel zu benutzen</subfield><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Band</subfield><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Early pruning</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Network efficacy modeling</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Network saliency</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Multi-output Gaussian process</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Foresight pruning</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Zhang, Yehong</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Low, Bryan Kian Hsiang</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">Enthalten in</subfield><subfield code="t">Machine learning</subfield><subfield code="d">Springer US, 1986</subfield><subfield code="g">112(2023), 7 vom: 14. März, Seite 2653-2684</subfield><subfield code="w">(DE-627)12920403X</subfield><subfield code="w">(DE-600)54638-0</subfield><subfield code="w">(DE-576)014457377</subfield><subfield code="x">0885-6125</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:112</subfield><subfield code="g">year:2023</subfield><subfield code="g">number:7</subfield><subfield code="g">day:14</subfield><subfield code="g">month:03</subfield><subfield code="g">pages:2653-2684</subfield></datafield><datafield tag="856" ind1="4" ind2="1"><subfield code="u">https://doi.org/10.1007/s10994-023-06304-1</subfield><subfield code="z">lizenzpflichtig</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_OLC</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-MAT</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">112</subfield><subfield code="j">2023</subfield><subfield code="e">7</subfield><subfield code="b">14</subfield><subfield code="c">03</subfield><subfield code="h">2653-2684</subfield></datafield></record></collection>
|
| author |
Rajpal, Mohit |
| spellingShingle |
Rajpal, Mohit ddc 150 misc Early pruning misc Network efficacy modeling misc Network saliency misc Multi-output Gaussian process misc Foresight pruning Pruning during training by network efficacy modeling |
| authorStr |
Rajpal, Mohit |
| ppnlink_with_tag_str_mv |
@@773@@(DE-627)12920403X |
| format |
Article |
| dewey-ones |
150 - Psychology 004 - Data processing & computer science |
| delete_txt_mv |
keep |
| author_role |
aut aut aut |
| collection |
OLC |
| remote_str |
false |
| illustrated |
Not Illustrated |
| issn |
0885-6125 |
| topic_title |
150 004 VZ Pruning during training by network efficacy modeling Early pruning Network efficacy modeling Network saliency Multi-output Gaussian process Foresight pruning |
| topic |
ddc 150 misc Early pruning misc Network efficacy modeling misc Network saliency misc Multi-output Gaussian process misc Foresight pruning |
| topic_unstemmed |
ddc 150 misc Early pruning misc Network efficacy modeling misc Network saliency misc Multi-output Gaussian process misc Foresight pruning |
| topic_browse |
ddc 150 misc Early pruning misc Network efficacy modeling misc Network saliency misc Multi-output Gaussian process misc Foresight pruning |
| format_facet |
Aufsätze Gedruckte Aufsätze |
| format_main_str_mv |
Text Zeitschrift/Artikel |
| carriertype_str_mv |
nc |
| hierarchy_parent_title |
Machine learning |
| hierarchy_parent_id |
12920403X |
| dewey-tens |
150 - Psychology 000 - Computer science, knowledge & systems |
| hierarchy_top_title |
Machine learning |
| isfreeaccess_txt |
false |
| familylinks_str_mv |
(DE-627)12920403X (DE-600)54638-0 (DE-576)014457377 |
| title |
Pruning during training by network efficacy modeling |
| ctrlnum |
(DE-627)OLC2144469385 (DE-He213)s10994-023-06304-1-p |
| title_full |
Pruning during training by network efficacy modeling |
| author_sort |
Rajpal, Mohit |
| journal |
Machine learning |
| journalStr |
Machine learning |
| lang_code |
eng |
| isOA_bool |
false |
| dewey-hundreds |
100 - Philosophy & psychology 000 - Computer science, information & general works |
| recordtype |
marc |
| publishDateSort |
2023 |
| contenttype_str_mv |
txt |
| container_start_page |
2653 |
| author_browse |
Rajpal, Mohit Zhang, Yehong Low, Bryan Kian Hsiang |
| container_volume |
112 |
| class |
150 004 VZ |
| format_se |
Aufsätze |
| author-letter |
Rajpal, Mohit |
| doi_str_mv |
10.1007/s10994-023-06304-1 |
| normlink |
(ORCID)0000-0002-8928-6302 |
| normlink_prefix_str_mv |
(orcid)0000-0002-8928-6302 |
| dewey-full |
150 004 |
| title_sort |
pruning during training by network efficacy modeling |
| title_auth |
Pruning during training by network efficacy modeling |
| abstract |
Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
| abstractGer |
Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
| abstract_unstemmed |
Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches. © The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law. |
| collection_details |
GBV_USEFLAG_A SYSFLAG_A GBV_OLC SSG-OLC-MAT |
| container_issue |
7 |
| title_short |
Pruning during training by network efficacy modeling |
| url |
https://doi.org/10.1007/s10994-023-06304-1 |
| remote_bool |
false |
| author2 |
Zhang, Yehong Low, Bryan Kian Hsiang |
| author2Str |
Zhang, Yehong Low, Bryan Kian Hsiang |
| ppnlink |
12920403X |
| mediatype_str_mv |
n |
| isOA_txt |
false |
| hochschulschrift_bool |
false |
| doi_str |
10.1007/s10994-023-06304-1 |
| up_date |
2024-07-03T22:27:20.497Z |
| _version_ |
1803598577531879424 |
| fullrecord_marcxml |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000naa a22002652 4500</leader><controlfield tag="001">OLC2144469385</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20240118095212.0</controlfield><controlfield tag="007">tu</controlfield><controlfield tag="008">240118s2023 xx ||||| 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1007/s10994-023-06304-1</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)OLC2144469385</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-He213)s10994-023-06304-1-p</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="082" ind1="0" ind2="4"><subfield code="a">150</subfield><subfield code="a">004</subfield><subfield code="q">VZ</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Rajpal, Mohit</subfield><subfield code="e">verfasserin</subfield><subfield code="0">(orcid)0000-0002-8928-6302</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Pruning during training by network efficacy modeling</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2023</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">ohne Hilfsmittel zu benutzen</subfield><subfield code="b">n</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Band</subfield><subfield code="b">nc</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">© The Author(s), under exclusive licence to Springer Science+Business Media LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Abstract Deep neural networks (DNNs) are costly to train. Pruning, an approach to alleviate model complexity by zeroing out or pruning DNN elements, has shown promise in reducing training costs for DNNs with little to no efficacy at a given task. This paper presents a novel method to perform early pruning of DNN elements (e.g., neurons or convolutional filters) during the training process while minimizing losses to model performance. To achieve this, we model the efficacy of DNN elements in a Bayesian manner conditioned upon efficacy data collected during the training and prune DNN elements with low predictive efficacy after training completion. Empirical evaluations show that the proposed Bayesian early pruning improves the computational efficiency of DNN training while better preserving model performance compared to other tested pruning approaches.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Early pruning</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Network efficacy modeling</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Network saliency</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Multi-output Gaussian process</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Foresight pruning</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Zhang, Yehong</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Low, Bryan Kian Hsiang</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">Enthalten in</subfield><subfield code="t">Machine learning</subfield><subfield code="d">Springer US, 1986</subfield><subfield code="g">112(2023), 7 vom: 14. März, Seite 2653-2684</subfield><subfield code="w">(DE-627)12920403X</subfield><subfield code="w">(DE-600)54638-0</subfield><subfield code="w">(DE-576)014457377</subfield><subfield code="x">0885-6125</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:112</subfield><subfield code="g">year:2023</subfield><subfield code="g">number:7</subfield><subfield code="g">day:14</subfield><subfield code="g">month:03</subfield><subfield code="g">pages:2653-2684</subfield></datafield><datafield tag="856" ind1="4" ind2="1"><subfield code="u">https://doi.org/10.1007/s10994-023-06304-1</subfield><subfield code="z">lizenzpflichtig</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_OLC</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-MAT</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">112</subfield><subfield code="j">2023</subfield><subfield code="e">7</subfield><subfield code="b">14</subfield><subfield code="c">03</subfield><subfield code="h">2653-2684</subfield></datafield></record></collection>
|
| score |
7.4011583 |