Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set
Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing...
Ausführliche Beschreibung
Gespeichert in:
| Autor*in: |
Lenselink, Eelke B. [verfasserIn] |
|---|
| Format: |
E-Artikel |
|---|---|
| Sprache: |
Englisch |
| Erschienen: |
2017 |
|---|
| Schlagwörter: |
|---|
| Anmerkung: |
© The Author(s) 2017 |
|---|
| Übergeordnetes Werk: |
Enthalten in: Journal of cheminformatics - London : BioMed Central, 2009, 9(2017), 1 vom: 14. Aug. |
|---|---|
| Übergeordnetes Werk: |
volume:9 ; year:2017 ; number:1 ; day:14 ; month:08 |
| Links: |
|---|
| DOI / URN: |
10.1186/s13321-017-0232-0 |
|---|
| Katalog-ID: |
SPR031344763 |
|---|
| LEADER | 01000caa a22002652 4500 | ||
|---|---|---|---|
| 001 | SPR031344763 | ||
| 003 | DE-627 | ||
| 005 | 20230519225805.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 201007s2017 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1186/s13321-017-0232-0 |2 doi | |
| 035 | |a (DE-627)SPR031344763 | ||
| 035 | |a (SPR)s13321-017-0232-0-e | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Lenselink, Eelke B. |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| 264 | 1 | |c 2017 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a © The Author(s) 2017 | ||
| 520 | |a Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. | ||
| 650 | 4 | |a Deep neural networks |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a ChEMBL |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a QSAR |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Proteochemometrics |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Chemogenomics |7 (dpeaa)DE-He213 | |
| 650 | 4 | |a Cheminformatics |7 (dpeaa)DE-He213 | |
| 700 | 1 | |a ten Dijke, Niels |4 aut | |
| 700 | 1 | |a Bongers, Brandon |4 aut | |
| 700 | 1 | |a Papadatos, George |4 aut | |
| 700 | 1 | |a van Vlijmen, Herman W. T. |4 aut | |
| 700 | 1 | |a Kowalczyk, Wojtek |4 aut | |
| 700 | 1 | |a IJzerman, Adriaan P. |4 aut | |
| 700 | 1 | |a van Westen, Gerard J. P. |0 (orcid)0000-0003-0717-1817 |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Journal of cheminformatics |d London : BioMed Central, 2009 |g 9(2017), 1 vom: 14. Aug. |w (DE-627)594779219 |w (DE-600)2486539-4 |x 1758-2946 |7 nnns |
| 773 | 1 | 8 | |g volume:9 |g year:2017 |g number:1 |g day:14 |g month:08 |
| 856 | 4 | 0 | |u https://dx.doi.org/10.1186/s13321-017-0232-0 |z kostenfrei |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a SYSFLAG_A | ||
| 912 | |a GBV_SPRINGER | ||
| 912 | |a SSG-OLC-PHA | ||
| 912 | |a GBV_ILN_11 | ||
| 912 | |a GBV_ILN_20 | ||
| 912 | |a GBV_ILN_22 | ||
| 912 | |a GBV_ILN_23 | ||
| 912 | |a GBV_ILN_24 | ||
| 912 | |a GBV_ILN_31 | ||
| 912 | |a GBV_ILN_39 | ||
| 912 | |a GBV_ILN_40 | ||
| 912 | |a GBV_ILN_60 | ||
| 912 | |a GBV_ILN_62 | ||
| 912 | |a GBV_ILN_63 | ||
| 912 | |a GBV_ILN_65 | ||
| 912 | |a GBV_ILN_69 | ||
| 912 | |a GBV_ILN_70 | ||
| 912 | |a GBV_ILN_73 | ||
| 912 | |a GBV_ILN_95 | ||
| 912 | |a GBV_ILN_105 | ||
| 912 | |a GBV_ILN_110 | ||
| 912 | |a GBV_ILN_151 | ||
| 912 | |a GBV_ILN_161 | ||
| 912 | |a GBV_ILN_170 | ||
| 912 | |a GBV_ILN_206 | ||
| 912 | |a GBV_ILN_213 | ||
| 912 | |a GBV_ILN_230 | ||
| 912 | |a GBV_ILN_285 | ||
| 912 | |a GBV_ILN_293 | ||
| 912 | |a GBV_ILN_602 | ||
| 912 | |a GBV_ILN_2003 | ||
| 912 | |a GBV_ILN_2005 | ||
| 912 | |a GBV_ILN_2009 | ||
| 912 | |a GBV_ILN_2011 | ||
| 912 | |a GBV_ILN_2014 | ||
| 912 | |a GBV_ILN_2027 | ||
| 912 | |a GBV_ILN_2055 | ||
| 912 | |a GBV_ILN_2111 | ||
| 912 | |a GBV_ILN_4012 | ||
| 912 | |a GBV_ILN_4037 | ||
| 912 | |a GBV_ILN_4112 | ||
| 912 | |a GBV_ILN_4125 | ||
| 912 | |a GBV_ILN_4126 | ||
| 912 | |a GBV_ILN_4249 | ||
| 912 | |a GBV_ILN_4305 | ||
| 912 | |a GBV_ILN_4306 | ||
| 912 | |a GBV_ILN_4307 | ||
| 912 | |a GBV_ILN_4313 | ||
| 912 | |a GBV_ILN_4322 | ||
| 912 | |a GBV_ILN_4323 | ||
| 912 | |a GBV_ILN_4324 | ||
| 912 | |a GBV_ILN_4325 | ||
| 912 | |a GBV_ILN_4326 | ||
| 912 | |a GBV_ILN_4335 | ||
| 912 | |a GBV_ILN_4338 | ||
| 912 | |a GBV_ILN_4367 | ||
| 912 | |a GBV_ILN_4700 | ||
| 951 | |a AR | ||
| 952 | |d 9 |j 2017 |e 1 |b 14 |c 08 | ||
| author_variant |
e b l eb ebl d n t dn dnt b b bb g p gp v h w t v vhwt vhwtv w k wk a p i ap api w g j p v wgjp wgjpv |
|---|---|
| matchkey_str |
article:17582946:2017----::eodhhpdenuantokotefretbihdehduigce |
| hierarchy_sort_str |
2017 |
| publishDate |
2017 |
| allfields |
10.1186/s13321-017-0232-0 doi (DE-627)SPR031344763 (SPR)s13321-017-0232-0-e DE-627 ger DE-627 rakwb eng Lenselink, Eelke B. verfasserin aut Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set 2017 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier © The Author(s) 2017 Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 ten Dijke, Niels aut Bongers, Brandon aut Papadatos, George aut van Vlijmen, Herman W. T. aut Kowalczyk, Wojtek aut IJzerman, Adriaan P. aut van Westen, Gerard J. P. (orcid)0000-0003-0717-1817 aut Enthalten in Journal of cheminformatics London : BioMed Central, 2009 9(2017), 1 vom: 14. Aug. (DE-627)594779219 (DE-600)2486539-4 1758-2946 nnns volume:9 year:2017 number:1 day:14 month:08 https://dx.doi.org/10.1186/s13321-017-0232-0 kostenfrei Volltext GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 9 2017 1 14 08 |
| spelling |
10.1186/s13321-017-0232-0 doi (DE-627)SPR031344763 (SPR)s13321-017-0232-0-e DE-627 ger DE-627 rakwb eng Lenselink, Eelke B. verfasserin aut Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set 2017 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier © The Author(s) 2017 Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 ten Dijke, Niels aut Bongers, Brandon aut Papadatos, George aut van Vlijmen, Herman W. T. aut Kowalczyk, Wojtek aut IJzerman, Adriaan P. aut van Westen, Gerard J. P. (orcid)0000-0003-0717-1817 aut Enthalten in Journal of cheminformatics London : BioMed Central, 2009 9(2017), 1 vom: 14. Aug. (DE-627)594779219 (DE-600)2486539-4 1758-2946 nnns volume:9 year:2017 number:1 day:14 month:08 https://dx.doi.org/10.1186/s13321-017-0232-0 kostenfrei Volltext GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 9 2017 1 14 08 |
| allfields_unstemmed |
10.1186/s13321-017-0232-0 doi (DE-627)SPR031344763 (SPR)s13321-017-0232-0-e DE-627 ger DE-627 rakwb eng Lenselink, Eelke B. verfasserin aut Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set 2017 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier © The Author(s) 2017 Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 ten Dijke, Niels aut Bongers, Brandon aut Papadatos, George aut van Vlijmen, Herman W. T. aut Kowalczyk, Wojtek aut IJzerman, Adriaan P. aut van Westen, Gerard J. P. (orcid)0000-0003-0717-1817 aut Enthalten in Journal of cheminformatics London : BioMed Central, 2009 9(2017), 1 vom: 14. Aug. (DE-627)594779219 (DE-600)2486539-4 1758-2946 nnns volume:9 year:2017 number:1 day:14 month:08 https://dx.doi.org/10.1186/s13321-017-0232-0 kostenfrei Volltext GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 9 2017 1 14 08 |
| allfieldsGer |
10.1186/s13321-017-0232-0 doi (DE-627)SPR031344763 (SPR)s13321-017-0232-0-e DE-627 ger DE-627 rakwb eng Lenselink, Eelke B. verfasserin aut Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set 2017 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier © The Author(s) 2017 Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 ten Dijke, Niels aut Bongers, Brandon aut Papadatos, George aut van Vlijmen, Herman W. T. aut Kowalczyk, Wojtek aut IJzerman, Adriaan P. aut van Westen, Gerard J. P. (orcid)0000-0003-0717-1817 aut Enthalten in Journal of cheminformatics London : BioMed Central, 2009 9(2017), 1 vom: 14. Aug. (DE-627)594779219 (DE-600)2486539-4 1758-2946 nnns volume:9 year:2017 number:1 day:14 month:08 https://dx.doi.org/10.1186/s13321-017-0232-0 kostenfrei Volltext GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 9 2017 1 14 08 |
| allfieldsSound |
10.1186/s13321-017-0232-0 doi (DE-627)SPR031344763 (SPR)s13321-017-0232-0-e DE-627 ger DE-627 rakwb eng Lenselink, Eelke B. verfasserin aut Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set 2017 Text txt rdacontent Computermedien c rdamedia Online-Ressource cr rdacarrier © The Author(s) 2017 Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 ten Dijke, Niels aut Bongers, Brandon aut Papadatos, George aut van Vlijmen, Herman W. T. aut Kowalczyk, Wojtek aut IJzerman, Adriaan P. aut van Westen, Gerard J. P. (orcid)0000-0003-0717-1817 aut Enthalten in Journal of cheminformatics London : BioMed Central, 2009 9(2017), 1 vom: 14. Aug. (DE-627)594779219 (DE-600)2486539-4 1758-2946 nnns volume:9 year:2017 number:1 day:14 month:08 https://dx.doi.org/10.1186/s13321-017-0232-0 kostenfrei Volltext GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 AR 9 2017 1 14 08 |
| language |
English |
| source |
Enthalten in Journal of cheminformatics 9(2017), 1 vom: 14. Aug. volume:9 year:2017 number:1 day:14 month:08 |
| sourceStr |
Enthalten in Journal of cheminformatics 9(2017), 1 vom: 14. Aug. volume:9 year:2017 number:1 day:14 month:08 |
| format_phy_str_mv |
Article |
| institution |
findex.gbv.de |
| topic_facet |
Deep neural networks ChEMBL QSAR Proteochemometrics Chemogenomics Cheminformatics |
| isfreeaccess_bool |
true |
| container_title |
Journal of cheminformatics |
| authorswithroles_txt_mv |
Lenselink, Eelke B. @@aut@@ ten Dijke, Niels @@aut@@ Bongers, Brandon @@aut@@ Papadatos, George @@aut@@ van Vlijmen, Herman W. T. @@aut@@ Kowalczyk, Wojtek @@aut@@ IJzerman, Adriaan P. @@aut@@ van Westen, Gerard J. P. @@aut@@ |
| publishDateDaySort_date |
2017-08-14T00:00:00Z |
| hierarchy_top_id |
594779219 |
| id |
SPR031344763 |
| language_de |
englisch |
| fullrecord |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">SPR031344763</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230519225805.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">201007s2017 xx |||||o 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1186/s13321-017-0232-0</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)SPR031344763</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(SPR)s13321-017-0232-0-e</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Lenselink, Eelke B.</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2017</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">© The Author(s) 2017</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural networks</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">ChEMBL</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">QSAR</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Proteochemometrics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Chemogenomics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Cheminformatics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">ten Dijke, Niels</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Bongers, Brandon</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Papadatos, George</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">van Vlijmen, Herman W. T.</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Kowalczyk, Wojtek</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">IJzerman, Adriaan P.</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">van Westen, Gerard J. P.</subfield><subfield code="0">(orcid)0000-0003-0717-1817</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">Enthalten in</subfield><subfield code="t">Journal of cheminformatics</subfield><subfield code="d">London : BioMed Central, 2009</subfield><subfield code="g">9(2017), 1 vom: 14. Aug.</subfield><subfield code="w">(DE-627)594779219</subfield><subfield code="w">(DE-600)2486539-4</subfield><subfield code="x">1758-2946</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:9</subfield><subfield code="g">year:2017</subfield><subfield code="g">number:1</subfield><subfield code="g">day:14</subfield><subfield code="g">month:08</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://dx.doi.org/10.1186/s13321-017-0232-0</subfield><subfield code="z">kostenfrei</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_SPRINGER</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_206</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2003</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2005</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2009</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2011</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2027</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2055</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2111</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4326</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">9</subfield><subfield code="j">2017</subfield><subfield code="e">1</subfield><subfield code="b">14</subfield><subfield code="c">08</subfield></datafield></record></collection>
|
| author |
Lenselink, Eelke B. |
| spellingShingle |
Lenselink, Eelke B. misc Deep neural networks misc ChEMBL misc QSAR misc Proteochemometrics misc Chemogenomics misc Cheminformatics Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| authorStr |
Lenselink, Eelke B. |
| ppnlink_with_tag_str_mv |
@@773@@(DE-627)594779219 |
| format |
electronic Article |
| delete_txt_mv |
keep |
| author_role |
aut aut aut aut aut aut aut aut |
| collection |
springer |
| remote_str |
true |
| illustrated |
Not Illustrated |
| issn |
1758-2946 |
| topic_title |
Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set Deep neural networks (dpeaa)DE-He213 ChEMBL (dpeaa)DE-He213 QSAR (dpeaa)DE-He213 Proteochemometrics (dpeaa)DE-He213 Chemogenomics (dpeaa)DE-He213 Cheminformatics (dpeaa)DE-He213 |
| topic |
misc Deep neural networks misc ChEMBL misc QSAR misc Proteochemometrics misc Chemogenomics misc Cheminformatics |
| topic_unstemmed |
misc Deep neural networks misc ChEMBL misc QSAR misc Proteochemometrics misc Chemogenomics misc Cheminformatics |
| topic_browse |
misc Deep neural networks misc ChEMBL misc QSAR misc Proteochemometrics misc Chemogenomics misc Cheminformatics |
| format_facet |
Elektronische Aufsätze Aufsätze Elektronische Ressource |
| format_main_str_mv |
Text Zeitschrift/Artikel |
| carriertype_str_mv |
cr |
| hierarchy_parent_title |
Journal of cheminformatics |
| hierarchy_parent_id |
594779219 |
| hierarchy_top_title |
Journal of cheminformatics |
| isfreeaccess_txt |
true |
| familylinks_str_mv |
(DE-627)594779219 (DE-600)2486539-4 |
| title |
Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| ctrlnum |
(DE-627)SPR031344763 (SPR)s13321-017-0232-0-e |
| title_full |
Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| author_sort |
Lenselink, Eelke B. |
| journal |
Journal of cheminformatics |
| journalStr |
Journal of cheminformatics |
| lang_code |
eng |
| isOA_bool |
true |
| recordtype |
marc |
| publishDateSort |
2017 |
| contenttype_str_mv |
txt |
| author_browse |
Lenselink, Eelke B. ten Dijke, Niels Bongers, Brandon Papadatos, George van Vlijmen, Herman W. T. Kowalczyk, Wojtek IJzerman, Adriaan P. van Westen, Gerard J. P. |
| container_volume |
9 |
| format_se |
Elektronische Aufsätze |
| author-letter |
Lenselink, Eelke B. |
| doi_str_mv |
10.1186/s13321-017-0232-0 |
| normlink |
(ORCID)0000-0003-0717-1817 |
| normlink_prefix_str_mv |
(orcid)0000-0003-0717-1817 |
| title_sort |
beyond the hype: deep neural networks outperform established methods using a chembl bioactivity benchmark set |
| title_auth |
Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| abstract |
Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. © The Author(s) 2017 |
| abstractGer |
Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. © The Author(s) 2017 |
| abstract_unstemmed |
Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract. © The Author(s) 2017 |
| collection_details |
GBV_USEFLAG_A SYSFLAG_A GBV_SPRINGER SSG-OLC-PHA GBV_ILN_11 GBV_ILN_20 GBV_ILN_22 GBV_ILN_23 GBV_ILN_24 GBV_ILN_31 GBV_ILN_39 GBV_ILN_40 GBV_ILN_60 GBV_ILN_62 GBV_ILN_63 GBV_ILN_65 GBV_ILN_69 GBV_ILN_70 GBV_ILN_73 GBV_ILN_95 GBV_ILN_105 GBV_ILN_110 GBV_ILN_151 GBV_ILN_161 GBV_ILN_170 GBV_ILN_206 GBV_ILN_213 GBV_ILN_230 GBV_ILN_285 GBV_ILN_293 GBV_ILN_602 GBV_ILN_2003 GBV_ILN_2005 GBV_ILN_2009 GBV_ILN_2011 GBV_ILN_2014 GBV_ILN_2027 GBV_ILN_2055 GBV_ILN_2111 GBV_ILN_4012 GBV_ILN_4037 GBV_ILN_4112 GBV_ILN_4125 GBV_ILN_4126 GBV_ILN_4249 GBV_ILN_4305 GBV_ILN_4306 GBV_ILN_4307 GBV_ILN_4313 GBV_ILN_4322 GBV_ILN_4323 GBV_ILN_4324 GBV_ILN_4325 GBV_ILN_4326 GBV_ILN_4335 GBV_ILN_4338 GBV_ILN_4367 GBV_ILN_4700 |
| container_issue |
1 |
| title_short |
Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set |
| url |
https://dx.doi.org/10.1186/s13321-017-0232-0 |
| remote_bool |
true |
| author2 |
ten Dijke, Niels Bongers, Brandon Papadatos, George van Vlijmen, Herman W. T. Kowalczyk, Wojtek IJzerman, Adriaan P. van Westen, Gerard J. P. |
| author2Str |
ten Dijke, Niels Bongers, Brandon Papadatos, George van Vlijmen, Herman W. T. Kowalczyk, Wojtek IJzerman, Adriaan P. van Westen, Gerard J. P. |
| ppnlink |
594779219 |
| mediatype_str_mv |
c |
| isOA_txt |
true |
| hochschulschrift_bool |
false |
| doi_str |
10.1186/s13321-017-0232-0 |
| up_date |
2024-07-03T23:16:49.779Z |
| _version_ |
1803601691051819008 |
| fullrecord_marcxml |
<?xml version="1.0" encoding="UTF-8"?><collection xmlns="http://www.loc.gov/MARC21/slim"><record><leader>01000caa a22002652 4500</leader><controlfield tag="001">SPR031344763</controlfield><controlfield tag="003">DE-627</controlfield><controlfield tag="005">20230519225805.0</controlfield><controlfield tag="007">cr uuu---uuuuu</controlfield><controlfield tag="008">201007s2017 xx |||||o 00| ||eng c</controlfield><datafield tag="024" ind1="7" ind2=" "><subfield code="a">10.1186/s13321-017-0232-0</subfield><subfield code="2">doi</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(DE-627)SPR031344763</subfield></datafield><datafield tag="035" ind1=" " ind2=" "><subfield code="a">(SPR)s13321-017-0232-0-e</subfield></datafield><datafield tag="040" ind1=" " ind2=" "><subfield code="a">DE-627</subfield><subfield code="b">ger</subfield><subfield code="c">DE-627</subfield><subfield code="e">rakwb</subfield></datafield><datafield tag="041" ind1=" " ind2=" "><subfield code="a">eng</subfield></datafield><datafield tag="100" ind1="1" ind2=" "><subfield code="a">Lenselink, Eelke B.</subfield><subfield code="e">verfasserin</subfield><subfield code="4">aut</subfield></datafield><datafield tag="245" ind1="1" ind2="0"><subfield code="a">Beyond the hype: deep neural networks outperform established methods using a ChEMBL bioactivity benchmark set</subfield></datafield><datafield tag="264" ind1=" " ind2="1"><subfield code="c">2017</subfield></datafield><datafield tag="336" ind1=" " ind2=" "><subfield code="a">Text</subfield><subfield code="b">txt</subfield><subfield code="2">rdacontent</subfield></datafield><datafield tag="337" ind1=" " ind2=" "><subfield code="a">Computermedien</subfield><subfield code="b">c</subfield><subfield code="2">rdamedia</subfield></datafield><datafield tag="338" ind1=" " ind2=" "><subfield code="a">Online-Ressource</subfield><subfield code="b">cr</subfield><subfield code="2">rdacarrier</subfield></datafield><datafield tag="500" ind1=" " ind2=" "><subfield code="a">© The Author(s) 2017</subfield></datafield><datafield tag="520" ind1=" " ind2=" "><subfield code="a">Abstract The increase of publicly available bioactivity data in recent years has fueled and catalyzed research in chemogenomics, data mining, and modeling approaches. As a direct result, over the past few years a multitude of different methods have been reported and evaluated, such as target fishing, nearest neighbor similarity-based methods, and Quantitative Structure Activity Relationship (QSAR)-based protocols. However, such studies are typically conducted on different datasets, using different validation strategies, and different metrics. In this study, different methods were compared using one single standardized dataset obtained from ChEMBL, which is made available to the public, using standardized metrics (BEDROC and Matthews Correlation Coefficient). Specifically, the performance of Naïve Bayes, Random Forests, Support Vector Machines, Logistic Regression, and Deep Neural Networks was assessed using QSAR and proteochemometric (PCM) methods. All methods were validated using both a random split validation and a temporal validation, with the latter being a more realistic benchmark of expected prospective execution. Deep Neural Networks are the top performing classifiers, highlighting the added value of Deep Neural Networks over other more conventional methods. Moreover, the best method (‘DNN_PCM’) performed significantly better at almost one standard deviation higher than the mean performance. Furthermore, Multi-task and PCM implementations were shown to improve performance over single task Deep Neural Networks. Conversely, target prediction performed almost two standard deviations under the mean performance. Random Forests, Support Vector Machines, and Logistic Regression performed around mean performance. Finally, using an ensemble of DNNs, alongside additional tuning, enhanced the relative performance by another 27% (compared with unoptimized ‘DNN_PCM’). Here, a standardized set to test and evaluate different machine learning algorithms in the context of multi-task learning is offered by providing the data and the protocols.Graphical Abstract.</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Deep neural networks</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">ChEMBL</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">QSAR</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Proteochemometrics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Chemogenomics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="650" ind1=" " ind2="4"><subfield code="a">Cheminformatics</subfield><subfield code="7">(dpeaa)DE-He213</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">ten Dijke, Niels</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Bongers, Brandon</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Papadatos, George</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">van Vlijmen, Herman W. T.</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">Kowalczyk, Wojtek</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">IJzerman, Adriaan P.</subfield><subfield code="4">aut</subfield></datafield><datafield tag="700" ind1="1" ind2=" "><subfield code="a">van Westen, Gerard J. P.</subfield><subfield code="0">(orcid)0000-0003-0717-1817</subfield><subfield code="4">aut</subfield></datafield><datafield tag="773" ind1="0" ind2="8"><subfield code="i">Enthalten in</subfield><subfield code="t">Journal of cheminformatics</subfield><subfield code="d">London : BioMed Central, 2009</subfield><subfield code="g">9(2017), 1 vom: 14. Aug.</subfield><subfield code="w">(DE-627)594779219</subfield><subfield code="w">(DE-600)2486539-4</subfield><subfield code="x">1758-2946</subfield><subfield code="7">nnns</subfield></datafield><datafield tag="773" ind1="1" ind2="8"><subfield code="g">volume:9</subfield><subfield code="g">year:2017</subfield><subfield code="g">number:1</subfield><subfield code="g">day:14</subfield><subfield code="g">month:08</subfield></datafield><datafield tag="856" ind1="4" ind2="0"><subfield code="u">https://dx.doi.org/10.1186/s13321-017-0232-0</subfield><subfield code="z">kostenfrei</subfield><subfield code="3">Volltext</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_USEFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SYSFLAG_A</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_SPRINGER</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">SSG-OLC-PHA</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_11</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_20</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_22</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_23</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_24</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_31</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_39</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_40</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_60</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_62</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_63</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_65</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_69</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_70</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_73</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_95</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_105</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_110</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_151</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_161</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_170</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_206</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_213</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_230</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_285</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_293</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_602</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2003</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2005</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2009</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2011</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2014</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2027</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2055</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_2111</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4012</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4037</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4112</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4125</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4126</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4249</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4305</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4306</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4307</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4313</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4322</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4323</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4324</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4325</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4326</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4335</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4338</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4367</subfield></datafield><datafield tag="912" ind1=" " ind2=" "><subfield code="a">GBV_ILN_4700</subfield></datafield><datafield tag="951" ind1=" " ind2=" "><subfield code="a">AR</subfield></datafield><datafield tag="952" ind1=" " ind2=" "><subfield code="d">9</subfield><subfield code="j">2017</subfield><subfield code="e">1</subfield><subfield code="b">14</subfield><subfield code="c">08</subfield></datafield></record></collection>
|
| score |
7.398567 |