Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report
Many different solutions to predicting the cognate epitope target of a T-cell receptor (TCR) have been proposed. However several questions on the advantages and disadvantages of these different approaches remain unresolved, as most methods have only been evaluated within the context of their initial...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-03-01
|
| Series: | ImmunoInformatics |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2667119023000046 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850073566871027712 |
|---|---|
| author | Pieter Meysman Justin Barton Barbara Bravi Liel Cohen-Lavi Vadim Karnaukhov Elias Lilleskov Alessandro Montemurro Morten Nielsen Thierry Mora Paul Pereira Anna Postovskaya María Rodríguez Martínez Jorge Fernandez-de-Cossio-Diaz Alexandra Vujkovic Aleksandra M. Walczak Anna Weber Rose Yin Anne Eugster Virag Sharma |
| author_facet | Pieter Meysman Justin Barton Barbara Bravi Liel Cohen-Lavi Vadim Karnaukhov Elias Lilleskov Alessandro Montemurro Morten Nielsen Thierry Mora Paul Pereira Anna Postovskaya María Rodríguez Martínez Jorge Fernandez-de-Cossio-Diaz Alexandra Vujkovic Aleksandra M. Walczak Anna Weber Rose Yin Anne Eugster Virag Sharma |
| author_sort | Pieter Meysman |
| collection | DOAJ |
| description | Many different solutions to predicting the cognate epitope target of a T-cell receptor (TCR) have been proposed. However several questions on the advantages and disadvantages of these different approaches remain unresolved, as most methods have only been evaluated within the context of their initial publications and data sets. Here, we report the findings of the first public TCR-epitope prediction benchmark performed on 23 prediction models in the context of the ImmRep 2022 TCR-epitope specificity workshop. This benchmark revealed that the use of paired-chain alpha-beta, as well as CDR1/2 or V/J information, when available, improves classification obtained with CDR3 data, independent of the underlying approach. In addition, we found that straight-forward distance-based approaches can achieve a respectable performance when compared to more complex machine-learning models. Finally, we highlight the need for a truly independent follow-up benchmark and provide recommendations for the design of such a next benchmark. |
| format | Article |
| id | doaj-art-c337aa296bc14f0e8040baaeb0bebb9e |
| institution | DOAJ |
| issn | 2667-1190 |
| language | English |
| publishDate | 2023-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | ImmunoInformatics |
| spelling | doaj-art-c337aa296bc14f0e8040baaeb0bebb9e2025-08-20T02:46:47ZengElsevierImmunoInformatics2667-11902023-03-01910002410.1016/j.immuno.2023.100024Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop reportPieter Meysman0Justin Barton1Barbara Bravi2Liel Cohen-Lavi3Vadim Karnaukhov4Elias Lilleskov5Alessandro Montemurro6Morten Nielsen7Thierry Mora8Paul Pereira9Anna Postovskaya10María Rodríguez Martínez11Jorge Fernandez-de-Cossio-Diaz12Alexandra Vujkovic13Aleksandra M. Walczak14Anna Weber15Rose Yin16Anne Eugster17Virag Sharma18AUDACIS, University of Antwerp, Antwerp, Belgium; ADREM data lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium; Corresponding author at: AUDACIS, University of Antwerp, Antwerp, Belgium.Institute of Structural and Molecular Biology, University of London, London, United KingdomDepartment of Mathematics, Imperial College London, London, United KingdomNational Institute for Biotechnology in the Negev, Ben-Gurion University of the Negev, Beer-Sheva, Israel; Department of Industrial Engineering and Management, Ben-Gurion University of the Negev, Beer-Sheva, IsraelINSERM U932, PSL University, Institut Curie, Paris 75005, France; Laboratoire de physique de l'Ecole normale superieure, CNRS, PSL University, Sorbonne Universite, Universite Paris-Cité, Paris 75005, FranceDepartment of Physics, University of Washington, Seattle, WA, USADepartment of Health Technology, Technical University of Denmark, Lyngby DK-2800, DenmarkDepartment of Health Technology, Technical University of Denmark, Lyngby DK-2800, DenmarkLaboratoire de physique de l'Ecole normale superieure, CNRS, PSL University, Sorbonne Universite, Universite Paris-Cité, Paris 75005, FranceLaboratoire de physique de l'Ecole normale superieure, CNRS, PSL University, Sorbonne Universite, Universite Paris-Cité, Paris 75005, France; Sanofi R&D, Chilly-Mazarin 91380, FranceAUDACIS, University of Antwerp, Antwerp, Belgium; ADREM data lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium; Clinical Virology Unit, Institute of Tropical Medicine, Antwerp, BelgiumIBM Research Europe, Säumerstrasse 4, Rüschlikon 8803, SwitzerlandLaboratoire de physique de l'Ecole normale superieure, CNRS, PSL University, Sorbonne Universite, Universite Paris-Cité, Paris 75005, FranceAUDACIS, University of Antwerp, Antwerp, Belgium; ADREM data lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium; Clinical Virology Unit, Institute of Tropical Medicine, Antwerp, BelgiumLaboratoire de physique de l'Ecole normale superieure, CNRS, PSL University, Sorbonne Universite, Universite Paris-Cité, Paris 75005, FranceIBM Research Europe, Säumerstrasse 4, Rüschlikon 8803, SwitzerlandDepartment of Chemical Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts, USACenter for Regenerative Therapies Dresden, Faculty of Medicine, TU Dresden, Dresden, Germany; Corresponding authors.Department of Chemical Sciences, School of Natural Sciences, University of Limerick, Limerick V94 T9PX, Ireland; Corresponding authors.Many different solutions to predicting the cognate epitope target of a T-cell receptor (TCR) have been proposed. However several questions on the advantages and disadvantages of these different approaches remain unresolved, as most methods have only been evaluated within the context of their initial publications and data sets. Here, we report the findings of the first public TCR-epitope prediction benchmark performed on 23 prediction models in the context of the ImmRep 2022 TCR-epitope specificity workshop. This benchmark revealed that the use of paired-chain alpha-beta, as well as CDR1/2 or V/J information, when available, improves classification obtained with CDR3 data, independent of the underlying approach. In addition, we found that straight-forward distance-based approaches can achieve a respectable performance when compared to more complex machine-learning models. Finally, we highlight the need for a truly independent follow-up benchmark and provide recommendations for the design of such a next benchmark.http://www.sciencedirect.com/science/article/pii/S2667119023000046 |
| spellingShingle | Pieter Meysman Justin Barton Barbara Bravi Liel Cohen-Lavi Vadim Karnaukhov Elias Lilleskov Alessandro Montemurro Morten Nielsen Thierry Mora Paul Pereira Anna Postovskaya María Rodríguez Martínez Jorge Fernandez-de-Cossio-Diaz Alexandra Vujkovic Aleksandra M. Walczak Anna Weber Rose Yin Anne Eugster Virag Sharma Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report ImmunoInformatics |
| title | Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report |
| title_full | Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report |
| title_fullStr | Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report |
| title_full_unstemmed | Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report |
| title_short | Benchmarking solutions to the T-cell receptor epitope prediction problem: IMMREP22 workshop report |
| title_sort | benchmarking solutions to the t cell receptor epitope prediction problem immrep22 workshop report |
| url | http://www.sciencedirect.com/science/article/pii/S2667119023000046 |
| work_keys_str_mv | AT pietermeysman benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT justinbarton benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT barbarabravi benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT lielcohenlavi benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT vadimkarnaukhov benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT eliaslilleskov benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT alessandromontemurro benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT mortennielsen benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT thierrymora benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT paulpereira benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT annapostovskaya benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT mariarodriguezmartinez benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT jorgefernandezdecossiodiaz benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT alexandravujkovic benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT aleksandramwalczak benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT annaweber benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT roseyin benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT anneeugster benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport AT viragsharma benchmarkingsolutionstothetcellreceptorepitopepredictionproblemimmrep22workshopreport |