Loading...
Tutorial: assessing metagenomics software with the CAMI benchmarking toolkit.
Meyer, Fernando Lesker, Till-Robin Koslicki, David Fritz, Adrian Gurevich, Alexey Darling, Aaron E Sczyrba, Alexander Bremges, Andreas McHardy, Alice C
Meyer, Fernando
Lesker, Till-Robin
Koslicki, David
Fritz, Adrian
Gurevich, Alexey
Darling, Aaron E
Sczyrba, Alexander
Bremges, Andreas
McHardy, Alice C
Citations
Altmetric:
Advisors
Editors
Other Contributors
Issue Date
2021-03-01
Submitted date
Other Titles
Abstract
Computational methods are key in microbiome research, and obtaining a quantitative and unbiased performance estimate is important for method developers and applied researchers. For meaningful comparisons between methods, to identify best practices and common use cases, and to reduce overhead in benchmarking, it is necessary to have standardized datasets, procedures and metrics for evaluation. In this tutorial, we describe emerging standards in computational meta-omics benchmarking derived and agreed upon by a larger community of researchers. Specifically, we outline recent efforts by the Critical Assessment of Metagenome Interpretation (CAMI) initiative, which supplies method developers and applied researchers with exhaustive quantitative data about software performance in realistic scenarios and organizes community-driven benchmarking challenges. We explain the most relevant evaluation metrics for assessing metagenome assembly, binning and profiling results, and provide step-by-step instructions on how to generate them. The instructions use simulated mouse gut metagenome data released in preparation for the second round of CAMI challenges and showcase the use of a repository of tool results for CAMI datasets. This tutorial will serve as a reference for the community and facilitate informative and reproducible benchmarking in microbiome research.
Citation
Nat Protoc. 2021 Apr;16(4):1785-1801. doi: 10.1038/s41596-020-00480-3. Epub 2021 Mar 1.
Publisher
Journal
PubMed ID
PubMed Central ID
Additional Links
Embedded video
Type
Review
Language
en
Description
Series/Report no.
ISSN
EISSN
1750-2799
ISBN
ISMN
Gov't Doc #
Sponsors
License
Attribution 4.0 International