Benchmarking Basics
摘要
This chapter provides a definition of the term “benchmark” followed by definitions of the major system quality attributes that are typically subject of benchmarking. After that, a classification of the different types of benchmarks is provided, followed by an overview of strategies for performance benchmarking. Next, the quality criteria for good benchmarks are discussed in detail. Reproducibility has gained increasing attention over the past decade as a basis for realizing the principles of findability, accessibility, interoperability, and reusability (FAIR) of research data. The chapter includes a definition of reproducibility as well as differentiation from related terms such as repeatability and replicability. It further elaborates on approaches for enhancing reproducibility, including how benchmarks help to improve the reproducibility of research results. Finally, the chapter is wrapped up by a discussion of application scenarios for benchmarks.