Critical Appraisal: Assessing the Quality of Evidence

Health and social care decisions should be made using the best available evidence. It is unlikely that anyone would disagree with that statement, but it is not always so clear what is meant by this. When reviewing literature there is one overriding question—are the results likely to be correct? Answering this question for each study will help the reviewer decide if the study should be included or not, and how strong the overall body of evidence is. Traditional approaches have employed fixed hierarchies of evidence and general quality appraisal tools. Recently, there has been a move towards a more nuanced method of assessing the risk of bias and risks to rigour. These should be appraised using the standard tools described in this chapter. It is also important to understand the hierarchical nature of reviews, and that the concepts described here apply to each study individually, but also to the review itself. Writing any kind of review consists of more than just reading a large pile of papers and summarising the results. If it were that easy, you would not need books such as this. However, reading and really understanding the papers is a pre-requisite for the next step, which is looking at the papers individually and as a group (what we will from now on refer to as ‘the body of literature’). There are a number of terms that are often used interchangeably, including critical appraisal, the more general term ‘critique’, and various versions of assessment of (methodological) quality. These terms are however different, and while each is important they have a role at different points in the review process. In this chapter, we will use a general term ‘appraisal’ to cover all of these.