C. Instruments for Assessing Quality of Individual Studies

A variety of assessment instruments are available to evaluate the quality of individual studies. Many of these are for assessing internal validity or risk of bias for benefits and harms; others focus on assessing external validity. These include instruments for assessing particular types of studies (e.g., RCTs, observational studies) and certain types of interventions (e.g., screening, diagnosis, and treatment).

A systematic review identified more than 20 scales (and their modifications) for assessing the quality of RCTs (Olivo 2008). Although most of these had not been rigorously developed or tested for validity and reliability, the systematic review found that one of the original scales, the Jadad Scale (Jadad 1996), shown in Box III-5, was the strongest.

The Cochrane Risk of Bias Tool for RCTs, shown in Box III-6, accounts for the domains of bias noted above (i.e., selection, performance, detection, attrition, and reporting bias), providing criteria for assessing whether there is low, unclear, or high risk of bias for each domain for individual RCTs as well as across a set of RCTs for a particular evidence question (Higgins, Altman, Sterne 2011).

Criteria and ratings for assessing internal validity of RCTs and cohort studies and of diagnostic accuracy studies used by the US Preventive Services Task Force (USPSTF) are shown in Box III-7 and Box III-8, respectively. Box III-9 shows a framework used by the USPSTF to rate the external validity of individual studies. QUADAS-2 is a quality assessment tool for diagnostic accuracy studies (Whiting 2011).

Among their numerous instruments for assessing the quality of individual studies, the AHRQ EPCs use a PICOS framework to organize characteristics that can affect the external validity of individual studies, which are used as criteria for evaluating study quality for internal validity, as shown in Box III-10.