Skip to Main Content

Northern Health Library: Critical appraisal


Would you like to:

   
 

What is critical appraisal?

Not all evidence is created equal -- critical appraisal is a systematic and careful reading of published research to see if it's robust, trustworthy, and relevant to your practice or query.

  1. Is this study valid?
  2. What are the results?
  3. Is this study relevant to my practice?

Critical Appraisal Checklists

  • CASP Critical Appraisal Skills Programme -- checklists for particular study designs with guidance and prompt questions

  • JBI Johanna Briggs Institute critical appraisal tools -- checklists for particular study designs with an emphasis on systematic review processes

  • CEBM Centre for Evidence-Based Medicine -- tools for critical appraisal; available in several languages

 

If you would like to enhance your critical appraisal by including antiracism, Naicker's supplementary checklist is useful.

For appraisal from an Indigenous perspective:

Harfield, S., et al. (2020). Assessing the quality of health research from an Indigenous perspective: the Aboriginal and Torres Strait Islander quality appraisal tool. BMC Medical Research Methodology, 20(1), 79. https://doi.org/10.1186/s12874-020-00959-3 
 

The library can offer individual or group training (in person or on MS Teams) in critical appraisal skills.

If you or your team are interested, please send us an email!

Training icons created by netscript - Flaticon

Hierarchy of Evidence

Some types of evidence are seen as stronger than others. For example, systematic reviews and meta-analyses are often thought of as the most robust and valid type of published research (although even these should be subjected to critical appraisal!).

A "hierarchy"--often in the form of a pyramid--is used to help researchers rank evidence according to validity and utility. The example below suggests that expertly appraised evidence is most useful, and that experimental studies are generally seen as more valid than observational studies or reports of expert opinion.

Source: This hierarchy of evidence pyramid is from Wikimedia Commons, under CC BY-SA 4.0 license.

Study Designs -- Pros & Cons

Different questions are best answered by different study designs. Every study design has inherent strengths and weaknesses. It can be useful to consider these as part of the appraisal process before you use a checklist.

Read about the advantages and disadvantages of study designs 

For example, while RCTs are generally thought of as high-quality evidence there are cases when an RCT is impossible and/or unethical. We can't intentionally expose humans to toxic chemicals in order to measure effects. However, we may be able to access data about populations that have been exposed to a particular toxic chemical, so we could undertake a cohort study.

Study design tree

Source: This tree of different study types is from the Centre of Evidence-Based Medicine and is available under CC BY 4.0 License.

 

 

The following books are useful guides to critical appraisal of evidence:

Testing Treatments (2nd ed., 2011) is a freely-available PDF that seeks to promote better and more critical public evaluation of the effects of medical treatments.

Critical Appraisal of AI Tools

A good starting place is the T.R.U.E. checklist -- Is It True? Is It Reproducible? Is It Useful? Is It Explainable?

This page collects a range of assessment tools that may be useful when appraising AI.

Interested in The No. 1 Question to Ask When Evaluating AI Tools?

Reddy, S. (2023). Evaluating large language models for use in healthcare: A framework for translational value assessment. Informatics in Medicine Unlocked, 41. https://doi.org/10.1016/j.imu.2023.101304