View Categories

Evidence-Based Medicine

38 min read

Introduction #

Evidence-based medicine (EBM) represents a paradigm shift in clinical practice that integrates the best available research evidence with clinical expertise and patient values to guide medical decision-making [1,2]. As a core component of health systems science, EBM provides the methodological foundation for delivering high-quality, cost-effective healthcare while reducing practice variation and improving patient outcomes [3,4]. The practice of EBM requires physicians to develop competencies in formulating clinical questions, systematically searching medical literature, critically appraising evidence, and applying findings to individual patient care [5,6].

The integration of EBM into health systems science reflects a broader understanding that optimal patient care depends not only on biological and clinical knowledge but also on understanding healthcare delivery systems, population health, and the translation of evidence into practice [7,8]. This approach acknowledges that clinical decisions occur within complex health systems that influence resource allocation, access to care, and implementation of evidence-based interventions [9,10].

Historical Development and Evolution #

The modern EBM movement emerged in the early 1990s at McMaster University, though its philosophical roots trace back to earlier efforts to ground medical practice in scientific evidence [11,12]. David Sackett and colleagues formally defined EBM as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” [1]. This definition emphasized the integration of individual clinical expertise with the best available external clinical evidence from systematic research [13].

The evolution of EBM has paralleled advances in clinical epidemiology, biostatistics, and information technology that have made systematic evidence synthesis and dissemination increasingly feasible [14,15]. The establishment of the Cochrane Collaboration in 1993 provided infrastructure for producing and maintaining systematic reviews of healthcare interventions [16]. Subsequent developments have included the proliferation of clinical practice guidelines, the GRADE (Grading of Recommendations Assessment, Development and Evaluation) system for evaluating evidence quality, and electronic health record integration of decision support tools [17,18,19].

Contemporary EBM has evolved beyond its initial focus on individual clinical decision-making to encompass health policy, healthcare management, and population health interventions [20,21]. This expansion reflects recognition that evidence-based approaches must address not only “what works” but also questions of cost-effectiveness, implementation feasibility, and health equity [22,23].

The Five-Step EBM Process #

Step 1: Formulating Answerable Clinical Questions

The foundation of EBM practice lies in converting information needs arising from patient care into focused, answerable questions [24,25]. The PICO framework (Patient/Population, Intervention, Comparison, Outcome) provides a structured approach to question formulation that facilitates efficient literature searching and critical appraisal [26,27]. Well-formulated questions specify the patient population or problem of interest, the intervention or exposure being considered, relevant comparison groups, and outcomes of importance [28].

Clinical questions can be categorized as background questions, which address general knowledge about conditions or interventions, or foreground questions, which address specific knowledge needed for clinical decisions [29]. Foreground questions may concern therapy, diagnosis, prognosis, harm, or prevention [30]. The specificity and clarity of question formulation directly influence the efficiency of subsequent evidence searching and the applicability of findings to patient care [31].

Step 2: Searching for the Best Evidence

Effective evidence searching requires familiarity with bibliographic databases, search strategies, and the hierarchy of evidence sources [32,33]. Primary databases for clinical evidence include MEDLINE/PubMed, Embase, and the Cochrane Library, each with distinct coverage and indexing approaches [34]. Pre-appraised evidence resources, such as systematic reviews, clinical practice guidelines, and evidence summaries, offer efficient access to synthesized evidence for time-constrained clinicians [35,36].

Search strategies should balance sensitivity (retrieving all relevant studies) with specificity (excluding irrelevant studies) based on the clinical question and available time [37]. The use of medical subject headings, Boolean operators, and search filters can improve search efficiency [38]. Increasingly, artificial intelligence and machine learning tools are being developed to assist with evidence identification and screening [39,40].

The challenge of information overload in medicine, with millions of articles published annually, necessitates strategic approaches to evidence searching [41]. Clinicians must develop skills in identifying high-quality secondary sources and understanding the trade-offs between comprehensive searching and pragmatic evidence retrieval in clinical practice [42,43].

Step 3: Critical Appraisal of Evidence

Critical appraisal involves systematically evaluating research studies for validity, reliability, and applicability to clinical practice [44,45]. This process requires understanding research methodology, potential sources of bias, and statistical analysis [46]. Key considerations include the appropriateness of study design for the research question, adequacy of sample size, completeness of follow-up, and potential for confounding or systematic error [47,48].

For therapy questions, randomized controlled trials represent the gold standard design for minimizing bias and establishing causal relationships [49,50]. Critical appraisal of treatment studies examines randomization methods, allocation concealment, blinding, intention-to-treat analysis, and similarity of treatment groups at baseline [51]. For diagnostic studies, assessment focuses on spectrum of disease, reference standard validity, blinding of test interpretation, and verification bias [52,53].

Understanding measures of treatment effect, including relative risk, odds ratios, absolute risk reduction, and numbers needed to treat, enables clinicians to interpret study results and communicate findings to patients [54,55]. Similarly, familiarity with diagnostic test characteristics such as sensitivity, specificity, likelihood ratios, and predictive values is essential for evaluating diagnostic accuracy studies [56,57].

Step 4: Applying Evidence to Patient Care

The application of evidence to individual patients requires integrating research findings with clinical expertise and patient preferences and values [58,59]. This step acknowledges that research evidence provides probabilistic information about groups that must be contextualized for individual decision-making [60]. Clinicians must assess whether study populations are sufficiently similar to their patient, whether relevant patient-important outcomes were measured, and whether potential benefits outweigh harms in the specific clinical context [61,62].

Shared decision-making represents a key mechanism for integrating evidence with patient values, particularly when multiple reasonable management options exist or when interventions involve significant trade-offs between benefits and harms [63,64]. Effective communication of evidence to patients requires translating statistical concepts into understandable formats and eliciting patient preferences regarding outcomes [65,66].

External factors influencing evidence application include healthcare system resources, institutional policies, reimbursement structures, and access to interventions [67]. Implementation science has emerged as a discipline focused on understanding and addressing barriers to translating evidence into routine clinical practice [68,69].

Step 5: Evaluating Performance

The final step involves reflecting on the EBM process and outcomes to identify areas for improvement [70]. This includes assessing the efficiency of question formulation and searching, the appropriateness of evidence selection, and the impact of evidence-based decisions on patient outcomes [71]. At the individual level, self-assessment and continuing medical education support ongoing development of EBM competencies [72].

At the health system level, evaluation of evidence-based practice involves monitoring adherence to clinical practice guidelines, tracking quality metrics, and identifying opportunities for performance improvement [73,74]. Learning health systems represent an evolving model that integrates routine collection of clinical data with continuous quality improvement and generation of new knowledge [75,76].

Hierarchy of Evidence and Study Designs #

Systematic Reviews and Meta-Analyses

Systematic reviews synthesize all available evidence on a specific question using explicit, reproducible methods to identify, select, and critically appraise relevant studies [77,78]. Meta-analysis applies statistical techniques to pool results across studies, potentially increasing statistical power and precision of effect estimates [79]. The systematic review process includes comprehensive literature searching, duplicate screening and data extraction, quality assessment, and synthesis of findings [80,81].

The quality of systematic reviews depends on methodological rigor, including efforts to identify unpublished studies and minimize publication bias [82]. Assessment of heterogeneity across studies helps determine the appropriateness of pooling results and guides exploration of potential sources of variation in treatment effects [83,84]. Network meta-analysis extends traditional pairwise meta-analysis to enable indirect comparisons between interventions that have not been directly compared in trials [85,86].

Randomized Controlled Trials

Randomized controlled trials (RCTs) minimize selection bias and confounding through random allocation of participants to intervention groups [87,88]. The randomization process ensures that known and unknown prognostic factors are, on average, equally distributed between groups, allowing causal inference about treatment effects [89]. Key methodological features that enhance RCT validity include allocation concealment, blinding of participants and outcome assessors, and intention-to-treat analysis [90,91].

Pragmatic trials represent a design variant that evaluates interventions under real-world conditions to inform clinical practice and policy decisions [92,93]. These studies prioritize external validity and applicability by using broad inclusion criteria, flexible intervention protocols, and clinically relevant outcomes [94]. Cluster randomized trials, which randomize groups rather than individuals, are particularly useful for evaluating health system interventions and implementation strategies [95,96].

Observational Studies

Observational studies, including cohort studies, case-control studies, and cross-sectional studies, play important roles in EBM despite greater susceptibility to bias than randomized trials [97,98]. Cohort studies follow defined populations over time to examine associations between exposures and outcomes, making them well-suited for studying prognosis, long-term treatment effects, and rare exposures [99,100]. Case-control studies efficiently investigate rare outcomes by comparing individuals with and without the outcome regarding past exposures [101,102].

Observational studies provide essential evidence when randomized trials are unethical, infeasible, or insufficient for addressing important clinical questions [103]. Advanced analytical methods, including propensity score matching, instrumental variable analysis, and regression discontinuity designs, aim to reduce confounding in observational research [104,105]. High-quality observational studies using rigorous methods can sometimes provide effect estimates similar to those from randomized trials [106,107].

Diagnostic Accuracy Studies

Studies of diagnostic test accuracy compare index test results with a reference standard to determine sensitivity, specificity, and predictive values [108,109]. Cross-sectional or cohort designs are typically employed, with emphasis on representing an appropriate spectrum of disease severity and including relevant differential diagnoses [110]. Methodological quality considerations include use of appropriate reference standards, blinding of test interpretation, and complete verification of test results [111,112].

The STARD (Standards for Reporting of Diagnostic Accuracy) statement provides guidelines for reporting diagnostic accuracy studies to facilitate critical appraisal and evidence synthesis [113]. Hierarchical summary ROC curves and bivariate meta-analysis methods enable pooling of diagnostic accuracy data across studies while accounting for potential threshold effects and correlation between sensitivity and specificity [114,115].

Critical Appraisal and Biostatistics #

Understanding Risk and Effect Measures

Interpretation of clinical research requires familiarity with measures of association and treatment effect [116]. Relative measures, including relative risk and odds ratios, express the ratio of event rates between exposure groups [117,118]. Absolute measures, such as absolute risk reduction and risk difference, quantify the actual difference in event rates and provide information about clinical impact [119]. The number needed to treat represents the number of patients who must receive an intervention to prevent one additional adverse outcome and facilitates communication of treatment effects [120,121].

Understanding the distinction between relative and absolute effects is crucial for evidence interpretation, as interventions producing large relative risk reductions may have minimal absolute benefits when applied to low-risk populations [122]. Conversely, modest relative effects can translate into substantial absolute benefits in high-risk groups [123]. Baseline risk strongly influences the absolute benefit of interventions and should inform treatment decisions [124,125].

Confidence Intervals and Statistical Significance

Confidence intervals provide a range of plausible values for the true effect size and convey both the magnitude of effect and the precision of the estimate [126,127]. The 95% confidence interval is conventionally interpreted as the range within which the true effect lies with 95% probability, though this interpretation requires careful qualification [128]. Narrow confidence intervals indicate precise estimates, while wide intervals suggest greater uncertainty [129].

P-values indicate the probability of observing results as extreme as those obtained if the null hypothesis of no effect were true [130]. The conventional threshold of p<0.05 for statistical significance is arbitrary and does not necessarily indicate clinical importance [131,132]. Growing awareness of limitations of p-values and null hypothesis significance testing has prompted recommendations to emphasize effect sizes, confidence intervals, and contextual interpretation over dichotomous significance testing [133,134].

Bias and Confounding

Bias refers to systematic errors in study design, conduct, or analysis that lead to distortion of results [135,136]. Selection bias arises when study participants differ systematically from the target population or when comparison groups differ in ways unrelated to the exposure of interest [137]. Information bias occurs when measurement of exposures or outcomes differs systematically between study groups [138]. Performance bias, detection bias, and attrition bias represent specific threats to validity in clinical trials [139,140].

Confounding occurs when an extraneous factor is associated with both the exposure and outcome, leading to apparent associations that do not reflect causal relationships [141,142]. Randomization prevents confounding in experimental studies by balancing both measured and unmeasured confounders across treatment groups [143]. In observational studies, restriction, matching, stratification, and multivariable adjustment aim to control for confounding [144,145].

Power and Sample Size

Statistical power represents the probability that a study will detect a true effect of a specified magnitude [146]. Inadequate power increases the risk of false-negative results and limits the information value of research [147,148]. Sample size calculations, performed during study planning, consider the anticipated effect size, variability in outcomes, desired power, and significance level [149]. Underpowered studies waste resources and potentially expose participants to risks without sufficient potential for generating reliable evidence [150].

Grading Evidence Quality #

The GRADE System

The GRADE system provides a structured framework for assessing evidence quality and formulating recommendations in clinical practice guidelines [17,18]. GRADE evaluates evidence quality based on study design, risk of bias, inconsistency, indirectness, imprecision, and other considerations including publication bias and dose-response relationships [18]. Evidence quality is rated as high, moderate, low, or very low, reflecting confidence that the true effect lies close to the estimate [18].

High-quality evidence from randomized trials may be downgraded based on limitations in study design, important inconsistency between studies, indirectness (differences between research populations and clinical questions of interest), imprecision of results, or high probability of publication bias [17]. Conversely, observational evidence may be upgraded when effect sizes are very large, dose-response gradients are present, or plausible confounding would reduce apparent effects [17].

Recommendations in GRADE-based guidelines are characterized as strong or conditional based on evidence quality, balance of benefits and harms, patient values and preferences, and resource implications [17]. Strong recommendations indicate that most informed individuals would choose the recommended course of action, while conditional recommendations acknowledge that different choices will be appropriate for different patients [17].

Assessing Study Quality and Risk of Bias

Multiple tools exist for assessing risk of bias in different study designs [17]. The Cochrane Risk of Bias tool evaluates randomized trials across domains including random sequence generation, allocation concealment, blinding, incomplete outcome data, selective reporting, and other potential biases [90]. The revised version (RoB 2) provides a more structured approach with signaling questions and algorithms for deriving risk of bias judgments [90].

For observational studies, tools such as ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) assess bias domains including confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurement of outcomes, and selective reporting [104]. Quality assessment informs evidence synthesis and helps identify studies at high risk of bias that may require exclusion or sensitivity analysis [17].

Clinical Practice Guidelines #

Guideline Development and Evaluation

Clinical practice guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate healthcare for specific clinical circumstances [9,10]. High-quality guidelines are based on systematic reviews of evidence, involve multidisciplinary panels including patient representatives, explicitly link recommendations to supporting evidence, and undergo rigorous peer review [9].

The AGREE II (Appraisal of Guidelines for Research and Evaluation) instrument provides standardized criteria for evaluating guideline quality across domains including scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence [9]. Poor performance on these quality criteria can undermine guideline validity and limit implementation [9].

Guideline development processes increasingly recognize the importance of addressing equity considerations, resource implications, and implementation feasibility [9]. Transparent reporting of conflicts of interest and funding sources is essential for assessing potential bias in guideline recommendations [9]. Regular updating of guidelines is necessary to incorporate new evidence and maintain currency [9].

Implementation and Adherence

Evidence-based guidelines only improve patient outcomes when successfully implemented in clinical practice [73,74]. Barriers to guideline adherence operate at multiple levels, including individual clinician factors (knowledge, attitudes, behavior), organizational factors (resources, culture, workflows), and system factors (policies, reimbursement) [73]. Effective implementation strategies address these multilevel barriers through education, audit and feedback, decision support systems, financial incentives, and organizational change [67,68].

The Choosing Wisely campaign represents an effort to promote appropriate use of evidence-based interventions while reducing low-value care through specialty society identification of commonly overused tests and treatments [74]. De-implementation of ineffective or harmful practices poses distinct challenges from implementation of new interventions and requires attention to disinvestment strategies [74].

Shared Decision-Making #

Principles and Process

Shared decision-making embodies the application of EBM to individual patient care by integrating best evidence with patient preferences and values [63,64]. This process is particularly important for preference-sensitive decisions where multiple reasonable options exist with different profiles of benefits and harms [63]. Core elements of shared decision-making include information exchange about options and outcomes, deliberation about preferences, and collaborative decision-making [63,64].

Effective shared decision-making requires clinicians to present evidence in understandable formats, elicit and incorporate patient preferences, and support patients in making informed choices consistent with their values [63,64]. This approach respects patient autonomy while ensuring decisions are informed by best available evidence [63]. Decision aids are structured tools that present evidence about options and help patients clarify preferences [63,64].

Communicating Risk and Uncertainty

Communicating medical evidence to patients requires translation of statistical concepts into formats that support informed decision-making [65,66]. Natural frequencies and absolute risk reductions are generally better understood than relative risks and percentages [65]. Visual aids, including pictographs and icon arrays, can improve risk comprehension compared to numerical presentation alone [65,66].

Framing effects, whereby presentation of equivalent information in terms of gains versus losses influences choices, highlight the importance of balanced presentation [65]. Clinicians should acknowledge uncertainty in evidence, help patients understand the quality and limitations of available information, and support decisions that reflect individual circumstances and preferences [66].

Implementation Science and Learning Health Systems #

Knowledge Translation

Knowledge translation encompasses the process of moving research evidence into clinical practice and health policy [68,69]. This field recognizes that passive dissemination of evidence is insufficient and that active implementation strategies are needed to change practice [68]. The “know-do gap” describes the consistent lag between generation of evidence and its application in routine care [68,69].

Theoretical frameworks in implementation science, including the Consolidated Framework for Implementation Research and the Theoretical Domains Framework, identify determinants of implementation success and guide strategy selection [68]. Common implementation strategies include educational meetings, audit and feedback, clinical decision support, local opinion leaders, and quality improvement collaboratives [67,68].

Learning Health Systems

Learning health systems integrate clinical care, research, quality improvement, and innovation to continuously improve health outcomes and value [75,76]. Key features include embedded data generation through electronic health records, continuous monitoring of outcomes, rapid-cycle evaluation of interventions, and systematic application of insights to improve care [75,76]. This model aims to make evidence generation and application a natural byproduct of healthcare delivery rather than separate activities [75,76].

Pragmatic clinical trials conducted within learning health systems offer efficient mechanisms for generating patient-centered evidence while minimizing the gap between research and practice [92,93]. Integration of real-world data from electronic health records with randomized trial designs enables large-scale comparative effectiveness research at reduced cost [75,76].

Limitations and Criticisms of EBM #

Challenges in Evidence Application

Despite its benefits, EBM faces important limitations and criticisms [14,15]. The evidence base is incomplete for many clinical questions, particularly in populations underrepresented in research, including children, older adults, pregnant women, and individuals with multiple comorbidities [60,61]. Lack of directly applicable evidence necessitates extrapolation and clinical judgment [60].

The time required for practicing EBM conflicts with clinical workflow pressures and limited consultation time [42,43]. While pre-appraised resources and clinical decision support systems aim to address this barrier, ensuring timely access to synthesized evidence remains challenging [35,36]. Additionally, commercial and specialty bias in research funding leads to gaps in evidence for interventions with limited profit potential [82].

Threats to Evidence Integrity

Publication bias, selective reporting of outcomes, and inappropriate data analysis threaten evidence validity [82]. Trials with positive results are more likely to be published and published more rapidly than those with negative or null findings [82]. Industry-sponsored research may be subject to bias through study design choices, outcome selection, and selective publication [82].

Research waste from poorly designed studies, redundant research, inadequate reporting, and failure to incorporate systematic reviews undermines the efficiency and validity of the evidence base [82]. Questionable research practices, including p-hacking and outcome switching, further compromise evidence reliability [133,134]. Reforms to improve research integrity include trial registration, prospective specification of analysis plans, data sharing, and improved statistical practices [133,134].

The Art and Science of Medicine

Critics argue that over-reliance on population-level evidence may undervalue clinical expertise, mechanistic reasoning, and individual patient characteristics [14,15]. The emphasis on randomized trials as the gold standard for evidence has been challenged when biological understanding strongly supports interventions or when randomization is unethical [14]. Some contend that EBM promotes cookbook medicine and diminishes the role of clinical judgment [14,15].

Proponents respond that EBM explicitly integrates clinical expertise with research evidence and patient values rather than replacing judgment with algorithms [1,2]. The appropriate role of EBM is to inform but not dictate clinical decisions, acknowledging that evidence provides probabilities that must be contextualized for individual patients [1]. Ongoing dialogue about the scope and limitations of EBM continues to refine its practice and teaching [14,15].

Teaching and Learning EBM #

Competencies and Curricula

EBM competencies for physicians include formulating answerable questions, efficiently searching medical literature, critically appraising evidence, applying findings to patient care, and evaluating performance [5,6]. Medical education increasingly integrates EBM throughout training rather than teaching it as a separate subject [5]. Effective EBM education combines didactic instruction in concepts and methods with clinical application and practice-based learning [5,6].

Vertical integration of EBM across undergraduate medical education, residency training, and continuing professional development supports progressive skill development [5,6]. Assessment of EBM competencies includes written tests of knowledge, evaluation of critical appraisal skills through journal clubs, and observation of evidence application in clinical practice [5,6].

Educational Innovations

Innovations in EBM education include flipped classroom approaches, online learning modules, simulation-based learning, and integration with clinical decision support systems [5,6]. Point-of-care teaching that addresses clinical questions arising during patient care provides authentic learning opportunities and models EBM application [5,6].

Development of online evidence resources and mobile applications has made evidence more accessible to learners and practicing clinicians [35,36]. However, teaching effective use of these tools and maintaining critical appraisal skills remain important educational goals [5,6]. Future directions include leveraging artificial intelligence to support evidence searching and synthesis while ensuring clinicians retain fundamental EBM competencies [39,40].

Future Directions #

The future of EBM will be shaped by advances in data science, precision medicine, implementation science, and health systems transformation [75,76]. Integration of genomic information, biomarkers, and electronic health record data promises more personalized evidence-based care [75,76]. Machine learning and natural language processing may automate evidence synthesis and enable real-time updating of practice guidelines [39,40].

Expanding focus on patient-reported outcomes, comparative effectiveness research, and health equity will broaden the scope of EBM beyond traditional efficacy questions [22,23]. The global nature of health challenges requires attention to evidence applicability across diverse healthcare contexts and resource settings [22,23]. Continued evolution of EBM methodology, education, and implementation will be essential to realizing its potential for improving health outcomes and health system performance [1,2].

Conclusion #

Evidence-based medicine represents a fundamental framework for high-quality healthcare delivery within modern health systems [1,2]. By systematically integrating research evidence, clinical expertise, and patient preferences, EBM provides a rigorous approach to medical decision-making that reduces unwarranted practice variation and improves patient outcomes [3,4]. The five-step EBM process of formulating questions, searching for evidence, critically appraising studies, applying findings, and evaluating performance equips clinicians with essential competencies for lifelong learning and practice improvement [5,6].

As a core domain of health systems science, EBM intersects with quality improvement, patient safety, healthcare delivery, and population health to optimize care at individual and system levels [7,8]. Challenges including incomplete evidence, time constraints, and threats to research integrity require ongoing attention and methodological innovation [14,15]. The future of EBM will increasingly leverage technological advances while maintaining commitment to the foundational principles of evidence-informed, patient-centered care [1,2].

References #

  1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ. 1996;312(7023):71-72. https://doi.org/10.1136/bmj.312.7023.71
  2. Guyatt G, Cairns J, Churchill D, et al. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420-2425. https://doi.org/10.1001/jama.1992.03490170092032
  3. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. 2017;390(10092):415-423. https://doi.org/10.1016/S0140-6736(16)31592-6
  4. Brownson RC, Baker EA, Deshpande AD, Gillespie KN. Evidence-based public health. 3rd ed. Oxford University Press; 2017. https://doi.org/10.1093/oso/9780190620936.001.0001
  5. Albarqouni L, Hoffmann T, Glasziou P. Evidence-based practice educational intervention studies: a systematic review of what is taught and how it is measured. BMC Med Educ. 2018;18(1):177. https://doi.org/10.1186/s12909-018-1284-1
  6. Straus SE, Glasziou P, Richardson WS, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. 5th ed. Elsevier; 2018. https://doi.org/10.1016/B978-0-7020-6296-4.00001-4
  7. Skochelak SE, Hawkins RE, Lawson LE, Starr SR, Borkan JM, Gonzalo JD. Health Systems Science. 2nd ed. Elsevier; 2020. https://doi.org/10.1016/B978-0-323-69496-8.00001-6
  8. Gonzalo JD, Dekhtyar M, Starr SR, et al. Health systems science curricula in undergraduate medical education: identifying and defining a potential curricular framework. Acad Med. 2017;92(1):123-131. https://doi.org/10.1097/ACM.0000000000001177
  9. Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, eds. Clinical Practice Guidelines We Can Trust. Institute of Medicine. National Academies Press; 2011. https://doi.org/10.17226/13058
  10. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001. https://doi.org/10.17226/10027
  11. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420-2425. https://doi.org/10.1001/jama.268.17.2420
  12. Claridge JA, Fabian TC. History and development of evidence-based medicine. World J Surg. 2005;29(5):547-553. https://doi.org/10.1007/s00268-005-7910-1
  13. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd ed. Churchill Livingstone; 2000. https://doi.org/10.1016/B978-0-443-06240-8.50001-9
  14. Greenhalgh T, Howick J, Maskrey N. Evidence based medicine: a movement in crisis? BMJ. 2014;348:g3725. https://doi.org/10.1136/bmj.g3725
  15. Masic I, Miokovic M, Muhamedagic B. Evidence based medicine – new approaches and challenges. Acta Inform Med. 2008;16(4):219-225. https://doi.org/10.5455/aim.2008.16.219-225
  16. Higgins JPT, Thomas J, Chandler J, et al., eds. Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. John Wiley & Sons; 2019. https://doi.org/10.1002/9781119536604
  17. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. https://doi.org/10.1136/bmj.39489.470347.AD
  18. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ. 2004;328(7454):1490. https://doi.org/10.1136/bmj.328.7454.1490
  19. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. https://doi.org/10.1136/bmj.38398.500764.8F
  20. Brownson RC, Fielding JE, Green LW. Building capacity for evidence-based public health: reconciling the pulls of practice and the push of research. Annu Rev Public Health. 2018;39:27-53. https://doi.org/10.1146/annurev-publhealth-040617-014746
  21. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012;7:50. https://doi.org/10.1186/1748-5908-7-50
  22. Neumann PJ, Sanders GD, Russell LB, Siegel JE, Ganiats TG. Cost-Effectiveness in Health and Medicine. 2nd ed. Oxford University Press; 2016. https://doi.org/10.1093/acprof:oso/9780190492939.001.0001
  23. Braveman P, Gottlieb L. The social determinants of health: it’s time to consider the causes of the causes. Public Health Rep. 2014;129 Suppl 2(Suppl 2):19-31. https://doi.org/10.1177/00333549141291S206
  24. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995;123(3):A12-13. https://doi.org/10.7326/ACPJC-1995-123-3-A12
  25. Huang X, Lin J, Demner-Fushman D. Evaluation of PICO as a knowledge representation for clinical questions. AMIA Annu Symp Proc. 2006;2006:359-363. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839740/
  26. Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inform Decis Mak. 2007;7:16. https://doi.org/10.1186/1472-6947-7-16
  27. Stone PW. Popping the (PICO) question in research and evidence-based practice. Appl Nurs Res. 2002;15(3):197-198. https://doi.org/10.1053/apnr.2002.34181
  28. Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med. 1997;127(5):380-387. https://doi.org/10.7326/0003-4819-127-5-199709010-00008
  29. Booth A. Clear and present questions: formulating questions for evidence based practice. Library Hi Tech. 2006;24(3):355-368. https://doi.org/10.1108/07378830610692127
  30. Guyatt GH, Sackett DL, Cook DJ. Users’ guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA. 1993;270(21):2598-2601. https://doi.org/10.1001/jama.1993.03510210084032
  31. Riva JJ, Malik KM, Burnie SJ, Endicott AR, Busse JW. What is your research question? An introduction to the PICOT format for clinicians. J Can Chiropr Assoc. 2012;56(3):167-171. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3430448/
  32. Lefebvre C, Glanville J, Briscoe S, et al. Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, et al., eds. Cochrane Handbook for Systematic Reviews of Interventions. 2nd ed. John Wiley & Sons; 2019:67-107. https://doi.org/10.1002/9781119536604.ch4
  33. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40-46. https://doi.org/10.1016/j.jclinepi.2016.01.021
  34. Bramer WM, Rethlefsen ML, Kleijnen J, Franco OH. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev. 2017;6(1):245. https://doi.org/10.1186/s13643-017-0644-y
  35. Haynes RB. Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. Evid Based Med. 2006;11(6):162-164. https://doi.org/10.1136/ebm.11.6.162-a
  36. DiCenso A, Bayley L, Haynes RB. Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model. Evid Based Nurs. 2009;12(4):99-101. https://doi.org/10.1136/ebn.12.4.99-b
  37. Bachmann LM, Coray R, Estermann P, Ter Riet G. Identifying diagnostic studies in MEDLINE: reducing the number needed to read. J Am Med Inform Assoc. 2002;9(6):653-658. https://doi.org/10.1197/jamia.M1124
  38. Wilczynski NL, Haynes RB. Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey. BMC Med. 2004;2:23. https://doi.org/10.1186/1741-7015-2-23
  39. O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. https://doi.org/10.1186/2046-4053-4-5
  40. Marshall IJ, Wallace BC. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev. 2019;8(1):163. https://doi.org/10.1186/s13643-019-1074-9
  41. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. https://doi.org/10.1371/journal.pmed.1000326
  42. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147(4):224-233. https://doi.org/10.7326/0003-4819-147-4-200708210-00179
  43. Alonso-Coello P, Schünemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016;353:i2016. https://doi.org/10.1136/bmj.i2016
  44. Guyatt GH, Oxman AD, Sultan S, et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol. 2011;64(12):1311-1316. https://doi.org/10.1016/j.jclinepi.2011.06.004
  45. Greenhalgh T. How to read a paper: the basics of evidence-based medicine and healthcare. 6th ed. Wiley-Blackwell; 2019. https://doi.org/10.1002/9781119484653
  46. Glasziou P, Irwig L, Bain C, Colditz G. Systematic Reviews in Health Care: A Practical Guide. Cambridge University Press; 2001. https://doi.org/10.1017/CBO9780511543500
  47. Juni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;323(7303):42-46. https://doi.org/10.1136/bmj.323.7303.42
  48. Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407-415. https://doi.org/10.1016/j.jclinepi.2010.07.017
  49. Akobeng AK. Understanding randomised controlled trials. Arch Dis Child. 2005;90(8):840-844. https://doi.org/10.1136/adc.2004.058222
  50. Kendall JM. Designing a research project: randomised controlled trials and their principles. Emerg Med J. 2003;20(2):164-168. https://doi.org/10.1136/emj.20.2.164
  51. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408-412. https://doi.org/10.1001/jama.1995.03520290060030
  52. Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
  53. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282(11):1061-1066. https://doi.org/10.1001/jama.282.11.1061
  54. Nuovo J, Melnikow J, Chang D. Reporting number needed to treat and absolute risk reduction in randomized controlled trials. JAMA. 2002;287(21):2813-2814. https://doi.org/10.1001/jama.287.21.2813
  55. Citrome L, Ketter TA. When does a difference make a difference? Interpretation of number needed to treat, number needed to harm, and likelihood to be helped or harmed. Int J Clin Pract. 2013;67(5):407-411. https://doi.org/10.1111/ijcp.12142
  56. Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329(7458):168-169. https://doi.org/10.1136/bmj.329.7458.168
  57. Jaeschke R, Guyatt GH, Sackett DL. Users’ guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? JAMA. 1994;271(9):703-707. https://doi.org/10.1001/jama.1994.03510330081039
  58. Haynes RB, Devereaux PJ, Guyatt GH. Clinical expertise in the era of evidence-based medicine and patient choice. ACP J Club. 2002;136(2):A11-14. https://doi.org/10.7326/ACPJC-2002-136-2-A11
  59. Straus SE, McAlister FA. Evidence-based medicine: a commentary on common criticisms. CMAJ. 2000;163(7):837-841. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC80509/
  60. Rothwell PM. External validity of randomised controlled trials: “to whom do the results of this trial apply?” Lancet. 2005;365(9453):82-93. https://doi.org/10.1016/S0140-6736(04)17670-8
  61. Guyatt GH, Haynes RB, Jaeschke RZ, et al. Users’ guides to the medical literature: XXV. Evidence-based medicine: principles for applying the Users’ Guides to patient care. JAMA. 2000;284(10):1290-1296. https://doi.org/10.1001/jama.284.10.1290
  62. Dans AL, Dans LF, Guyatt GH, Richardson S. Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. JAMA. 1998;279(7):545-549. https://doi.org/10.1001/jama.279.7.545
  63. Elwyn G, Frosch D, Thomson R, et al. Shared decision making: a model for clinical practice. J Gen Intern Med. 2012;27(10):1361-1367. https://doi.org/10.1007/s11606-012-2077-6
  64. Charles C, Gafni A, Whelan T. Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med. 1997;44(5):681-692. https://doi.org/10.1016/S0277-9536(96)00221-3
  65. Trevena LJ, Zikmund-Fisher BJ, Edwards A, et al. Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers. BMC Med Inform Decis Mak. 2013;13 Suppl 2(Suppl 2):S7. https://doi.org/10.1186/1472-6947-13-S2-S7
  66. Hoffmann TC, Del Mar C. Patients’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med. 2015;175(2):274-286. https://doi.org/10.1001/jamainternmed.2014.6016
  67. Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139. https://doi.org/10.1186/1748-5908-8-139
  68. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3:32. https://doi.org/10.1186/s40359-015-0089-9
  69. Eccles MP, Mittman BS. Welcome to implementation science. Implement Sci. 2006;1:1. https://doi.org/10.1186/1748-5908-1-1
  70. Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ. 2004;329(7473):1017. https://doi.org/10.1136/bmj.329.7473.1017
  71. Green ML. Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicine: a critical review of curricula. Acad Med. 1999;74(6):686-694. https://doi.org/10.1097/00001888-199906000-00017
  72. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006;296(9):1094-1102. https://doi.org/10.1001/jama.296.9.1094
  73. Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458-1465. https://doi.org/10.1001/jama.282.15.1458
  74. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225-1230. https://doi.org/10.1016/S0140-6736(03)14546-1
  75. Institute of Medicine. The Learning Healthcare System: Workshop Summary. National Academies Press; 2007. https://doi.org/10.17226/11903
  76. Friedman CP, Rubin JC, Sullivan KJ. Toward an information infrastructure for global health improvement. Yearb Med Inform. 2017;26(1):16-23. https://doi.org/10.15265/IY-2017-004
  77. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097
  78. Petticrew M, Roberts H. Systematic Reviews in the Social Sciences: A Practical Guide. Blackwell Publishing; 2006. https://doi.org/10.1002/9780470754887
  79. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. John Wiley & Sons; 2009. https://doi.org/10.1002/9780470743386
  80. Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557-560. https://doi.org/10.1136/bmj.327.7414.557
  81. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008
  82. Sterne JAC, Sutton AJ, Ioannidis JPA, et al. Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ. 2011;343:d4002. https://doi.org/10.1136/bmj.d4002
  83. Thompson SG, Higgins JPT. How should meta-regression analyses be undertaken and interpreted? Stat Med. 2002;21(11):1559-1573. https://doi.org/10.1002/sim.1187
  84. Ioannidis JPA, Patsopoulos NA, Evangelou E. Uncertainty in heterogeneity estimates in meta-analyses. BMJ. 2007;335(7626):914-916. https://doi.org/10.1136/bmj.39343.408449.80
  85. Caldwell DM, Ades AE, Higgins JPT. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ. 2005;331(7521):897-900. https://doi.org/10.1136/bmj.331.7521.897
  86. Salanti G, Higgins JPT, Ades AE, Ioannidis JPA. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008;17(3):279-301. https://doi.org/10.1177/0962280207080643
  87. Sibbald B, Roland M. Understanding controlled trials: why are randomised controlled trials important? BMJ. 1998;316(7126):201. https://doi.org/10.1136/bmj.316.7126.201
  88. Hariton E, Locascio JJ. Randomised controlled trials – the gold standard for effectiveness research. BJOG. 2018;125(13):1716. https://doi.org/10.1111/1471-0528.15199
  89. Jadad AR, Enkin MW. Randomized Controlled Trials: Questions, Answers, and Musings. 2nd ed. Blackwell Publishing; 2007. https://doi.org/10.1002/9780470691922
  90. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. https://doi.org/10.1136/bmj.c332
  91. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. https://doi.org/10.1136/bmj.c869
  92. Ford I, Norrie J. Pragmatic trials. N Engl J Med. 2016;375(5):454-463. https://doi.org/10.1056/NEJMra1510059
  93. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13(2):217-224. https://doi.org/10.31887/DCNS.2011.13.2/npatsopoulos
  94. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147. https://doi.org/10.1136/bmj.h2147
  95. Campbell MK, Piaggio G, Elbourne DR, Altman DG. Consort 2010 statement: extension to cluster randomised trials. BMJ. 2012;345:e5661. https://doi.org/10.1136/bmj.e5661
  96. Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol. 2006;35(5):1292-1300. https://doi.org/10.1093/ije/dyl129
  97. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002;359(9302):248-252. https://doi.org/10.1016/S0140-6736(02)07451-2
  98. Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med J. 2003;20(1):54-60. https://doi.org/10.1136/emj.20.1.54
  99. Thiese MS. Observational and interventional study design types; an overview. Biochem Med (Zagreb). 2014;24(2):199-210. https://doi.org/10.11613/BM.2014.022
  100. Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. 2010;126(6):2234-2242. https://doi.org/10.1097/PRS.0b013e3181f44abc
  101. Schulz KF, Grimes DA. Case-control studies: research in reverse. Lancet. 2002;359(9304):431-434. https://doi.org/10.1016/S0140-6736(02)07605-5
  102. Vandenbroucke JP, Pearce N. Case-control studies: basic concepts. Int J Epidemiol. 2012;41(5):1480-1489. https://doi.org/10.1093/ije/dys147
  103. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342(25):1887-1892. https://doi.org/10.1056/NEJM200006223422507
  104. Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011;46(3):399-424. https://doi.org/10.1080/00273171.2011.568786
  105. Baiocchi M, Cheng J, Small DS. Instrumental variable methods for causal inference. Stat Med. 2014;33(13):2297-2340. https://doi.org/10.1002/sim.6128
  106. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014;2014(4):MR000034. https://doi.org/10.1002/14651858.MR000034.pub2
  107. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. 2016;352:i493. https://doi.org/10.1136/bmj.i493
  108. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:h5527. https://doi.org/10.1136/bmj.h5527
  109. Leeflang MM, Rutjes AW, Reitsma JB, Hooft L, Bossuyt PM. Variation of a test’s sensitivity and specificity with disease prevalence. CMAJ. 2013;185(11):E537-E544. https://doi.org/10.1503/cmaj.121286
  110. Rutjes AW, Reitsma JB, Di Nisio M, Smidt N, van Rijn JC, Bossuyt PM. Evidence of bias and variation in diagnostic accuracy studies. CMAJ. 2006;174(4):469-476. https://doi.org/10.1503/cmaj.050090
  111. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25. https://doi.org/10.1186/1471-2288-3-25
  112. Lijmer JG, Mol BW, Heisterkamp S, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282(11):1061-1066. https://doi.org/10.1001/jama.282.11.1061
  113. Korevaar DA, van Enst WA, Spijker R, Bossuyt PM, Hooft L. Reporting quality of diagnostic accuracy studies: a systematic review and meta-analysis of investigations on adherence to STARD. Evid Based Med. 2014;19(2):47-54. https://doi.org/10.1136/eb-2013-101637
  114. Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. https://doi.org/10.1016/j.jclinepi.2005.02.022
  115. Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, eds. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 1.0. The Cochrane Collaboration; 2010. https://doi.org/10.1002/9780470057049.ch10
  116. Ranganathan P, Aggarwal R, Pramesh CS. Common pitfalls in statistical analysis: measures of agreement. Perspect Clin Res. 2017;8(4):187-191. https://doi.org/10.4103/picr.PICR_123_17
  117. Schmidt CO, Kohlmann T. When to use the odds ratio or the relative risk? Int J Public Health. 2008;53(3):165-167. https://doi.org/10.1007/s00038-008-7068-3
  118. Altman DG, Deeks JJ, Sackett DL. Odds ratios should be avoided when events are common. BMJ. 1998;317(7168):1318. https://doi.org/10.1136/bmj.317.7168.1318
  119. Newcombe RG. Interval estimation for the difference between independent proportions: comparison of eleven methods. Stat Med. 1998;17(8):873-890. https://doi.org/10.1002/(SICI)1097-0258(19980430)17:8<873::AID-SIM779>3.0.CO;2-I
  120. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ. 1995;310(6977):452-454. https://doi.org/10.1136/bmj.310.6977.452
  121. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988;318(26):1728-1733. https://doi.org/10.1056/NEJM198806303182605
  122. Sinclair JC, Bracken MB. Clinically useful measures of effect in binary analyses of randomized trials. J Clin Epidemiol. 1994;47(8):881-889. https://doi.org/10.1016/0895-4356(94)90191-0
  123. Zipkin DA, Umscheid CA, Keating NL, et al. Evidence-based risk communication: a systematic review. Ann Intern Med. 2014;161(4):270-280. https://doi.org/10.7326/M14-0295
  124. Chatellier G, Zapletal E, Lemaitre D, Menard J, Degoulet P. The number needed to treat: a clinically useful nomogram in its proper context. BMJ. 1996;312(7028):426-429. https://doi.org/10.1136/bmj.312.7028.426
  125. McAlister FA. The “number needed to treat” turns 20–and continues to be used and misused. CMAJ. 2008;179(6):549-553. https://doi.org/10.1503/cmaj.080484
  126. Gardner MJ, Altman DG. Statistics with Confidence: Confidence Intervals and Statistical Guidelines. 2nd ed. BMJ Books; 2000. https://doi.org/10.1002/9780470173862
  127. Cumming G, Finch S. Inference by eye: confidence intervals and how to read pictures of data. Am Psychol. 2005;60(2):170-180. https://doi.org/10.1037/0003-066X.60.2.170
  128. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337-350. https://doi.org/10.1007/s10654-016-0149-3
  129. du Prel JB, Hommel G, Rohrig B, Blettner M. Confidence interval or p-value?: part 4 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2009;106(19):335-339. https://doi.org/10.3238/arztebl.2009.0335
  130. Wasserstein RL, Lazar NA. The ASA statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129-133. https://doi.org/10.1080/00031305.2016.1154108
  131. Ioannidis JPA. The proposal to lower p value thresholds to .005. JAMA. 2018;319(14):1429-1430. https://doi.org/10.1001/jama.2018.1536
  132. Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305-307. https://doi.org/10.1038/d41586-019-00857-9
  133. Wasserstein RL, Schirm AL, Lazar NA. Moving to a world beyond “p < 0.05”. Am Stat. 2019;73(sup1):1-19. https://doi.org/10.1080/00031305.2019.1583913
  134. McShane BB, Gal D, Gelman A, Robert C, Tackett JL. Abandon statistical significance. Am Stat. 2019;73(sup1):235-245. https://doi.org/10.1080/00031305.2018.1527253
  135. Delgado-Rodriguez M, Llorca J. Bias. J Epidemiol Community Health. 2004;58(8):635-641. https://doi.org/10.1136/jech.2003.008466
  136. Pannucci CJ, Wilkins EG. Identifying and avoiding bias in research. Plast Reconstr Surg. 2010;126(2):619-625. https://doi.org/10.1097/PRS.0b013e3181de24bc
  137. Haukoos JS, Lewis RJ. The propensity score. JAMA. 2015;314(15):1637-1638. https://doi.org/10.1001/jama.2015.13480
  138. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc. 2016;9:211-217. https://doi.org/10.2147/JMDH.S104807
  139. Wood L, Egger M, Gluud LL, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601-605. https://doi.org/10.1136/bmj.39465.451748.AD
  140. Savovic J, Jones HE, Altman DG, et al. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157(6):429-438. https://doi.org/10.7326/0003-4819-157-6-201209180-00537
  141. Pourhoseingholi MA, Baghestani AR, Vahedi M. How to control confounding effects by statistical analysis. Gastroenterol Hepatol Bed Bench. 2012;5(2):79-83. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4017459/
  142. Skelly AC, Dettori JR, Brodt ED. Assessing bias: the importance of considering confounding. Evid Based Spine Care J. 2012;3(1):9-12. https://doi.org/10.1055/s-0031-1298595
  143. Senn S. Seven myths of randomisation in clinical trials. Stat Med. 2013;32(9):1439-1450. https://doi.org/10.1002/sim.5713
  144. Stürmer T, Joshi M, Glynn RJ, Avorn J, Rothman KJ, Schneeweiss S. A review of the application of propensity score methods yielded increasing use, advantages in specific settings, but not substantially different estimates compared with conventional multivariable methods. J Clin Epidemiol. 2006;59(5):437-447. https://doi.org/10.1016/j.jclinepi.2005.07.004
  145. Martens EP, Pestman WR, de Boer A, Belitser SV, Klungel OH. Instrumental variables: application and limitations. Epidemiology. 2006;17(3):260-267. https://doi.org/10.1097/01.ede.0000215160.88317.cb
  146. Button KS, Ioannidis JP, Mokrysz C, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14(5):365-376. https://doi.org/10.1038/nrn3475
  147. Schulz KF, Grimes DA. Sample size calculations in randomised trials: mandatory and mystical. Lancet. 2005;365(9467):1348-1353. https://doi.org/10.1016/S0140-6736(05)61034-3
  148. Lenth RV. Some practical guidelines for effective sample size determination. Am Stat. 2001;55(3):187-193. https://doi.org/10.1198/000313001317098149
  149. Jones SR, Carley S, Harrison M. An introduction to power and sample size estimation. Emerg Med J. 2003;20(5):453-458. https://doi.org/10.1136/emj.20.5.453
  150. Halpern SD, Karlawish JH, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA. 2002;288(3):358-362. https://doi.org/10.1001/jama.288.3.358

Updated on December 11, 2025