Scientific Clinical Decision-making & Practice
The University of Arizona Department of Pediatrics Residency Training Program teaches evidence-based medicine (EBM) and quality improvement concepts in an effort to promote scientific life-long learning for best practices to our trainees. We do this during 13 one hour “Journal Club” sessions during each year of training.
I. Philosophy & Overview:
Each resident will progress from learning how to identify pertinent medical literature and the fundamentals used to interpret medical literature during their internship to utilizing medical literature, analyzing guidelines to guide evidence based clinical decision-making, and developing a strategy for life-long learning during their second and third years of residency.
During residency the EBM Practice Curriculum will progress along 3 dimensions:
1. From a critical review of predetermined seminal articles emphasizing study design and biostatistics by all PL-1s to a group review of multiple recent articles on a given clinical topic by PL-3s.
2. From an attending-driven standardized educational experience emphasizing the tools needed to identify and analyze a given piece of medical literature to an attending-guided experience emphasizing the utilization of the learned tools to interpret a finite body of medical literature or guideline related to a specific timely clinical issue.
3. From a description of opportunities for grant funding, review of articles submitted for peer-review, review of grants submitted for funding, submission of articles and other scholarly activities during their PL-1 year to availability for participation in such activities with faculty supervision during their PL-2 or PL-3 years.
II. Conceptual Foundation:
A. Knowing Best Practice
EBM: We utilize the definition of evidence-based medicine from David Sackett, M.D. & colleagues, “Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”
B. Performing Best Practice
CQI: Continuous quality improvement is an approach maximizing quality that builds upon traditional quality assurance methods by emphasizing the organization and systems. For the purpose of this curriculum “organization” primarily includes one’s office, such as infrastructure within a medical home, and “systems” refers to resources within one’s community. CQI classically involves the use of the Plan, Do, Study, Act cycle, which makes up the EBM “Assess” component.
The UA program integrates these two concepts in a manner consistent with more current EBM practices by focusing both on the skills listed below and the continuous and circular relationship among these skills. This integrated approach builds upon concepts of Practice-based Learning & Improvement and involves System-based practice discussions regarding not only what best practice is, but also how to achieve it in one’s current environment. We believe this offers the best hope for overall improvements in pediatric care.
Ask: Construct an appropriate clinical question pertinent to your patient or practice
Acquire: Retrieve the current best evidence that answers the question
Appraise: Critically appraise the evidence for validity to your specific situation
Apply: Apply the evidence appropriately to your patient or practice
Assess: Evaluate your performance and, when appropriate, system adequacy
III. Elements of Program:
Interns and senior residents are separated into two groups for our Journal Club sessions in order to provide a unique focus to each group of trainees. While all of the 5 A’s are covered for each trainee group the focus for interns is on the how to accomplish each “A,” while the focus for senior residents is on applying these skills in practice. Nevertheless, the structure of all Journal Club sessions is similar, with more lecture time provided for interns and more discussion time provided for senior residents.
Journal Club Session structure:
Faculty presentation of core concept, e.g. power, bias, or effect size.
Resident presentation of clinical question and critical review of article using a structured format.
Group problem-solving exercise, e.g. calculation of NNT or RR.
Open discussion among entire group of article validity and its application into clinical practice. Barriers to clinical application are also considered.
Assessment of achievement of competence:
At each Journal Club session the supervising faculty member (one of 4 core faculty supervisors) will assess the competence of the presenting trainee on a set of predetermined criteria reflecting the 5 A’s listed above. This is recorded by the faculty supervisor in New Innovations and shared with the trainee along with a discussion of their strengths and opportunities for improvement.
Annually each trainee will complete an assessment survey (Fresno Test) designed to evaluate their competence in scientific clinical decision-making and practice. This allows us to assess the impact of this training program and identify areas of increased emphasis for the subsequent year.
Learning objectives:
Concepts are repeated throughout the 3 years of training at least twice and whenever pertinent to the review of a given article.
Actual learning objectives are listed by month (see monthly sessions following this section).
Below are a list of concepts covered for PL-1’s & Senior Residents
PL-1’s:
I. Types of data:
1. Categorical types
a. nominal
b. ordinal--scales
2. Continuous
3. Rates
II. Statistics: Are the groups probably different?
1. Inference
a. Power
b. P-Value
2. Types of analysis—univariate, bivariate, multivariate
a. Chi-square test—2 groups
b. Student t-test—2 groups
c. More than 2 groups: ANOVA
III. Statistics: How different are the groups?
1. Risk ratio
2. Odds ratio
3. Sensitivity & Specificity
4. Positive & negative predictive value
5. Likelihood ratio
6. Number to treat (or harm)
IV. How precise is your estimate?
1. 95% confidence intervals
2. Standard deviations
3. Standard error of mean
V. Study designs & their flaws
1. Levels of evidence & types of study designs
a. RCTs
b. Cohort Studies/Observational studies
c. Case-Control Studies
d. Cross-Sectional Surveys
e. Case Reports
f. Expert Opinion
2. Reviews, Meta-analyses & Guidelines
3. Common flaws
a. Observer bias
b. Selection bias
c. Response bias
d. Inter-observer variability
e. Confounding
f. Interactions
VI. EBM Resources & literature searching techniques
1. 5 A’s of EBM
2. PICO
3. Search tools
PL-2 & 3’s:
I. Types of data:
1. Likert scales
2. Other created scales, e.g. pain scales, behavior scales
3. Non-normally distributed continuous data
4. Incidence & prevalence rates
II. Statistics:
1. Correlation coefficients—Pearson, Spearman, & Kappa
2. Multiple variate analyses—logistic & linear regression
3. Subgroup analysis & Bonferroni
4. Genetics
III. Study designs & their flaws
1. The value in:
a. Cross-Sectional Surveys
b. Case Reports
c. Expert Opinion
2. Intention to treat studies
IV. EBM resources
1. How to keep up-to-date
2. Resources
a. Clinical
b. Research
c. Community