How To Calculate Effect Size

Calculate Cohen’s d, Hedges’ g, or Glass’s Δ for your statistical analysis

Comprehensive Guide: How to Calculate Effect Size in Statistical Analysis

Effect size is a quantitative measure of the magnitude of an experimental effect, representing the standardized difference between two means. Unlike statistical significance (p-values), effect size provides information about the practical importance of research findings. This guide explains how to calculate and interpret effect sizes, with a focus on Cohen’s d, Hedges’ g, and Glass’s Δ.

Why Effect Size Matters

Effect size addresses critical questions that p-values cannot:

• Practical significance: Is the effect large enough to be meaningful in real-world applications?
• Study comparison: Can results be compared across studies with different sample sizes?
• Meta-analysis: Provides a common metric for combining results from multiple studies
• Power analysis: Essential for determining appropriate sample sizes for future studies

Types of Effect Size Measures

1. Cohen’s d

The most common effect size measure for comparing two means. Calculated as:

d = (M₁ – M₂) / SD_pooled

Where SD_pooled is the pooled standard deviation of both groups.

2. Hedges’ g

A corrected version of Cohen’s d that accounts for small sample bias:

g = (M₁ – M₂) / SD_pooled × (1 – 3/(4df – 1))

Where df = n₁ + n₂ – 2 (degrees of freedom).

3. Glass’s Δ

Uses only the standard deviation of the control group:

Δ = (M₁ – M₂) / SD_control

Useful when treatment groups have different variances or when comparing to normative data.

Interpreting Effect Sizes

Cohen (1988) provided general guidelines for interpreting effect sizes:

Effect Size	Small	Medium	Large
Cohen’s d	0.2	0.5	0.8
Hedges’ g	0.2	0.5	0.8
Glass’s Δ	0.2	0.5	0.8

Important note: These are general guidelines. Interpretation should always consider:

• The specific field of study (effect sizes vary by discipline)
• The context of the research question
• Previous research in the area
• The cost/benefit ratio of the intervention

Step-by-Step Calculation Process

1. Collect your data:
   • Group 1 mean (M₁) and standard deviation (SD₁)
   • Group 2 mean (M₂) and standard deviation (SD₂)
   • Sample sizes for both groups (n₁ and n₂)
2. Calculate the pooled standard deviation (for Cohen’s d and Hedges’ g):

   SD_pooled = √[( (n₁-1)×SD₁² + (n₂-1)×SD₂² ) / (n₁ + n₂ – 2)]

3. Compute the basic effect size:

   For Cohen’s d: d = (M₁ – M₂) / SD_pooled

   For Glass’s Δ: Δ = (M₁ – M₂) / SD_control

4. Apply small sample correction (for Hedges’ g):

   g = d × (1 – 3/(4df – 1)) where df = n₁ + n₂ – 2

5. Calculate confidence intervals:

   Provides a range of values within which the true effect size likely falls.

6. Interpret the results:

   Compare to field-specific benchmarks and previous research.

Common Mistakes to Avoid

• Ignoring directionality: Effect size can be positive or negative, indicating the direction of the effect
• Mixing up groups: Always clearly define which group is Group 1 vs. Group 2
• Assuming equal variances: When variances differ significantly, Glass’s Δ may be more appropriate
• Overinterpreting small effects: Statistically significant ≠ practically meaningful
• Neglecting confidence intervals: Always report these to show precision of estimates

Effect Size in Different Research Designs

Research Design	Appropriate Effect Size	Key Considerations
Two independent groups	Cohen’s d, Hedges’ g, Glass’s Δ	Most common scenario for these measures
Paired samples (pre-post)	Cohen’s dz (standardized mean difference)	Uses standard deviation of the differences
One-way ANOVA	Eta-squared (η²), Omega-squared (ω²)	Measures proportion of variance explained
Correlational studies	Pearson’s r, Cohen’s f²	r of 0.1 = small, 0.3 = medium, 0.5 = large
Binary outcomes	Odds ratio, Risk ratio, Cohen’s h	Hinge’s g can be adapted for proportions

Advanced Considerations

1. Confidence Intervals for Effect Sizes

Calculating confidence intervals provides crucial information about the precision of your effect size estimate. The formula for the standard error of Cohen’s d is:

SE_d = √[(n₁ + n₂)/(n₁n₂) + d²/(2(n₁ + n₂))]

The 95% confidence interval is then:

CI = d ± 1.96 × SE_d

2. Effect Size in Meta-Analysis

Effect sizes are particularly valuable in meta-analysis because:

• They standardize results across studies with different measures
• Allow for meaningful comparisons between studies
• Can be weighted by sample size for more accurate pooled estimates
• Help identify potential moderator variables

3. Software Implementation

Most statistical software can calculate effect sizes:

• R: Use the compute.es package or effsize package
• Python: pingouin or scipy.stats libraries
• SPSS: Requires manual calculation or custom syntax
• Excel: Can be calculated with basic formulas
• Jamovi: Built-in effect size calculations in many analyses

Real-World Examples

Example 1: Education Intervention

A study compares a new teaching method (n=40, M=85, SD=12) to traditional instruction (n=42, M=78, SD=10).

Calculation:

SD_pooled = √[(39×12² + 41×10²)/(40+42-2)] = 10.95

Cohen’s d = (85-78)/10.95 = 0.64 (medium effect)

Example 2: Medical Treatment

A drug trial compares treatment (n=30, M=120, SD=15) to placebo (n=30, M=100, SD=20).

Calculation:

SD_pooled = √[(29×15² + 29×20²)/(30+30-2)] = 17.68

Glass’s Δ = (120-100)/20 = 1.00 (large effect)

Example 3: Psychology Experiment

A memory study with small samples (n₁=15, M₁=7.2, SD₁=1.5; n₂=15, M₂=5.8, SD₂=1.2).

Calculation:

Cohen’s d = (7.2-5.8)/1.36 = 1.03

Hedges’ g = 1.03 × (1 – 3/(4×28 – 1)) = 0.99 (large effect)

Authoritative Resources:

For more detailed information on effect size calculation and interpretation:

• American Psychological Association: Understanding Effect Sizes
• UCLA Statistical Consulting: What is Effect Size?
• NIH Guide to Statistical Reporting (includes effect size)

Frequently Asked Questions

Q: When should I use Hedges’ g instead of Cohen’s d?

A: Hedges’ g is preferred when working with small sample sizes (typically n < 20 per group) because it corrects for the positive bias in Cohen's d that occurs with small samples. For large samples, Cohen's d and Hedges' g yield very similar results.

Q: How do I calculate effect size for more than two groups?

A: For designs with three or more groups (like one-way ANOVA), you would typically use eta-squared (η²) or omega-squared (ω²) which represent the proportion of variance in the dependent variable that’s explained by the independent variable. These range from 0 to 1, with values of 0.01, 0.06, and 0.14 representing small, medium, and large effects respectively.

Q: Can effect size be negative?

A: Yes, effect size can be negative, which simply indicates the direction of the effect. A negative effect size means the second group’s mean is higher than the first group’s mean. The absolute value indicates the magnitude regardless of direction.

Q: How does effect size relate to statistical power?

A: Effect size is one of the four main components of statistical power (along with sample size, significance level, and power). Larger effect sizes require smaller sample sizes to detect significant differences, all else being equal. Power analysis often uses expected effect sizes to determine necessary sample sizes.

Q: Should I always report effect sizes?

A: Yes, the American Psychological Association and many other scientific organizations recommend always reporting effect sizes alongside statistical significance tests. Effect sizes provide information about the practical significance of findings that p-values cannot.

Conclusion

Understanding and properly calculating effect sizes is essential for modern statistical reporting. While p-values tell us whether an effect exists, effect sizes tell us how large that effect is – information that’s crucial for both scientific progress and practical application of research findings.

Remember these key points:

• Effect size measures the strength of a phenomenon, independent of sample size
• Cohen’s d, Hedges’ g, and Glass’s Δ are the most common measures for mean differences
• Always interpret effect sizes in the context of your specific field
• Report confidence intervals to show the precision of your estimates
• Effect sizes are more important than p-values for understanding practical significance

By mastering effect size calculation and interpretation, you’ll significantly improve the quality and impact of your research reporting, making your findings more useful to both the scientific community and practical applications.