Statistical Power and Sample Size

What power is. Statistical power is the probability that a study will detect an effect when there truly is an effect to detect: the probability of correctly rejecting the null hypothesis when the alternative is true. It is written as 1 minus beta, beta being the Type II error rate. A study with 80% power has an 80% chance of detecting a real effect and a 20% chance of missing it.

View larger

Reading a power level. At 50% a study is a coin flip, as likely to miss the effect as to find it. The conventional band is 80-90%; going higher buys a very high chance of detection at the price of a much larger sample.

Power and sample size. A larger sample increases power, but with diminishing returns. Power increases steeply at first and then plateaus, so doubling the sample does not double power.

What moves power. Power rises with the sample size, with the size of the effect being sought and with alpha, and falls as the variability of the outcome (its SD) rises.

The trade-offs. Every margin of safety is paid for in patients. Higher power requires a larger sample, with more cost and time. A smaller target effect (more clinically conservative) and a lower alpha (more statistically conservative) each require a larger sample too.

Every sample size calculation requires four inputs: alpha, power, the effect size and the variability of the outcome.

Alpha. The Type I error rate, the probability of falsely rejecting the null hypothesis: a false positive. The conventional choice is 0.05, accepting a 5% chance of finding a difference when none exists. A lower alpha such as 0.01 reduces false positives but requires a larger sample. When multiple outcomes are tested, the Bonferroni correction divides alpha by the number of tests to maintain the overall Type I error rate.

Power. Conventionally 0.80, an 80% chance of finding the effect if it exists and a 20% risk of missing it. A power of 0.90 is more conservative and needs a larger sample; it is used when the consequences of missing an effect are serious. Setting adequate power prevents underpowered studies that waste resources.

Effect size. The magnitude of difference the study sets out to detect. It should be clinically meaningful, not just statistically detectable, which is why it is set at the minimal clinically important difference (MCID): the smallest change that patients perceive as beneficial. Smaller effect sizes require much larger samples to detect.

Published MCIDs. Use validated values from the literature:

• WOMAC: approximately 10 points (100-point scale)
• VAS pain: approximately 15-20 mm (100-mm scale)
• SF-36 PCS: approximately 5 points
• Oxford Knee Score: 5 points
• Harris Hip Score: 10 points
• DASH: 10-15 points

Variability. The standard deviation, the spread of outcome values in the population, estimated from a pilot study, from published literature on the same outcome measure, or from prior studies in a similar population. Higher variability requires a larger sample to detect the same effect: if WOMAC scores vary widely (SD 20), more patients are needed than if they are consistent (SD 10).

The formula. For comparing means between two groups with a continuous outcome, the sample size per group is

n = 2 × (Zα + Zβ)² × SD² / MCID²

where Zα and Zβ are the Z-scores for alpha and beta, SD is the standard deviation and the MCID is the effect size. The Z-scores in common use:

Worked example. How many patients are needed per group to detect a 10-point improvement in WOMAC score? Take the MCID as 10 points, the SD as 20 points from the literature, alpha 0.05 (Zα 1.96) and power 0.80 (Zβ 0.84):

• n = 2 × (1.96 + 0.84)² × 20² / 10²
• n = 2 × 7.84 × 400 / 100
• n = 2 × 31.36
• n = 63 patients per group

Allowing for dropout. The sample must be inflated for anticipated loss to follow-up. Expecting 15% dropout, n = 63 / 0.85 = 74 patients per group, a total enrolment of 148.

Absolute and standardised. An effect size expressed in the units of the outcome, such as a 10-point WOMAC difference, is an absolute effect size. Dividing it by the standard deviation gives the standardised effect size, or Cohen's d, which is dimensionless and lets effects be compared across different scales.

Cohen's d = (mean difference) divided by the (pooled standard deviation)

Why small effects are expensive. The required sample size is roughly proportional to 1 divided by the square of the standardised effect size. Halving the effect you wish to detect roughly quadruples the patients needed, and detecting a small standardised effect needs roughly an order of magnitude more patients than a large one, which is why a clinically tiny difference demands an enormous trial.

When there is no MCID. A distribution-based estimate of 0.5 standard deviation (a medium effect) is sometimes used as a transparent fallback. An anchor-based MCID is preferred, and where none exists an anchor-based MCID study is the alternative.

A priori analysis. The sample size is calculated before any patient is enrolled, using an effect size and SD estimated from the literature or a pilot. This is the recommended approach, and it is what ensures a study is designed with adequate power.

Post hoc analysis. Power calculated after the study is complete, from the observed data, is often done to explain a non-significant result. It is mathematically redundant: post hoc power is derived directly from the p-value, so if p is not significant, post hoc power will be low by definition.

By study design. The method of calculation changes with the design.

Underpowered orthopaedic trials. Across 117 randomised fracture-care trials including 19,942 patients, mean study power for the primary outcome was 24.65% (range 2-99%) and the Type II error rate was 90.52%. A field-wide mean power near a quarter means the typical trial had roughly a one-in-four chance of finding a difference that was really there. Small samples fail to detect clinically meaningful differences, so their results are inconclusive, not negative, and this is the most quotable statistic in orthopaedic evidence appraisal.

Statistical versus clinical significance. Statistical significance is p less than 0.05; clinical significance is a difference that exceeds the MCID. A statistically significant finding may not be clinically important, so always check whether the difference exceeds the MCID. Large studies detect trivial differences, and small studies miss important ones.

Pilot studies. A pilot estimates variability (the SD) and feasibility before the full trial and helps refine the sample size calculation. Do not use it for hypothesis testing.

How big the pilot should be. Not the 20 or 30 patients usually assumed, and the answer runs opposite to intuition: the smaller the effect you expect, the larger the pilot must be. For a main trial powered at 90% with two-sided alpha 0.05, the recommended pilot sizes per arm are 75, 25, 15 and 10 for extra-small (0.1), small (0.2), medium (0.5) and large (0.8) standardised effect sizes. A pilot exists to pin down the SD, and a small target effect leaves no room for an imprecise SD estimate; a pilot too small to fix the SD simply passes its error into the definitive trial.

The hypothesis a trial is designed to test determines its sample size and how a "negative" result may be read.

View larger

The non-inferiority bargain. A non-inferiority trial accepts that the new treatment may be marginally worse, in exchange for advantages such as lower cost, toxicity or convenience, provided it stays within the pre-specified margin (delta). The margin must be clinically justified: too wide allows a genuinely inferior treatment to pass, too narrow demands an impractically large sample.

Assay sensitivity and the analysis set. Such trials require assay sensitivity, meaning the trial could have detected a true difference if one existed. Unlike superiority trials, the per-protocol analysis is at least as important as intention-to-treat, because dropouts and non-compliance bias toward apparent non-inferiority. Yet the per-protocol analysis may itself bias toward non-inferiority.

Online calculators. The ClinCalc sample size calculator is free online, OpenEpi covers power calculation for epidemiological studies, and Sealed Envelope offers clinical trial tools.

Simulation-based power. Simulation is used for complex designs (adaptive, cluster), non-standard distributions and multiple endpoints with correlations. Thousands of hypothetical datasets are simulated and each is analysed with the planned method; the proportion achieving significance is the estimated power.

Multicentre trials and registries. When a single centre cannot recruit an adequate sample, multicentre collaboration achieves power. The AOANJRR and international registries provide large samples.

When power cannot be achieved. Three situations can put adequate power out of reach:

• Rare conditions may need registry-based or multinational studies
• Ethical constraints: more patients cannot be enrolled, for safety reasons
• Resource limitations: lower power is accepted, with pre-specified disclosure

Other approaches. Bayesian analysis can provide evidence even with small samples, and interpreting the confidence interval focuses attention on precision. Meta-analysis combines the effect estimates of multiple underpowered studies, and the pooled analysis has greater power than any individual study; it requires a systematic search and quality assessment, and heterogeneity (I²) must be evaluated.

Adaptive designs. Sample size re-estimation uses a pre-planned interim analysis to adjust the sample size to the observed variability, and maintains study validity if done properly. Group sequential designs run multiple pre-planned analyses during the trial, can stop early for efficacy or futility, and adjust alpha spending to maintain the overall Type I error.

Cluster trials. Ignoring the intraclass correlation (ICC) severely underestimates the sample, and an ICC of 0.05 can double or triple the required sample. The design needs many clusters, not just many individuals per cluster.

Reporting the calculation. A sample size statement should include:

• The alpha level
• The power level (usually 80%)
• The effect size, justified (MCID with reference)
• The source of the SD (literature or pilot)
• The statistical test
• The dropout or attrition adjustment
• The software or formula used

Why Power and Sample Size Are a Global Concern

Underpowering is not confined to any one country. The landmark survey of orthopaedic trauma RCTs found a mean primary-outcome power of roughly 25 percent and a Type-II error rate over 90 percent, and fragility analyses of sports-surgery trials worldwide report a median Fragility Index of only two patients. Adequately powered, registry-linked and multinational collaborations are the international response.

Registries as a Power Solution

Large national arthroplasty and trauma registries supply the sample sizes single centres cannot. The AOANJRR (Australia), NJR (England, Wales, Northern Ireland and Isle of Man), AJRR (USA), SHAR (Sweden), the Norwegian Arthroplasty Register and the NZJR pool hundreds of thousands of procedures, giving the statistical power to detect small but clinically important differences in implant survival and revision rates that no individual RCT could achieve.

High- versus Limited-Resource Practice Variation

Where Experts Still Disagree