Statistical Power and Sample Size

Before you start

You are comfortable with p-values and basic distributions. New to those? Start with Stats you cannot avoid.
Any term new? The Glossary has it.

Learning objectives

By the end of this lesson you will be able to: define statistical power, name the four levers that raise it, explain why underpowered studies both miss and exaggerate effects, and run a power analysis in R to choose a sample size before the experiment.

What power actually is

Statistical power is the probability that your study detects a real effect when one truly exists. Put another way, it is one minus the false-negative rate: if an effect is real and your power is 0.8, you have an 80 percent chance of finding it and a 20 percent chance of missing it (a false negative, or "Type II error").

Most fields aim for power of at least 0.8. A study with power well below that is "underpowered": even when the biology is real, the experiment is more likely than not to come up empty.

1The four levers that raise power

Power goes up when any of these improve:

Lever	Power rises when...	Why
Effect size	the true effect is larger	a bigger difference is easier to see above the noise
Sample size (n)	you collect more samples	more data sharpens the estimate and shrinks chance variation
Variability	the measurements are less noisy	tighter data makes a real difference stand out
Alpha	you allow a less strict alpha	a looser significance threshold is easier to cross (but raises false positives)

You usually cannot change the true effect size or the biology's variability much, and loosening alpha trades away false-positive control. That leaves sample size as the main lever you actually control. In sequencing experiments, that means adding biological replicates.

2More samples push the curve past 0.8

Power climbs with sample size and levels off near 1. The dashed green line is the usual 0.8 target; the smallest n that crosses it is the sample size you want.

3Why underpowered studies are doubly dangerous

An underpowered study does two bad things, not one.

First, it misses real effects. That is the obvious cost: low power means most true findings slip through undetected, wasting the experiment.

Second, the "significant" hits it does report tend to be wrong. When power is low, only an unusually large random bump clears the significance threshold, so the effects that survive are exaggerated in size, and sometimes even wrong in their direction (sign). This is the winner's curse: the published effect from a small study is often bigger than the truth, and fails to replicate. Low power therefore hurts both sensitivity and the trustworthiness of whatever does turn up.

In practice: RNA-seq replicates

Adding biological replicates is the main lever for power. For RNA-seq, 3 replicates per group is a common practical minimum, and more is better, especially for detecting smaller fold changes or working with noisy tissue. Technical replicates do not substitute for biological ones.

4Do the power analysis before the study

Power, effect size, sample size, and alpha are linked: fix any three and the fourth is determined. A power analysis uses that link, before you collect data, to solve for the sample size that will give you adequate power for the effect you care about. You do this with the R pwr package, or by simulation for more complex designs.

Never report post-hoc power

"Post-hoc power" computed from your own observed p-value tells you nothing new: it is just the p-value re-expressed, and a non-significant result will always look underpowered. Plan power before the study using an effect size you decide in advance, not after the fact from your own result.

5Worked example: solving for n in R

Suppose you want power of 0.8 at the usual alpha of 0.05, and you care about a moderately large effect (Cohen's d of 0.8). Ask the pwr package how many samples per group that needs:

# install.packages("pwr"); library(pwr)
pwr::pwr.t.test(d = 0.8, power = 0.8, sig.level = 0.05)

#      Two-sample t test power calculation
#               n = 25.5      <- per group, round UP to 26
#               d = 0.8
#       sig.level = 0.05
#           power = 0.8

How to read it:

You leave out n and the function solves for it. Here it returns about 25.5 per group, which you always round up to 26 so you do not fall short of the target.
Want to detect a smaller effect? Lower d and the required n rises sharply, which is exactly why subtle effects need big studies.
The same function works in reverse: give it n and it returns the power you would have, useful for checking a planned design.

Check your understanding

What does statistical power measure?

Right. Power is the chance of finding a real effect, equal to one minus the false-negative (Type II error) rate.

Which change will increase the power of a study?

Correct. Power rises with larger effect size, larger sample size, lower variability, and a less strict alpha. Sample size is usually the lever you control.

Besides missing real effects, what else does an underpowered study tend to do?

Exactly. When power is low, only unusually large random bumps reach significance, so reported effects are inflated and can even point the wrong way.

For an RNA-seq experiment, what is a commonly cited practical minimum number of biological replicates per group?

Right. Three biological replicates per group is a common minimum for RNA-seq, and adding more improves power, especially for smaller effects.

When should you do a power analysis, and which value should you not report?

Correct. Plan power before collecting data using a pre-specified effect size. Post-hoc power from your own result just restates the p-value and is uninformative.

Next in Track 5