Research Trouble Spot 4: Small Sample Size

How big does a study need to be in order to be a valid study? This is a question that actually has an answer! There’s a way to calculate that from the outset of the study. It’s called a power analysis. it involves all kinds of factors like the number of variables being studied, the size of the effect, and sometimes data from a pilot study or other similar research.

Most researchers will recruit more than the minimum number the power analysis suggests, because study participants do tend to drop out and not complete the study. So they also try to guess how many people might drop out and recruit a higher number so they still have a big enough number.

So why would a study with a small sample size even get done, much less published? There are a variety of reasons: Sometimes it’s a way to share a new idea, sometimes to show that something tricky to study can be studied, and sometimes it’s to attract the kind of attention that can result in funding.

Be wary of acting on any experimental study that has a small sample size, even if the results report a significant effect, as they may or may not stand up to further research.