One thing that researchers sometimes do is to keep studies smaller in scope to minimize the scope and size. This is often a budgetary thing. So a study might be limited to first time parents with no complications. That’s normal and appropriate, but as a reader, we need to remember that those limited samples don’t …
Research Trouble Spot 8: Sample isn’t representative Read More »
Peer review is the process of having experts in the field carefully review studies before publication to ensure they meet a high standard of quality. While it’s not a perfect system, it is a key part of the process, and studies that do not go through the peer review process should be taken with a …
Blinding is when the researchers don’t tell which group people are in. It’s done to avoid any intentional or unconscious bias in how the study is carried out. There are two kinds of blinding: Participant Blinding where the people getting treatment in the study don’t know which treatment group they are in. Researcher Blinding where …
It’s pretty common for people just starting to read research to confuse correlation (when two things seem to change together) with causation (when one thing causes another). Correlation can be a clue towards finding causation, but it is not the same thing. There are lots of reasons why two data points might be correlated without …
Research Trouble Spot 5: Correlation – Causation Confusion Read More »
How big does a study need to be in order to be a valid study? This is a question that actually has an answer! There’s a way to calculate that from the outset of the study. It’s called a power analysis. it involves all kinds of factors like the number of variables being studied, the …
Experimental studies that show something is safe or improves outcomes should, by definition, have something to compare the new treatment against. If they don’t, it’s hard to know if the new treatment is a better option or not. Control groups can come in several varieties: No Treatment Group – Researchers would compare the outcomes with …
Sometimes the conflict is really easy to see. The stereotype of the used car salesman telling you that he knows the little old lady who used to own this car, and she only ever used it to drive to the grocery store and church, and followed the maintenance schedule religiously. The restaurants who all have …
The Trouble Spot here is not necessarily in the study itself. Plenty of excellent, well done studies have been promoted with sensational headlines. The trouble spot here is when you let yourself fall for the sensationalism. Remember that in this day and age, web sites and news outlets are competing like crazy for the attention …
Today I’m starting a new series on the top ten trouble spots to witch out for when evaluating research. Enjoy this sneak peek infographic and tune in every Tuesday morning for the next 10 weeks for details on each one!
It can be really annoying how the full text of so many studies is behind a paywall! But don’t despair, there’s still lots of options for getting the full text of any study you need. It might take some work, but it can be done. I generally keep a list of studies that grab my …