Reading numbers without a statistics degree.

You do not need to be able to run the analysis to read it. You need to know which numbers constrain the claim being made. Four ideas cover most of what you will meet in a results table.

The p-value answers a narrower question than people think

A p-value is the probability of seeing a difference at least this large if there were genuinely no difference in the population. That is all it is.

It is not the probability that the finding is true. It is not the size of the effect. It is not the probability that the result will replicate. And .05 is a convention, not a discovery / p = .049 and p = .051 are the same evidence, and treating one as a finding and the other as nothing is the single most common error in undergraduate write-ups.

Practically: a low p-value tells you the difference is probably not pure noise. It says nothing about whether it matters.

Effect size is the number that matters

Effect size is how big the difference is. Ask for it in the units of the thing itself before you ask for the standardized version: seven percentage points, four marks out of a hundred, eleven minutes.

Then ask the only real question: would this change anything? A four-mark difference in coursework can move a grade boundary. A 0.3-point difference on a seven-point satisfaction scale, statistically significant across 4,000 respondents, probably changes nothing anyone would notice.

When you see standardized measures / Cohen's d, correlation r / the rough conventions are around 0.2 small, 0.5 medium, 0.8 large for d. Treat them as loose landmarks, not thresholds; what counts as large depends entirely on the field.

Sample size cuts both ways

Small samples are noisy. A study of 20 people can show a large difference that is mostly chance, and it can also miss a real difference entirely because it lacked the power to detect it. "No significant difference" in a small study means "we could not tell", not "there is nothing there".

Very large samples have the opposite problem. With 50,000 records, almost everything is statistically significant, including differences too small to care about. Whenever a paper reports significance on a large dataset, go straight to the effect size.

Confidence intervals tell you more than a single number

A 95% confidence interval gives the range of values compatible with the data. Two studies can both report an increase of 5 points: one with an interval of 3 to 7, one with an interval of -2 to 12. The first is a finding. The second is barely distinguishable from no effect and should be read as such.

If a paper reports intervals, read them and largely ignore the stars next to the coefficients. If it does not report them, that is worth noting.

The four sentences

Fill these in for any results table and you will understand the paper's actual evidential position:

1. The comparison was between ____ and ____, with n = ____ in each.
2. The difference was ____, in the original units.
3. The uncertainty around it was ____ (interval, or standard error).
4. Which means, in practical terms, that ____.

If you cannot fill in sentence four, either the effect is too small to interpret or the paper has not told you enough. Both are findings about the paper.

Three things that should make you slow down

• Many outcomes tested, one reported. Test twenty things at p < .05 and one will look significant by chance alone. If the method mentions measures that never reappear in the results, ask why.
• Subgroups appearing only in the discussion. "The effect was strongest among first-generation students" is a hypothesis if it was not planned in advance.
• Percentages without denominators. "40% improved" from a base of ten people is four people.

---

When you genuinely cannot judge an analysis, say so in your notes rather than nodding along. "I cannot evaluate the mixed model; the descriptive difference is 6 marks" is a professional position and an honest one. Nobody reads every method confidently, including the people who wrote them.