[…] The analysis of data inevitably involves some trafficking with the field of statistics, that grey area which is as surely not a branch of mathematics as it is neither a branch of science. In the following sections, you will repeatedly encounter the following paradigm:
· apply some formula to the data to compute “a statistic”
· compute where the value of that statistic falls in a probability distribution that is computed on the basis of some “null hypothesis”
· if it falls in a very unlikely spot, way out on a tail of the distribution, conclude that the null hypothesis is false for your data set
If a statistic falls in a reasonable part of the distribution, you must not make the mistake of concluding that the null hypothesis is “verified” or “proved”. That is the curse of statistics, that it can never prove things, only disprove them! At best, you can substantiate a hypothesis by ruling out, statistically, a whole long list of competing hypothesis, every one that has ever been proposed. After a while your adversaries and competitors will give up trying to think of alternative hypothesis, or else they will grow old and die, and then your hypothesis will become accepted. Sounds crazy, we know, but that’s how science works!
Numerical Recipes, §13.0 – Press et al.