18.4 Standard errors
As we have seen, each sample is likely to be different, so any statistic estimated from the sample is likely to be different for each sample. That is, sample statistics have a distribution (a sampling distribution) and, under certain conditions, the sampling distribution is a normal distribution.
If the sampling distribution is a normal distribution, it is reasonable to ask what the value of the standard deviation of that normal distribution is.
Figs. 18.1 and and 18.2 show that the standard deviation appears to get smaller as the sample sizes get larger: the sample statistics show less variation for larger \(n\). This makes sense: larger samples generally produce more precise estimates. (After all, that’s the advantage of using larger samples: all else being equal, larger samples are preferred as they produce more precise estimates.)
In other words, the sample statistic varies less in larger samples: the value of the standard deviation of the sampling distribution is smaller for larger samples. The standard error is a measure of how precisely the sample statistic estimates the population parameter.
Example 18.1 (Standard errors) Suppose the sample proportion of odd-spins on the roulette wheel (Sect. 18.2) is estimated as \(\hat{p} = 0.51\). If the standard error was 0.01, this estimate is relatively precise: the standard error is very small, which means the value of \(\hat{p}\) is not likely to vary greatly from one sample to the next. Any single estimate of \(p\) is likely to be close to \(p\).
However, if the standard error was 0.2, the estimate of the population proportion is less precise: the standard error is larger, so the value of \(\hat{p}\) is likely to vary a lot from one sample to the next. Any single estimate* of \(p\) may not be close to \(p\).Definition 18.2 (Standard error) A standard error is the standard deviation of the sampling distribution of a statistic.
Any quantity estimated from a sample has a standard error.To expand: If every possible sample (found a certain way, and of a given size) was found, and the statistic computed from each sample, the standard deviation of these estimates is the standard error.
Recall from Sect. 18.1 that, for many sample statistics, the variation from sample to sample can be approximately described by a normal distribution (the sampling distribution) if certain conditions are met. Furthermore, the standard deviation of this normal distribution is the standard error.
Notice that the standard error is a special type of standard deviation; the variation in a sample estimate from sample to sample.