10.1 Two Population Means with Unknown Standard Deviations

Chapter 9 tested a parameter against a fixed number; this chapter tests two populations against each other, and the six steps of a hypothesis test carry over unchanged. What changes is the parameter, which is now the difference of two population means, its point estimate, which is the difference of the two sample means, and the standard error, which combines the variability of both samples by adding their variances. Because the population standard deviations are unknown and estimated by the two sample standard deviations, the test statistic follows a Student t distribution, and its degrees of freedom come from the Aspin-Welch formula — a calculation the book calls complicated, which is rarely a whole number and is done by machine. The book is emphatic that the sample variances are not pooled, since pooling would assume the two populations share a standard deviation that this test deliberately does not assume.

Subject: Statistics · 67 slides · symbolic lesson

Open the interactive version of this deck

What this lesson covers

The lesson, slide by slide

1. Two Population Means with Unknown Standard Deviations

Title

Statistics · Chapter 10 — Hypothesis Testing with Two Samples

Is the gap between two sample means bigger than chance alone would make it?

2. What this lesson gives you

Objectives

Chapter 9 tested one population against a fixed number. Most real questions compare two groups instead: an online class against a face-to-face class, girls against boys, one college against another. This section adapts the same hypothesis test to that comparison, when neither population's standard deviation is known.

The method is mostly Chapter 9's. What is new is one formula for the standard error, one for the degrees of freedom, and a set of decisions about which group is which. Most wrong answers in this section come from those decisions, not from the arithmetic, so each one gets its own slide.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 511-520 — the chapter introduction and the section this deck covers

3. Before anything new: a one-sample t statistic

Warm-up

Section 9.3 built every test statistic the same way: the estimate, minus the value the null hypothesis claims, divided by the estimate's standard error. Recall it on a small case before you need it for two samples.

Discussion prompt

Sixteen boys aged seven to eleven play sport for a mean of 3.2 hours a day, with sample standard deviation 1 hour. Find the t statistic for testing whether the population mean is 2.7 hours, and its degrees of freedom.

Every piece there has a two-sample counterpart. The sample mean becomes a difference of two sample means, the claimed value becomes a claimed difference, and s over the square root of n becomes a combination of two such terms.

Hold on to the shape of that calculation: a gap, measured in standard errors. The two-sample statistic has exactly the same shape. What changes is the gap, which becomes the difference between two sample means, and the standard error, which has to account for two samples instead of one.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §9.3 — the one-sample t statistic; Table 10.1's boys are the sample used here

4. Two groups, one parameter

Section

Part 1

5. Two groups to compare: an online and a face-to-face class

Concept

A professor gave the same statistics final to an online class and a face-to-face class, and drew 30 scores at random from each. Here are all sixty, from the book's Example 10.3, drawn on one scale.

Figure (svg): Two stacked dot plots on one scale from 30 to 100 points, in bins five points wide. The online class's 30 scores spread from 35.5 to 97.1, with a dashed line at its mean of 72.85. The face-to-face class's 30 scores bunch between 57.6 and 100, with its mean at 84.98.

The two clouds overlap a great deal, yet their centres sit about twelve points apart. Whether that gap is real is exactly the question this section answers.

The face-to-face scores average 84.98 points and the online scores 72.85, a gap of about twelve points. But look at the overlap: several online students beat most of the face-to-face class. The question is whether the gap reflects the two populations, or just which 30 students happened to be drawn.

The two samples here are independent: knowing one online student's score tells you nothing about any face-to-face student's score, because different people are in each group. That is one of the two designs the chapter separates.

independent groups — Two samples in which values selected from one population are not related in any way to values selected from the other. Their parameters can be means or proportions.

matched pairs — Two samples that are dependent, such as the same people measured twice. The parameter is the mean of the differences, and Section 10.4 handles it.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10 introduction, p. 511; §10.1, p. 516 — independent groups, matched pairs, and Table 10.3-10.4

6. The parameter is a difference of two means

Concept

In Chapter 9 the parameter was one population mean. Here there are two, μ1 and μ2, and the question is about how they compare. The single parameter that captures that comparison is their difference.

\[ \text{parameter: } \mu_1 - \mu_2 \qquad \text{estimate: } \bar{x}_1 - \bar{x}_2 \]

The random variable is the difference of the sample means. Different samples give different differences, and that variation is what the test has to measure. The null hypothesis says there is no difference, and the book writes it two equivalent ways.

\[ H_0: \mu_1 = \mu_2 \quad \Longleftrightarrow \quad H_0: \mu_1 - \mu_2 = 0 \]

The second form is the useful one. It shows the null still names a single number, zero, just as every null in Chapter 9 did, so there is something to centre a distribution on. The alternative is one of three: the difference is not zero, is greater than zero, or is less than zero.

\[ H_a: \mu_1 - \mu_2 \ne 0 \quad \text{or} \quad > 0 \quad \text{or} \quad < 0 \]

Example 10.1 gives the wording rule. The words the same tell you the null has an equals sign; with no other words to indicate a direction, assume the alternative says is different, and the test is two-tailed.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 512-513 — the random variable and Example 10.1's hypotheses

7. Independent groups or matched pairs?

Sorting

Before choosing any formula, decide which design you have. The test in this section is for independent groups only. Ask one question of each study: is each value in one sample tied to a particular value in the other?

Sort into buckets

Sort each study by its design.

independent groups
heart attack rates in an aspirin group and a separate placebo group; hours of sport for a sample of girls and a separate sample of boys; final exam scores of an online class and a face-to-face class; years with the company for workers sampled at two different firms
matched pairs
the same students' SAT scores before and after a preparation course; reaction times of each driver with and without a phone; blood pressure of twins, one twin on each diet
i
Different people are in each sample, and nothing links a value in one to a particular value in the other. These are the designs this section tests.
m
Each value has a partner: the same student twice, the same driver twice, or two twins. The natural analysis takes each pair's difference, which is Section 10.4.

The twins are the subtle case. They are two different people, so the samples can look independent, but each twin was chosen because of the other. Their values are related, and treating them as independent throws away the pairing that makes the comparison sharp.

The aspirin study is the chapter's own opening example: one group takes aspirin, a separate group takes a placebo, and heart attack rates are compared over several years. Two separate groups of people make it independent.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10 introduction, p. 511 — the aspirin studies and the two designs

8. Which tail does the claim point to?

Discrimination

Once the groups are named, the alternative hypothesis comes from the claim's wording. Take group 1 as the group named first in each claim, and decide whether the claim says the difference is positive, negative, or just not zero.

Sort into buckets

With group 1 as the group named first, sort each claim by the alternative it gives.

Ha: μ1 − μ2 > 0 (right)
graduates of college A have taken more math classes than those of college B; Company A retains its workers longer than Company B
Ha: μ1 − μ2 < 0 (left)
the online class's mean exam score is lower than the face-to-face class's; brand 1's sodium content is lower than brand 2's
Ha: μ1 − μ2 ≠ 0 (two)
girls and boys are believed to play sport the same amount; is there a difference?; is there a difference in weighted alpha between northeast and western banks?
r
More classes and longer retention both say group 1's mean is bigger, so the difference is positive and the test is right-tailed.
l
Lower scores and lower sodium both say group 1's mean is smaller, so the difference is negative and the test is left-tailed.
t
A question about whether there is a difference, with no direction stated, gives the not-equal alternative and a two-tailed test.

Notice that the tail depends on two choices, not one: the direction in the claim and which group you call group 1. Swap the groups in the college claim and the same claim becomes a left-tailed test of college B minus college A. Neither labelling is wrong; mixing them is.

Words to listen for: more, longer, greater and higher point one way; less, lower, fewer and shorter point the other. Same, equal and is there a difference point nowhere, and the null keeps the equals sign. The claim being tested always goes in the alternative, never in the null.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 513-518 — Examples 10.1-10.3 and Try Its 10.2, 10.3, 10.5

9. A gap of 1.2 hours: is that proof?

Prediction

In Example 10.1, nine girls average 2 hours of sport a day and sixteen boys average 3.2 hours. The sample means differ by 1.2 hours. Commit to an answer before you read on.

Predict first

Does a gap of 1.2 hours between the sample means show that the population means differ?

  • It depends on how much the hours vary within each group
  • Yes, more than an hour a day is clearly a real difference
  • No, two samples this small can never show a difference
  • Yes, because the boys' sample is larger than the girls'

Correct: It depends on how much the hours vary within each group

Why: The book puts it directly: very different means can occur by chance if there is great variation among the individual samples. A gap is judged against the spread, so the same 1.2 hours can be strong evidence or none at all.

The instinct that a big-looking gap must be real is the one this section replaces. A gap has no meaning on its own; it has to be compared with how much a gap would wobble from sample to sample if the populations were identical.

That wobble depends on two things for each group: how spread out the individual values are, and how many values were sampled. Wide spreads and small samples make big gaps easy to get by luck. The next slide shows the same gap twice, once with each.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — very different means can occur by chance

10. Same gap, different spread

Picture it

Here are two made-up pairs of samples. In both, the girls' mean is exactly 2 hours and the boys' mean exactly 3.2 hours, so the gap is the same 1.2 hours. Only the spread within each group differs.

Figure (svg): Two panels, each with a row of twelve girls' values and a row of twelve boys' values, hours per day. In both panels the girls' mean is exactly 2 and the boys' exactly 3.2, marked by dashed lines. On the left the values within each group sit within about half an hour of their mean, so the rows do not overlap. On the right the values spread over several hours and the rows overlap almost completely.

Both panels show a gap of 1.2 hours between the means. On the left that gap is unmistakable; on the right the same gap could easily be luck. The gap alone cannot tell you which.

On the left every girl plays between about one and a half and two and a half hours, and every boy between about three and three and a half. No girl plays as much as any boy, and nobody would believe that pattern came from identical populations.

On the right the individual values spread over several hours. The two rows overlap so much that if you shuffled the labels, you could easily get a gap of 1.2 hours between two random halves. The same gap is weak evidence there.

That is why the test divides the gap by a standard error. Dividing turns a gap measured in hours into a gap measured in units of sampling wobble, and only the second number says whether the gap is surprising. The next part builds that standard error.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the difference is standardised by dividing by the standard error

11. Trap: letting the order of subtraction drift

Trap

The trap

Example 10.3 sets group 1 as online and claims the online mean is lower. A first attempt writes the alternative that way, then subtracts the bigger mean first:

\[ H_a: \mu_1 - \mu_2 < 0 \]

\[ t = \frac{84.98 - 72.85}{3.757} = +3.23 \]

\[ p = P(T < 3.23) = 0.9989 \]

Wrong. The statistic was built for a different subtraction.

The fix

The hypotheses fix group 1 as online, so the estimate must be online minus face-to-face. Keep the order from start to finish:

\[ t = \frac{72.85 - 84.98}{3.757} = -3.23 \]

\[ p = P(T < -3.23) = 0.0011 \]

The same data gave a p-value near one and a p-value near zero. Write group 1 and group 2 at the top of every problem, and check that the sign of t agrees with the direction of the alternative when the evidence supports it.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 517 — be careful not to mix up the information for Group 1 and Group 2

12. The standard error of a difference

Section

Part 2

13. Why the two variances add

Concept

Each sample mean has its own sampling variability, which Chapter 7 measured by its standard error, σ over the square root of n. Squaring gives each mean's variance. The question is what happens to variability when you subtract one independent quantity from another.

\[ \text{Var}(\bar{X}_1) = \frac{\sigma_1^2}{n_1}, \qquad \text{Var}(\bar{X}_2) = \frac{\sigma_2^2}{n_2} \]

For independent random quantities, variances add, whether you add the quantities or subtract them. Subtracting does not cancel any wobble: a high first mean and a low second mean make the difference extra large, and the reverse makes it extra small.

\[ \text{Var}(\bar{X}_1 - \bar{X}_2) = \frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2} \]

Here is that rule on Example 10.1's numbers, simulated. Three thousand times, a girls' mean was drawn with standard error 0.289 and a boys' mean with standard error 0.250, and their difference recorded.

Figure (svg): On one axis from minus 3 to 4.5 hours: a narrow orange curve centred at 2 for the girls' sample mean, a narrow blue curve centred at 3.2 for the boys', and a histogram of 3000 simulated differences, girls' mean minus boys' mean, centred at minus 1.2. The histogram is wider and lower than either curve; its standard deviation is 0.380.

Each sample mean wobbles a little. Their difference wobbles more than either, because it inherits the wobble of both.

The simulated differences have standard deviation 0.380, close to the 0.382 the rule predicts and larger than either 0.289 or 0.250. The population sigmas are unknown, so the formula you will use replaces each σ by its sample s. The result is called the estimated standard error.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the standard error of the difference in sample means

14. Reading the standard error piece by piece

Notation

This formula does most of the new work in the section. Read it as two Chapter 8 standard errors, each squared, added, and then square-rooted back to the original units.

Annotate

On: \( \text{SE} = \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}} \)

  • Group 1's sample variance divided by its own sample size: the square of group 1's standard error, s1/√n1.
  • The same for group 2. Each group is divided by its own n; the sizes are never combined.
  • Always a plus, even though the estimate is a difference. Independent variability adds.
  • The square root returns to the units of the data: hours, points, years. A standard error has the same units as the means.

Two quick consequences are worth knowing by sight. The standard error is always larger than either group's own standard error, since you add something positive under the root. And it is always smaller than the two standard errors added end to end, because square roots of sums are smaller than sums of square roots.

Those two bounds give you a free check on every standard error you compute in this deck. If your answer falls outside them, a minus sign or a missing square has crept in.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the standard error formula, with s1 and s2 estimating σ1 and σ2

15. Example 10.1: the standard error of the gap

Worked example

Table 10.1: nine girls, sample standard deviation 0.866 hours; sixteen boys, sample standard deviation 1.00 hour. Find the estimated standard error of the difference in sample means.

The plan is to compute each group's variance term separately, add them, and only then take the root. Keeping the two terms apart lets you see which group contributes more uncertainty, which matters for the design questions later.

\[ s_g = 0.866, \; n_g = 9 \qquad s_b = 1.00, \; n_b = 16 \]

Square the girls' standard deviation

Why: the formula uses variances.

\[ s_g^2 = 0.866^2 = 0.7500 \]

Divide by the girls' sample size

Why: each group is divided by its own n.

\[ \frac{s_g^2}{n_g} = \frac{0.7500}{9} = 0.0833 \]

The boys' term the same way

Why: one squared is one.

\[ \frac{s_b^2}{n_b} = \frac{1.00^2}{16} = 0.0625 \]

Add the two terms

Why: the variance of the difference.

\[ 0.0833 + 0.0625 = 0.1458 \]

Add before you take any root. The sum, 0.1458, is the variance of the gap in hours squared, which is why it looks so small; a variance is never the final answer, because its units are squared. The square root on the next line brings it back to hours.

Take the square root

Why: back to hours.

\[ \text{SE} = \sqrt{0.1458} = 0.3819 \text{ hours} \]

In context: if you drew many pairs of samples like these, the difference between the girls' mean and the boys' mean would typically wander about 0.38 hours, roughly 23 minutes, from the true difference. The girls' term is the larger of the two even though their spread is smaller, because only nine girls were sampled.

Check against the two bounds

Why: the answer must exceed each group's own standard error and fall short of their sum.

\[ 0.289 < 0.382 < 0.289 + 0.250 = 0.539 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 512-513 — Example 10.1 and Table 10.1

16. Variances add like squares on a right triangle

Picture it

The rule that variances add while standard errors do not has a picture you already know. Put the two standard errors at right angles as the legs of a triangle.

Figure (svg): A right triangle. Its horizontal leg, 0.289, is the standard error of the girls' mean, with an orange square of area 0.0833 drawn on it. Its vertical leg, 0.250, is the standard error of the boys' mean, with a blue square of area 0.0625. A purple square of area 0.1458 sits on the hypotenuse, whose length is 0.382.

The squares add; the sides do not. The standard error of the gap is the hypotenuse, longer than either leg and shorter than the two legs laid end to end.

Each leg's square has area equal to that group's variance term: 0.0833 for the girls and 0.0625 for the boys. Pythagoras says the squares on the legs add to the square on the hypotenuse, 0.1458, so the hypotenuse is its square root, 0.382. That is exactly the standard error formula.

The picture also shows the two bounds from the decoder slide. A hypotenuse is always longer than either leg, and always shorter than the two legs laid end to end, which would be 0.539. Adding standard errors directly walks the long way round the triangle.

Statisticians really do think of independent sources of variability as perpendicular. Neither one can cancel or reinforce the other on average, just as motion along one axis does nothing along the other.

17. How many standard errors apart?

Estimation

Example 10.1's sample means differ by 1.2 hours, and the standard error you just computed is 0.382 hours. Estimate before you calculate.

Predict first

Roughly how many standard errors apart are the girls' and boys' sample means?

  • About 3
  • About 1
  • About one third
  • About 12

Correct: About 3

Why: One point two divided by about 0.38 is a little over three. A gap of three standard errors is rare if the population means are equal, so this already suggests a small p-value.

\[ \frac{1.2}{0.382} \approx 3.1 \]

Rough benchmarks from Chapter 9 carry over: a gap of about one standard error is ordinary, about two is borderline at the 5 percent level, and three or more is strong evidence. Estimating this ratio first tells you what kind of p-value to expect before any software runs.

The wrong options are the classic slips: one third comes from dividing the wrong way, and 12 from forgetting the square root and dividing by the variance, 0.1458, instead of the standard error.

18. Slide the sample sizes and watch the standard error

Tweak it

The curve shows the standard error as group 1's sample size grows, with group 2 held fixed. Predict first: if you could add more people to one group only, which group should get them?

Parameter explorer

The curve is the standard error against n1. Set s1 and s2 to Example 10.1's 0.87 and 1, with n2 = 16. How far does the curve fall as n1 grows, and what floor does it level off at?

\[ \text{SE} = \sqrt{\frac{({s1})^2}{n_1} + \frac{({s2})^2}{{n2}}} \]

  • s1 — from 0.2 to 3: group 1 standard deviation s1
  • s2 — from 0.2 to 3: group 2 standard deviation s2
  • n2 — from 2 to 100: group 2 sample size n2

What to notice: with n2 fixed at 16, the curve levels off at 0.25, the boys' own standard error. Piling people into the girls' group cannot push the standard error below what the boys' sample alone contributes. The smaller or noisier group is the bottleneck.

So a fixed budget of extra participants does most good in the group whose variance term is currently larger. When the two populations have similar spreads, that means balancing the sample sizes, which is why so many studies use equal groups.

19. Find the error: a standard error that is too large

Error analysis

Someone computes Example 10.1's standard error like this. The number is plausible, which is what makes the slip dangerous.

Annotate

On: \( \text{SE} = \frac{0.866}{\sqrt{9}} + \frac{1.00}{\sqrt{16}} = 0.539 \)

  • It adds the two standard errors, 0.289 and 0.250, instead of adding the two variance terms under one root.
  • 0.539 is the sum of the legs of the triangle, not the hypotenuse. It overstates the uncertainty by about 40 percent.
  • t becomes −1.2/0.539 = −2.23 instead of −3.14: the evidence looks much weaker than it is.
  • Square, divide, add, then take one square root: √(0.0833 + 0.0625) = 0.382.

The opposite slip also happens: subtracting the variance terms because the estimate is a difference. That gives the square root of 0.0208, about 0.144, which is smaller than either group's own standard error and fails the first bound at once.

Both slips are caught by the same check. The correct answer must lie strictly between the larger of the two group standard errors, 0.289, and their sum, 0.539. An answer equal to the upper bound means you added the wrong things.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the standard error formula

20. The statistic and its degrees of freedom

Section

Part 3

21. The test statistic: a gap measured in standard errors

Concept

With the standard error in hand, the test statistic has the shape from the warm-up: the observed difference, minus the difference the null claims, divided by the standard error.

\[ t = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}} \]

Under the null hypothesis the difference μ1 minus μ2 is zero, so in every test in this section the middle term drops out and the statistic is simply the gap divided by its standard error.

\[ t = \frac{\bar{x}_1 - \bar{x}_2}{\text{SE}} = \frac{2 - 3.2}{0.3819} \]

\[ t = \frac{-1.2}{0.3819} = -3.14 \]

The sign follows the order of subtraction: girls minus boys is negative because the girls' sample mean is smaller. The size, a little over three, says the gap is about three standard errors, as you estimated.

Because the population standard deviations are unknown and estimated by s1 and s2, this statistic does not follow a normal curve exactly. The book says it is approximated by Student's t distribution, with a number of degrees of freedom that needs its own formula.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the t-score test statistic

22. Reading the t statistic

Notation

Every piece of the statistic has a job, and two of them are easy to get backwards. Read the notes before moving on.

Annotate

On: \( t = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\text{SE}} \)

  • The observed gap between the sample means, in the order the hypotheses fixed: group 1 minus group 2.
  • The gap the null hypothesis claims. In this section it is always 0, so this term vanishes.
  • √(s1²/n1 + s2²/n2): the typical size of a sample gap when the population gap is the null's value.
  • A pure number: how many standard errors the observed gap sits from the claimed gap. Its sign carries the direction.

The bars matter. The sample means, with bars, are numbers you calculate. The population means, Greek letters, are unknown, and the hypotheses are statements about them. A conclusion is always about the Greek letters, even though the arithmetic is all done with the barred ones.

Because t has no units, the same cutoffs apply whether the data are hours, exam points or years. That is the point of standardising: it puts every comparison on one scale.

Read a value of t as a sentence. Minus 3.14 for the girls and boys says: the girls' sample mean sits a little over three standard errors below the boys', a distance that equal population means would rarely produce.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — s1, s2 estimate σ1, σ2; x̄1, x̄2 are sample means

23. The Aspin-Welch degrees of freedom

Concept

A one-sample t had n minus 1 degrees of freedom. With two samples of different sizes and different spreads there is no simple count, and the book gives a formula developed by Aspin and Welch. Write the two variance terms as A and B to keep it readable.

\[ A = \frac{s_1^2}{n_1}, \qquad B = \frac{s_2^2}{n_2} \]

\[ df = \frac{(A + B)^2}{\dfrac{A^2}{n_1 - 1} + \dfrac{B^2}{n_2 - 1}} \]

The numerator is the square of the variance you already computed for the standard error. The denominator weights each group's term by its own degrees of freedom. The book calls this a somewhat complicated calculation that a calculator or computer does easily, and says the result is not always a whole number.

You should still do it by hand once, because seeing the pieces explains its behaviour. In particular, the result always lies between the smaller group's n minus 1 and the total n1 plus n2 minus 2, a fact the next slides use as a check.

The book adds two notes. When both sample sizes are five or larger, a Student's t approximation is very good. And the sample variances are not pooled; that instruction gets its own slide in this part.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 512-513 — the degrees of freedom formula and the note on n of five or more

24. Reading the degrees-of-freedom formula

Notation

Decode the formula once and you will know what to expect from any calculator output.

Annotate

On: \( df = \frac{(A + B)^2}{A^2/(n_1 - 1) + B^2/(n_2 - 1)} \)

  • The square of the variance of the gap, the same A + B that sits under the standard error's root.
  • Group 1's term squared, divided by group 1's own degrees of freedom. A noisy small group makes this large.
  • The same for group 2. The two pieces are added, never pooled into one sample.
  • Usually not a whole number. Between min(n1, n2) − 1 and n1 + n2 − 2.

When one group's term is much larger than the other's, that group dominates, and df falls towards that group's own n minus 1. When the two terms are balanced against their sample sizes, df rises towards n1 plus n2 minus 2. A later slider lets you push the formula to both ends.

Why should the degrees of freedom behave like that? Degrees of freedom measure how much information the standard error is built on. If almost all the uncertainty comes from one small group, the standard error is only as trustworthy as that group's s, and that s rests on its own n minus 1 values.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the Aspin-Welch degrees of freedom

25. Example 10.1: the degrees of freedom by hand

Worked example

Using Example 10.1's two variance terms, compute the Aspin-Welch degrees of freedom. The book's calculator reports about 18.8462.

The plan follows the formula's shape: square the total for the numerator; square each term and divide by its group's n minus 1 for the denominator; divide. Carry five or six decimals, because the denominator is small and rounding it early changes the answer.

\[ A = 0.083328, \qquad B = 0.0625 \]

Add the terms

Why: the same sum as under the standard error's root.

\[ A + B = 0.145828 \]

Square the sum for the numerator

Why: this is the variance of the gap, squared.

\[ (A + B)^2 = 0.145828^2 = 0.021266 \]

The girls' denominator piece

Why: A squared over nine minus one.

\[ \frac{A^2}{8} = \frac{0.006944}{8} = 0.000868 \]

The boys' denominator piece

Why: B squared over sixteen minus one.

\[ \frac{B^2}{15} = \frac{0.003906}{15} = 0.000260 \]

Look at the two pieces before adding them. The girls' piece is more than three times the boys', because the girls have the larger variance term and fewer degrees of freedom to divide it by. That imbalance is what pulls the result below the pooled 23.

Add the two pieces

Why: the full denominator.

\[ 0.000868 + 0.000260 = 0.001128 \]

Divide

Why: numerator over denominator.

\[ df = \frac{0.021266}{0.001128} = 18.85 \]

In context: the girls-against-boys comparison carries about 18.85 degrees of freedom. That matches the book's 18.8462 to the two decimals shown; carried unrounded, the computation gives 18.8466 from s equal to 0.866, and exactly 18.8462 from the square root of 0.75 that 0.866 rounds.

Check the bracket

Why: df must lie between the smaller group's n minus 1 and the pooled n1 + n2 − 2.

\[ 9 - 1 = 8 \; < \; 18.85 \; < \; 9 + 16 - 2 = 23 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 513 — Example 10.1: df is approximately 18.8462

26. Push one group's variance term to dominate

Edge cases

The bracket on the last slide is not a coincidence. Here the degrees of freedom are plotted against the ratio of the two variance terms, A divided by B, for two sample sizes you choose.

Parameter explorer

Set n1 = 9 and n2 = 16. What does df approach when the ratio is near 0, and when it is very large? Where is df highest?

\[ df = \frac{(r + 1)^2}{r^2/({n1} - 1) + 1/({n2} - 1)}, \quad r = A/B \]

  • n1 — from 3 to 40: group 1 sample size n1
  • n2 — from 3 to 40: group 2 sample size n2

What to notice: the curve starts at n2 minus 1, rises to a peak of exactly n1 plus n2 minus 2, and then sinks towards n1 minus 1. When one group's uncertainty swamps the other's, the comparison is only as informative as that one group, and df says so.

Example 10.1 sits at a ratio of about 1.33, close to the peak, which is why its 18.85 is near 23. The made-up comparison later in this part sits at a ratio of 1000, where df falls almost to the smaller group's 4.

27. Example 10.2's degrees of freedom

Fill the middle

College A: 11 graduates, standard deviation 1.5 math classes. College B: 9 graduates, standard deviation 1 class. The two variance terms are filled in; complete the sum and the degrees of freedom.

Fill in the blanks

A = \frac0.315717.40 = 0.2045, \; B = \frac______ = 0.1111, \; A + B = ___, \; df = \frac______ = ___

Why: The sum is 0.2045 plus 0.1111, about 0.3157, and its square is about 0.09964. The denominator is 0.2045 squared over 10 plus 0.1111 squared over 8, about 0.004184 plus 0.001543, which is 0.005727. Dividing gives about 17.40 degrees of freedom.

\[ \frac{0.0996}{0.0057} = 17.40, \qquad 9 - 1 = 8 < 17.40 < 18 = 11 + 9 - 2 \]

Check the bracket as always: 17.40 lies between 8 and 18, and close to the top, because college A's larger term comes with the larger sample. The old pooled rule would have said 18, and here the difference is small.

A calculator gives the same 17.40. Doing it once by hand is how you learn to spot a nonsense output, such as a df above n1 plus n2 minus 2, which can only mean the inputs were typed in wrongly.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 514-515 — Example 10.2's data

28. What the degrees of freedom change: the tails

Intuition

Why fuss over 18.85 against 23, or against 4? Because the degrees of freedom set the shape of the t curve, and the p-value is an area in its tails.

Figure (svg): Three bell curves centred at zero for t from minus 4.5 to 4.5. The dashed white standard normal curve is tallest. The blue t curve with 18.85 degrees of freedom lies almost on top of it, slightly lower with slightly thicker tails. The orange t curve with 4 degrees of freedom is visibly lower in the middle and higher in the tails.

With nearly nineteen degrees of freedom the t curve is close to normal. With four it is not: more of its area sits far out, so the same t value earns a larger p-value.

With few degrees of freedom the curve is lower in the middle and fatter in the tails, because the two sample standard deviations are themselves shaky estimates of σ1 and σ2. Fat tails mean a given t value cuts off more area, so the p-value is larger and the test is more cautious.

\[ P(T > 3.14 \mid df = 4) = 0.0174, \qquad P(T > 3.14 \mid df = 18.85) = 0.0027 \]

The same t of 3.14 gives a one-tailed area more than six times larger with 4 degrees of freedom than with 18.85. So an overstated df makes evidence look stronger than it is, and that is the real danger in the shortcut on the next slide.

As degrees of freedom grow, the t curve approaches the standard normal. That is the basis of the book's note, in Part 6, that large samples may use the normal instead.

29. Do not pool the variances

Concept

The book is blunt: notice that the sample variances are not pooled, and if the question comes up, do not pool them. Pooling means combining the two samples' variances into one estimate of a single shared spread.

\[ s_p^2 = \frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2} \]

That formula only makes sense if both populations have the same standard deviation, σ1 equal to σ2, so that both samples estimate one number. The Aspin-Welch test makes no such assumption. Its name in the book is the test for population standard deviations that are unknown and possibly unequal.

Figure (svg): Two normal curves on one axis from 20 to 95. A tall narrow blue curve, mean 50 and standard deviation 1, rises to a height of about 0.4. A low wide orange curve, mean 58 and standard deviation 10, is barely above the axis, spreading from about 30 to 88.

Pooling would replace these two very different spreads by one shared value. No single standard deviation describes both curves.

When the spreads really differ, as in this made-up pair, there is no single σ for a pooled estimate to estimate. The pooled value lands somewhere between the two, wrong for both groups, and the test built on it can be badly misleading.

The Business Statistics text covers the pooled test, assuming equal variances, in its own section. In this course, follow the instruction: on a calculator set Pooled to No.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 513 — do not pool the variances; the TI line: Pooled: No

OpenStax Introductory Business Statistics 2e, §10.3 Test for Differences in Means: Assuming Equal Population Variances §10.3 — the pooled test, which assumes equal population variances

30. When pooling goes badly wrong

Counterexample

Pooling feels harmless because it often changes little. Here is a made-up case where it changes everything: a small, very noisy group against a large, very tidy one.

\[ n_1 = 5, \; \bar{x}_1 = 58, \; s_1 = 10 \qquad n_2 = 50, \; \bar{x}_2 = 50, \; s_2 = 1 \]

Discussion prompt

Compute the two-tailed test of equal means both ways, with the Aspin-Welch standard error and df and with the pooled ones. Which conclusion should you trust at the 5 percent level, and why?

Notice how the Aspin-Welch degrees of freedom tell the same story. They are just over 4, the noisy group's own n minus 1, because that group's term, 20, is a thousand times the other group's 0.02. The comparison really does rest on five values.

This is why the book says do not pool. Pooling is a bet that the spreads are equal; when the bet is wrong and the sample sizes are unequal, it can turn a p-value of 0.15 into one below one in a million.

31. Trap: pooling because the arithmetic is shorter

Trap

The trap

For Example 10.1, a first attempt skips the Aspin-Welch formula:

\[ df = 9 + 16 - 2 = 23 \]

\[ s_p = 0.956, \quad t = -3.01 \]

\[ p = 0.0062 \]

Wrong. That is a different test, with an extra assumption.

The fix

The pooled test assumes σ1 equals σ2. This section's test does not, and the book says not to pool. Use the two separate variance terms and the Aspin-Welch df:

\[ df = 18.85, \quad t = -3.14 \]

\[ p = 0.0054 \]

Here the decision happens to be the same, because the spreads, 0.866 and 1, are similar. The counterexample showed it need not be. Never choose a test because its arithmetic is easier.

On a TI-83 or TI-84 the choice is one line on the 2-SampTTest screen, Pooled: No. Check it every time; a calculator left on Yes from another problem quietly runs the other test.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 513 — if the question comes up, do not pool the variances

32. Running the test

Section

Part 4

33. Six steps, one new statistic

Concept

The book's examples answer the same list of questions every time, and they are Chapter 9's steps with the new parts slotted in. Use the list as a template for every problem in this part.

  1. Name the groups. Two means or two proportions? Which group is group 1?
  2. Standard deviations of the populations known or unknown? Unknown here, so Student's t.
  3. Random variable: the difference of the sample means, group 1 minus group 2.
  4. Hypotheses in words and symbols, about μ1 − μ2, with the tail from the claim.
  5. Compute SE, t, the Aspin-Welch df, and the p-value in the tail Ha names.
  6. Decide against α, and conclude in context: the level, the direction, the populations.

\[ \text{reject } H_0 \iff p\text{-value} < \alpha \]

The p-value has its Chapter 9 meaning unchanged: the probability, if the population means were equal, of a sample gap at least as extreme as the one observed, in the direction the alternative names. Only the curve it is measured on is new.

The calculator route, 2-SampTTest on a TI-83 or TI-84 with Pooled set to No, does steps five's arithmetic in one go. The deck shows the arithmetic every time as well, so you can check the calculator and so you do not need one to follow.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 513-517 — the question lists of Examples 10.2 and 10.3, and the TI instructions

34. Example 10.1: girls, boys and hours of sport

Worked example

Is there a difference in the mean time girls and boys aged seven to eleven play sport each day? Nine girls average 2 hours, s 0.866; sixteen boys average 3.2 hours, s 1.00. Both populations are normal. Test at the 5 percent level.

The work of Parts 2 and 3 slots in here. Set up the hypotheses from the wording, reuse the standard error and degrees of freedom already found, and then do the one genuinely new thing, reading the p-value from both tails.

Name the groups and the hypotheses

Why: the same tells you the null has an equals sign; no direction is given, so two-tailed.

\[ H_0: \mu_g - \mu_b = 0 \qquad H_a: \mu_g - \mu_b \ne 0 \]

The random variable

Why: girls' sample mean minus boys', in that order.

\[ \bar{X}_g - \bar{X}_b, \qquad \bar{x}_g - \bar{x}_b = 2 - 3.2 = -1.2 \]

The standard error, from Part 2

Why: the two variance terms added under one root.

\[ \text{SE} = \sqrt{0.0833 + 0.0625} = 0.3819 \]

The test statistic

Why: the gap in standard errors.

\[ t = \frac{-1.2}{0.3819} = -3.14 \]

The distribution

Why: Aspin-Welch degrees of freedom, not pooled.

\[ T \sim t_{18.85} \]

The p-value, both tails

Why: half the area is below minus 3.14 and half above plus 3.14.

\[ p = 2P(T < -3.14) = 2(0.0027) = 0.0054 \]

Why double? The alternative says only that the means differ, so a gap of 1.2 hours in either direction would count against the null. The p-value must include the equally extreme gap the other way, boys below girls by 1.2, and the t curve is symmetric, so that area is the same.

Figure (svg): A t curve with 18.85 degrees of freedom. Both tails beyond minus 3.14 and plus 3.14 are shaded yellow, together 0.0054. Red dashed lines mark the 5 percent critical values, plus and minus 2.094, well inside the shaded tails.

The observed t of minus 3.14 lies beyond the critical value, so its tails are smaller than the 5 percent the test allows.

Decide

Why: the p-value is smaller than alpha.

\[ 0.0054 < 0.05 \Rightarrow \text{reject } H_0 \]

Conclusion in context: at the 5 percent level, the sample data give sufficient evidence that the mean number of hours girls and boys aged seven to eleven play sport per day is different. The samples point to boys playing more, about 1.2 hours a day more on average.

Check with the critical value

Why: a two-tailed 5 percent test rejects when t lies beyond the critical value for 18.85 df.

\[ |-3.14| = 3.14 > 2.094 = t_{0.025} \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 513-514 — Example 10.1 and Figure 10.2

35. Example 10.1 on the scale of hours

Picture it

The book draws this test on the original scale rather than on t: half the p-value is below minus 1.2 and half is above 1.2, in hours. Both pictures carry the same areas, and this one keeps the context in view.

Figure (svg): A bell curve for the difference of sample means if the true difference is zero, centred at 0 hours, with standard error 0.382 hours. The tails beyond minus 1.2 and plus 1.2 hours are shaded yellow; a dot marks the observed minus 1.2.

If girls and boys really played the same amount, sample gaps would mostly fall within about three quarters of an hour of zero. A gap of 1.2 hours lies far out in the tails.

The curve shows where the difference of sample means would land if girls and boys really averaged the same. Its centre is zero, the null's claim, and its spread is the standard error, 0.382 hours. Most gaps would fall within about three quarters of an hour either side.

The observed gap, minus 1.2 hours, sits out in the left tail. The shaded areas beyond minus 1.2 and plus 1.2 together make the p-value, 0.0054. Rescaling the axis by the standard error turns this picture into the t curve on the previous slide, with minus 1.2 hours becoming minus 3.14.

Drawing a test this way is the best defence against misreading it. You can see the null's world, see where the data actually fell, and see how small the shaded tails are, all in the problem's own units.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 513 — Figure 10.2: half the p-value below −1.2, half above 1.2

36. Try It 10.1: populations A and B

Worked example

Sample A: n = 25, mean 5, s = 1. Sample B: n = 16, mean 4.7, s = 1.2. Both populations are normal, and their means are thought to be the same. Is there a difference? Test at the 5 percent level.

No direction is claimed, so this is two-tailed like Example 10.1. Before computing, compare the gap of 0.3 with the spreads of about 1: a gap that small against spreads that size should not be surprising.

Hypotheses

Why: the same, with no direction: two-tailed.

\[ H_0: \mu_A - \mu_B = 0 \qquad H_a: \mu_A - \mu_B \ne 0 \]

The two variance terms

Why: each variance over its own n.

\[ A = \frac{1^2}{25} = 0.04, \qquad B = \frac{1.2^2}{16} = \frac{1.44}{16} = 0.09 \]

The standard error

Why: add, then take the root.

\[ \text{SE} = \sqrt{0.04 + 0.09} = \sqrt{0.13} = 0.3606 \]

The test statistic

Why: A minus B.

\[ t = \frac{5 - 4.7}{0.3606} = \frac{0.3}{0.3606} = 0.832 \]

Degrees of freedom

Why: the numerator is 0.13 squared.

\[ df = \frac{0.0169}{0.04^2/24 + 0.09^2/15} = \frac{0.0169}{0.000607} = 27.86 \]

The p-value, both tails

Why: twice the area beyond 0.832.

\[ p = 2P(T > 0.832) = 0.4125 \]

Here group B has the larger variance term, 0.09 against 0.04, despite being the smaller sample, so its 15 degrees of freedom weigh most in the denominator. The result, about 28, still sits comfortably inside the bracket from 15 to 39.

Figure (svg): A t curve with 27.86 degrees of freedom. The tails beyond minus 0.83 and plus 0.83 are shaded yellow and cover a large part of the curve, together 0.4125.

The shaded tails hold about 41 percent of the area. A t this close to zero is what equal means produce all the time.

Decide

Why: the p-value is far larger than alpha.

\[ 0.4125 > 0.05 \Rightarrow \text{do not reject } H_0 \]

Conclusion in context: at the 5 percent level, there is not sufficient evidence that the two population means differ. That is not the same as showing they are equal; the data simply cannot tell them apart.

Check the size of t against the gap

Why: a gap smaller than one standard error must give a large p-value.

\[ 0.3 < 0.361 \Rightarrow |t| < 1, \quad 16 - 1 < 27.86 < 39 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 514 — Try It 10.1 and Table 10.2; the book prints no answer in the section

37. Example 10.2: math classes at two colleges

Worked example

College A samples 11 graduates: mean 4 math classes, s 1.5. College B samples 9: mean 3.5, s 1. A community group believes A's graduates take more math classes on average. Both populations are normal. Test at the 1 percent level.

The book splits this into questions (a) to (i). The first three are classification: two means, standard deviations unknown, so Student's t. The rest is the test. Group 1 is college A, because the claim is about A having more.

(d) The random variable

Why: A's sample mean minus B's.

\[ \bar{X}_A - \bar{X}_B, \qquad \bar{x}_A - \bar{x}_B = 4 - 3.5 = 0.5 \]

(e) Hypotheses

Why: more is the claim, so it goes in the alternative.

\[ H_0: \mu_A \le \mu_B \qquad H_a: \mu_A > \mu_B \]

(f) The tail

Why: greater than points right.

\[ H_a: \mu_A - \mu_B > 0 \Rightarrow \text{right-tailed} \]

The null is written with less than or equal to because it must cover everything the claim denies, but the calculation uses its boundary, a difference of exactly zero. If a gap of 0.5 is not surprising when the true difference is zero, it is even less surprising when B's mean is the larger.

The standard error

Why: variance terms from the fill-in slide.

\[ \text{SE} = \sqrt{0.2045 + 0.1111} = \sqrt{0.3157} = 0.5618 \]

The statistic and df

Why: df from the fill-in slide.

\[ t = \frac{0.5}{0.5618} = 0.890, \qquad df = 17.40 \]

(g) The p-value

Why: the area to the right of 0.890.

\[ p = P(T > 0.890) = 0.1928 \]

Figure (svg): A t curve with 17.40 degrees of freedom. The right tail beyond 0.89 is shaded yellow, area 0.1928. A red dashed line far to the right marks the 1 percent critical value, 2.561.

To reject at the 1 percent level, t would have to pass 2.56. The observed 0.89 is nowhere near it.

(h) Decide

Why: the p-value is far above alpha.

\[ 0.1928 > 0.01 \Rightarrow \text{do not reject } H_0 \]

(i) Conclusion in context: at the 1 percent level, from the sample data, there is not sufficient evidence to conclude that a student who graduates from college A has taken more math classes, on average, than a student who graduates from college B.

Check with the critical value

Why: a right-tailed 1 percent test needs t beyond the 99th percentile of t on 17.40 df.

\[ 0.890 < 2.561 = t_{0.01} \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 514-516 — Example 10.2 (a)-(i) and Figure 10.3

38. Writing the conclusion: level, evidence, direction, context

Concept

Every example in the book ends with a sentence of the same shape, and Example 10.3 even prints it as a template to fill in. Learn the template; most lost marks on two-sample tests are lost in this one sentence.

\[ \text{At the } \alpha \text{ level, there is / is not sufficient evidence that } \ldots \]

The level comes first, because the decision depends on it: Example 10.2's p-value of 0.19 fails at 1 percent and would fail at 5 percent too, but a p-value of 0.03 would pass one and fail the other. Then comes is or is not sufficient evidence, never proves.

Then the claim, stated about the populations and in their units: the mean number of math classes taken by graduates of college A, not the eleven graduates sampled. The sample means are known exactly; there is nothing to conclude about them.

Finally, when the test is two-tailed and you reject, add the direction. Example 10.1's book conclusion says the means are different, then notes it could be either group that plays more. The samples settle which: boys averaged 3.2 hours to girls' 2, so say boys play more.

And when you do not reject, say not sufficient evidence of a difference, never that the means are equal. Absence of evidence for a gap is not evidence that there is none, a distinction the error-analysis slide at the end of this part makes concrete.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 514-517 — the conclusions of Examples 10.1-10.3

39. Before any arithmetic: which class is group 1?

Step zero

Example 10.3 is the professor's question about online and face-to-face finals, with all sixty scores given. The book adds a note: be careful not to mix up the information for Group 1 and Group 2.

Discussion prompt

The professor believed the online class's mean would be lower. Choose group 1, then write the alternative hypothesis and say which sign you expect t to have if the professor is right.

The book chooses online as group 1, and the deck follows it. Writing that choice down before touching the data is the step-zero habit: it fixes the order of subtraction, the sign of t and the tail all at once, so none of them can drift later.

A good sense check falls out for free. Look at the dot plot from Part 1: the online scores sit lower. So with online as group 1, you expect a negative t. If your calculator later shows a positive t, the lists were entered the wrong way round.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 517 — be careful not to mix up the information for Group 1 and Group 2

40. Example 10.3: online against face-to-face, from raw data

Worked example

Thirty online final exam scores and thirty face-to-face scores were drawn at random. Is the online mean lower? Test at the 5 percent level. Group 1 is online, from the step-zero slide, so the test is left-tailed.

Raw data adds one stage: computing each group's mean and standard deviation. The book does this with a calculator's list entry; the sums below are the same arithmetic written out, so each number can be traced to the tables.

Sum and mean of the online scores

Why: add all thirty, divide by thirty.

\[ \bar{x}_1 = \frac{2185.5}{30} = 72.85 \]

Sum and mean of the face-to-face scores

Why: the same for the other table.

\[ \bar{x}_2 = \frac{2549.4}{30} = 84.98 \]

Standard deviations

Why: the sum of squared deviations from each mean, over n minus 1, then the root.

\[ s_1 = \sqrt{\frac{8301.09}{29}} = 16.92, \quad s_2 = \sqrt{\frac{3979.43}{29}} = 11.71 \]

Notice the spreads differ: the online scores have a standard deviation about 5 points larger, which the dot plot showed as a longer tail of low scores. That is a good reason not to assume one shared spread, and the Aspin-Welch test does not.

The two variance terms

Why: each variance over thirty.

\[ A = \frac{16.92^2}{30} = 9.541, \qquad B = \frac{11.71^2}{30} = 4.574 \]

The standard error

Why: add and take the root.

\[ \text{SE} = \sqrt{9.541 + 4.574} = 3.757 \]

The test statistic

Why: online minus face-to-face.

\[ t = \frac{72.85 - 84.98}{3.757} = \frac{-12.13}{3.757} = -3.229 \]

Degrees of freedom

Why: the Aspin-Welch formula with n minus 1 = 29 for both.

\[ df = \frac{199.25}{3.139 + 0.721} = 51.61 \]

The p-value, left tail

Why: the area below minus 3.23.

\[ p = P(T < -3.229) = 0.0011 \]

Figure (svg): A t curve with 51.61 degrees of freedom. The left tail below minus 3.23 is shaded yellow, a sliver of area 0.0011. A red dashed line marks the 5 percent critical value, -1.675.

The online class's mean sits more than three standard errors below the face-to-face mean. Under equal means a gap that far down happens about once in a thousand samples.

Decide and conclude: 0.0011 is below 0.05, so reject the null. At the 5 percent level there is sufficient evidence that the mean final exam score of the online class is less than that of the face-to-face class. The professor was correct, by about twelve points.

Check with the critical value

Why: a left-tailed 5 percent test rejects when t falls below the 5th percentile.

\[ -3.229 < -1.675 = -t_{0.05}, \quad \text{and } 0.0011 \text{ matches the book's } 0.0011 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 516-517 — Example 10.3, Tables 10.3-10.4 and Figure 10.4

41. Try It 10.2: how long workers stay

Worked example

Company A samples 15 workers: mean 5 years with the company, s 1.2. Company B samples 20: mean 4.5 years, s 0.8. Populations normal. Does A retain its workers longer? Test at the 5 percent level.

Part (a) asks whether the population standard deviations are known: no, only sample values are given, so this is the t test. The claim, longer, points right with A as group 1.

Hypotheses

Why: longer goes in the alternative.

\[ H_0: \mu_A \le \mu_B \qquad H_a: \mu_A > \mu_B \]

The variance terms

Why: variance over n for each company.

\[ A = \frac{1.2^2}{15} = \frac{1.44}{15} = 0.096, \qquad B = \frac{0.8^2}{20} = \frac{0.64}{20} = 0.032 \]

The standard error

Why: 0.096 plus 0.032 is 0.128.

\[ \text{SE} = \sqrt{0.128} = 0.3578 \]

The test statistic

Why: A minus B.

\[ t = \frac{5 - 4.5}{0.3578} = 1.398 \]

Degrees of freedom

Why: 0.128 squared on top.

\[ df = \frac{0.016384}{0.096^2/14 + 0.032^2/19} = 23.01 \]

The p-value, right tail

Why: the area beyond 1.398.

\[ p = P(T > 1.398) = 0.0878 \]

Before reading the p-value, estimate it: t is 1.40, a little short of the one-tailed 5 percent benchmark of about 1.7. So expect a tail somewhat above 0.05, perhaps 0.08 or 0.09, and a decision not to reject.

Figure (svg): A t curve with 23.01 degrees of freedom. The right tail beyond 1.40 is shaded yellow, area 0.0878. A red dashed line marks the 5 percent critical value, 1.714, a little further right.

Close, but not past the line: the tail holds about 9 percent, more than the 5 percent the test allows.

Decide and conclude: 0.0878 is above 0.05, so do not reject. At the 5 percent level there is not sufficient evidence that Company A retains its workers longer, on average, than Company B. At the 10 percent level the decision would flip, which is exactly why the level belongs in the sentence.

Check with the critical value

Why: the 5 percent right-tail cutoff on 23 df.

\[ 1.398 < 1.714 = t_{0.05}, \quad 14 < 23.01 < 33 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 516 — Try It 10.2; no answer is printed in the section

42. Try It 10.3: two professors' exam scores

Worked example

Ten final exam scores are drawn from each of two professors' classes. Professor A says the mean of their class is more than the mean of professor B's class. Test at the 5 percent level, with A as group 1.

Figure (svg): Two dot plots of ten exam scores each, 50 to 100 points, bins of five. Professor A's scores have mean 76.5; professor B's have mean 74.6. The two rows overlap almost completely.

A gap of under two points between means, inside spreads of about thirteen points. The picture already says the evidence will be weak.

The picture already suggests the answer: the dots overlap almost completely and the means differ by under two points. Run the test anyway, because a picture can mislead when the samples are small.

One condition needs a look first. With only ten scores per class, the test needs roughly normal populations, and this Try It does not say so. The dot plots show no extreme skew or outliers, so the t test is reasonable, but say that you checked.

Means

Why: each class's ten scores summed, over ten.

\[ \bar{x}_A = \frac{765}{10} = 76.5, \qquad \bar{x}_B = \frac{746}{10} = 74.6 \]

Standard deviations

Why: squared deviations over 9, then the root.

\[ s_A = \sqrt{\frac{1514.5}{9}} = 12.97, \quad s_B = \sqrt{\frac{1320.4}{9}} = 12.11 \]

The standard error

Why: each variance over ten.

\[ \text{SE} = \sqrt{16.83 + 14.67} = 5.612 \]

Statistic, df and p-value

Why: right-tailed, since A claims more.

\[ t = \frac{1.9}{5.612} = 0.339, \; df = 17.92, \; p = 0.3694 \]

Decide and conclude: 0.369 is far above 0.05, so do not reject. At the 5 percent level, the sample data do not give sufficient evidence that professor A's class has the higher mean final exam score. A gap of 1.9 points is a third of a standard error.

Check the bracket and the cutoff

Why: df must sit between 9 and 18, and t must fall short of the 5 percent cutoff.

\[ 9 < 17.92 < 18, \qquad 0.339 < 1.735 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 517-518 — Try It 10.3, Tables 10.5-10.6

43. Order the tests by strength of evidence

Ranking

Five tests from this part, each summarised by its statistic, degrees of freedom and tail. Order them from the strongest evidence against the null to the weakest, which means from the smallest p-value to the largest.

Put in order

Order from strongest evidence (smallest p-value) to weakest.

  1. online vs face-to-face: t = −3.23, df 51.6, left-tailed
  2. girls vs boys: t = −3.14, df 18.8, two-tailed
  3. Company A vs B: t = 1.40, df 23.0, right-tailed
  4. college A vs B: t = 0.89, df 17.4, right-tailed
  5. populations A vs B: t = 0.83, df 27.9, two-tailed

Why: The p-values are 0.0011, 0.0054, 0.0878, 0.1928 and 0.4125. A larger t in the direction of the alternative means a smaller tail, and a two-tailed test doubles its tail, which is why the last item comes after the college test despite a similar t.

\[ 0.0011 < 0.0054 < 0.0878 < 0.1928 < 0.4125 \]

The top two are the interesting pair. Their t values are close, 3.23 and 3.14 in size, but the first has more degrees of freedom, so thinner tails, and is one-tailed, so it counts only one tail. Both effects make its p-value smaller.

The bottom two show the opposite pull. The college test and the populations test have almost the same t, but the two-tailed test adds the area from both sides, roughly doubling its p-value. Ordering by t alone would get this pair wrong.

44. Find the error: a conclusion that says too much

Error analysis

Here is a conclusion written for Example 10.2, the two colleges. The arithmetic behind it is right; the final step is not.

Annotate

On: \( p = 0.1928 > 0.01 \Rightarrow \mu_A = \mu_B \)

  • Correct: the right-tail area beyond t = 0.890 on 17.40 df.
  • Correct comparison: the p-value exceeds α, so H0 is not rejected.
  • The error. Not rejecting does not prove the null; the data are merely consistent with it.
  • There is not sufficient evidence that A's graduates take more math classes on average.

A large p-value says the data could easily have come from equal means. It does not say the means are equal: the data could just as easily have come from a true gap of half a class, which is what the samples estimated. With eleven and nine graduates, the test simply cannot tell.

The same slip appears in a quieter form: the colleges are the same. Both claim far more than a failure to reject supports. Chapter 9 put it as the decision being do not reject, never accept, and the reason is this one.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 516 — Example 10.2(i): there is not sufficient evidence to conclude

45. How big is the difference? Cohen's d

Section

Part 5

46. Effect size: Cohen's d

Concept

A hypothesis test answers one question: is there evidence of any difference at all? It does not say whether the difference is big. For that the book introduces Cohen's d, named for the statistician Jacob Cohen, which measures the gap between the means in units of the spread.

\[ d = \frac{\bar{x}_1 - \bar{x}_2}{s_p}, \qquad s_p = \sqrt{\frac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}} \]

The denominator is the pooled standard deviation. That may look like it contradicts the rule against pooling, but it does not: d is a description of the samples, not a test, and it needs one typical spread to measure the gap against. The test still uses the unpooled standard error.

size of effectd
small0.2
medium0.5
large0.8

Figure (svg): Three panels, each with two bell curves of equal spread. In the first the curves are 0.2 standard deviations apart and almost coincide; in the second they are 0.5 apart; in the third 0.8 apart, with a visible gap between their peaks but still a large overlap.

Even a large effect, d of 0.8, leaves the two groups overlapping heavily. Cohen's d measures how far apart the centres are in units of the spread.

Cohen's standards come with pictures worth keeping in mind. Even a large effect, 0.8 standard deviations, leaves the two populations overlapping heavily. The size is judged by the absolute value of d, since which group is subtracted from which only changes its sign.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 518 — Cohen's d and Table 10.7

47. Example 10.4: Cohen's d for the two colleges

Worked example

Calculate Cohen's d for Example 10.2: college A, n 11, mean 4, s 1.5; college B, n 9, mean 3.5, s 1. Is the effect small, medium or large, and what does that mean here?

Pool the two variances, weighting each by its degrees of freedom, take the root, and divide the gap by it. The book lists the sample means as μ1 and μ2; they are sample means, x-bar, since the population means are what the test is about and are unknown.

Weight each variance by n minus 1

Why: college A's ten and college B's eight.

\[ (11 - 1)(1.5)^2 = 10(2.25) = 22.5, \qquad (9 - 1)(1)^2 = 8 \]

Add and divide by the total df

Why: 11 plus 9 minus 2 is 18.

\[ s_p^2 = \frac{22.5 + 8}{18} = \frac{30.5}{18} = 1.6944 \]

Take the root

Why: the pooled standard deviation, in classes.

\[ s_p = \sqrt{1.6944} = 1.3017 \]

Divide the gap by it

Why: the gap is 4 minus 3.5.

\[ d = \frac{0.5}{1.3017} = 0.384 \]

In context: graduates of college A took, on average in these samples, about four tenths of a standard deviation more math classes. By Cohen's standard that is a small effect, between 0.2 and 0.5. The book adds that this indicates there is not a significant difference; that runs two separate ideas together, as the trap slide in this part shows.

Check the pooled SD lies between the two

Why: a weighted average of the variances must sit between them.

\[ 1 < 1.302 < 1.5, \qquad \text{and } 0.2 < 0.384 < 0.5 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 519 — Example 10.4; the book prints d = 0.384

48. Example 10.5: Cohen's d for the online and face-to-face classes

Worked example

Calculate Cohen's d for Example 10.3: online, n 30, mean 72.85, s 16.92; face-to-face, n 30, mean 84.98, s 11.71. Is the effect small, medium or large?

With equal sample sizes the weights are equal, so the pooled variance is just the average of the two variances. That shortcut only works when n1 equals n2.

The two variances

Why: square each standard deviation.

\[ s_1^2 = 16.92^2 = 286.24, \qquad s_2^2 = 11.71^2 = 137.22 \]

Pool them

Why: equal weights of 29, so an ordinary average.

\[ s_p^2 = \frac{29(286.24) + 29(137.22)}{58} = 211.73 \]

The pooled standard deviation

Why: back to points.

\[ s_p = \sqrt{211.73} = 14.55 \]

Divide the gap by it

Why: use the size of the gap.

\[ d = \frac{|72.85 - 84.98|}{14.55} = \frac{12.13}{14.55} = 0.834 \]

The figure draws what d means. Two curves with the shared spread of 14.55 points, their centres 12.13 points apart. The gap is a large slice of one spread, but far from all of it, which is why the curves still overlap so much.

Figure (svg): Two bell curves with the same spread, 14.55 points, the pooled standard deviation. The orange curve is centred at the online mean 72.85, the blue at the face-to-face mean 84.98. A double arrow between the centres is labelled 12.13 points, which is 0.834 pooled standard deviations.

A large effect by Cohen's standard, and still the curves share much of their ground: plenty of online students outscore plenty of face-to-face ones.

In context: the face-to-face class scored about 0.83 pooled standard deviations higher, a large effect by Cohen's standard since 0.834 exceeds 0.8. The book prints d as 0.834 without a sign; online minus face-to-face is negative, so it is reporting the size.

Check against the picture

Why: 0.834 standard deviations of 14.55 points should be the twelve-point gap.

\[ 0.834 \times 14.55 = 12.14 \approx 12.13 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 519 — Example 10.5: d = 0.834, large

49. Try It 10.4: Cohen's d for the two companies

Worked example

Calculate Cohen's d for Try It 10.2: Company A, n 15, mean 5 years, s 1.2; Company B, n 20, mean 4.5, s 0.8. Is the effect small, medium or large?

This time the sample sizes differ, so the weights differ: Company B's variance counts nineteen times, Company A's fourteen.

Weight each variance

Why: n minus 1 times s squared.

\[ 14(1.2)^2 = 14(1.44) = 20.16, \qquad 19(0.8)^2 = 19(0.64) = 12.16 \]

Pool

Why: divide by 15 plus 20 minus 2, which is 33.

\[ s_p^2 = \frac{20.16 + 12.16}{33} = \frac{32.32}{33} = 0.9794 \]

A pooled variance just under one means a pooled standard deviation just under one year, so a gap of half a year should give a d a little above one half. Predicting that before dividing is a quick way to catch a slip.

Root, then divide the gap

Why: the gap is half a year.

\[ s_p = 0.9896, \qquad d = \frac{0.5}{0.9896} = 0.505 \]

In context: Company A's workers stayed about half a standard deviation longer in these samples, a medium effect by Cohen's standard, just past 0.5. Yet the test in Try It 10.2 did not reject at 5 percent. A medium-sized effect with 15 and 20 workers is not enough evidence; with larger samples it could be.

Check the pooled SD lies between the two

Why: a weighted average of 0.64 and 1.44 must sit between them, nearer the more heavily weighted 0.64.

\[ 0.8 < 0.990 < 1.2 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 519 — Try It 10.4; the book prints no answer in the section

50. Trap: reading the effect size as the test result

Trap

The trap

Example 10.4's own interpretation reads:

\[ d = 0.384 \Rightarrow \text{small} \]

\[ \Rightarrow \text{not a significant difference} \]

Wrong as a rule, though the test here also failed to reject.

The fix

Significance depends on the sample sizes; d does not. The same d of 0.384 with 200 graduates per college gives:

\[ t \approx 0.384 \sqrt{100} = 3.84 \]

\[ p = 0.00007 \]

A small effect, and highly significant. And Try It 10.4 gave a medium effect that was not significant. The test asks whether there is a gap; d asks how big it is. Report both, and never infer one from the other.

The reason is in the formulas: t grows with the square root of the sample sizes, while d has no n in it at all.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 519 — Examples 10.4 and 10.5

51. Small, medium or large?

Sorting

Here are the effect sizes for every two-group comparison in this deck, computed with the pooled standard deviation. Sort each by Cohen's standards, using the size of d and the cutoffs 0.2, 0.5 and 0.8.

Sort into buckets

Sort each comparison by its effect size.

below small (under 0.2)
bank weighted alpha, northeast vs west: d = 0.040; professor A vs professor B: d = 0.151
small (0.2 to 0.5)
college A vs college B: d = 0.384
medium (0.5 to 0.8)
Company A vs Company B: d = 0.505
large (0.8 or more)
online vs face-to-face: d = 0.834; girls vs boys, hours of sport: d = 1.256
n
Both are under 0.2, smaller than Cohen's small effect: the groups' centres are almost on top of each other relative to their spreads.
s
At 0.384, between 0.2 and 0.5: a small effect, the book's own reading of Example 10.4.
m
Just past 0.5: a medium effect, half a standard deviation between the means.
l
Both pass 0.8. The girls and boys gap of 1.2 hours is well over one pooled standard deviation of 0.96 hours.

Now compare with the test results. The two large effects were both significant, and the two below small were both far from it. But the middle two show the trap: a small effect and a medium effect, and neither was significant at its level. Significance follows the evidence, not the size.

The girls and boys effect is worth a moment. Its d of 1.26 comes from a gap of 1.2 hours over a pooled standard deviation of 0.956 hours: the populations would overlap far less than any of Cohen's pictures.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 518 — Table 10.7, Cohen's standard effect sizes

52. Try It 10.5: weighted alpha of bank stocks

Worked example

Weighted alpha measures a stock's risk-adjusted performance over a year. The top 30 bank stocks in the northeast and in the west, as identified by Nasdaq on 24 May 2013, are listed in the book. Is there a difference in mean weighted alpha? Test at 5 percent, then find Cohen's d.

Figure (svg): Two dot plots of 30 weighted alphas each, from 20 to 130, bins of five. The northeast banks have mean 47.96; the western banks have mean 47.06, with two values above 110. Both rows pile up at the low end and trail off to the right.

Both groups are skewed right, and their means are less than one point apart. Thirty in each group is large enough that the skew does not stop the test.

Look at the western row's two values above 110. Those two stocks alone inflate the western standard deviation, which is why it comes out about seven points larger than the northeast's, though the two clouds look alike otherwise.

No direction is claimed, so two-tailed, with northeast as group 1. Both dot plots are skewed right, but with thirty in each group the test does not require normal populations, as Part 6 explains.

Means

Why: each table's thirty values summed, over thirty.

\[ \bar{x}_1 = \frac{1438.9}{30} = 47.96, \qquad \bar{x}_2 = \frac{1411.7}{30} = 47.06 \]

Standard deviations

Why: squared deviations over 29, then the root.

\[ s_1 = 18.99, \qquad s_2 = 26.23 \]

The standard error

Why: each variance over thirty.

\[ \text{SE} = \sqrt{12.02 + 22.93} = 5.912 \]

Statistic, df and p-value

Why: two-tailed, so double the tail.

\[ t = \frac{0.91}{5.912} = 0.153, \; df = 52.86, \; p = 0.8787 \]

Cohen's d

Why: the gap over the pooled standard deviation.

\[ s_p = 22.90, \qquad d = \frac{0.91}{22.90} = 0.040 \]

Conclusion in context: at the 5 percent level there is not sufficient evidence of a difference in mean weighted alpha between the top 30 northeast and western bank stocks. And Cohen's d of 0.04 says the sample means are almost identical relative to the spread: the effect is far below small.

Check with the critical value

Why: a two-tailed 5 percent test needs t beyond the 97.5th percentile.

\[ 0.153 < 2.006, \qquad 29 < 52.86 < 58 \]

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 519-520 — Try It 10.5, Tables 10.8-10.9

53. Conditions and the normal approximation

Section

Part 6

54. What the test needs

Concept

The book lists the conditions at the top of the section, before any formula. They decide whether the p-value means anything at all, so check them first, not last.

  1. The two independent samples are simple random samples from two distinct populations.
  2. If the sample sizes are small, the population distributions matter: they should be normal.
  3. If the sample sizes are large, the distributions are not important: they need not be normal.

The first condition is about how the data were collected, and no sample size repairs it. If the online students chose their format, and stronger students tend to choose face-to-face, then the gap may reflect who signed up rather than the format. A random sample from each population says nothing about why the populations differ.

The second and third are the central limit theorem from Chapter 7 at work. Each sample mean is close to normal when its sample is large, whatever the population's shape; when a sample is small, only a normal population guarantees it. That is why Examples 10.1 and 10.2, with samples of 9 to 16, state that both populations are normal.

Notice what is not on the list: equal standard deviations, and equal sample sizes. The Aspin-Welch test needs neither, which is the whole reason for its more complicated degrees of freedom.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 512 — the two conditions at the head of the section

55. When a normal curve will do, and why to use t anyway

Concept

The book notes that when the sum of the sample sizes is larger than 30, you can use the normal distribution to approximate Student's t distribution. Then it adds: use Student's t distribution, however, whenever possible.

\[ n_1 + n_2 > 30 \Rightarrow \text{normal approximation allowed} \]

The reason for the caution is in the tails. The t curve always has more area far out than the normal, and p-values live far out. Here is Example 10.1's tail beyond 3.14, on both curves.

Figure (svg): The right tails of two curves from t equals 2 to 4.5. The blue t curve with 18.85 degrees of freedom stays above the dashed normal curve. Beyond 3.14 the t tail, shaded orange, has area 0.0027; the normal tail, shaded blue, has area 0.0008.

Far out, the t curve holds about three times as much area as the normal. Using the normal there would make the evidence look stronger than it is.

Example 10.1 has n1 plus n2 equal to 25, so the approximation is not even allowed, and the picture shows why: the normal would give a two-tailed p-value of 0.0017 against the correct 0.0054, more than three times too small.

\[ \text{Example 10.3: } P(T < -3.23) = 0.0011, \quad P(Z < -3.23) = 0.0006 \]

Example 10.3, with sixty scores, qualifies, and the decision is the same either way. But the normal p-value is still about half the t one. Any error from the shortcut always runs in the same direction, overstating the evidence, so there is no reason to take it when a calculator gives the t for free.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, p. 514 — the note on n1 + n2 > 30

56. Does this section's test apply?

Discrimination

For each study, decide whether the two-sample t test of this section applies as it stands, applies only if the populations can be taken as normal, or is the wrong test altogether.

Sort into buckets

Sort each study by whether this section's test fits.

applies as it stands
40 randomly chosen commuters in each of two cities; mean commute times, skewed; 35 and 50 randomly sampled bulbs from two factories, mean lifetimes; 9 girls and 16 boys, hours of sport, from normal populations
applies only if populations are normal
6 plots with fertiliser and 7 without, crop yield, distribution unknown
wrong test
each patient's blood pressure before and after a drug; proportion of voters supporting a measure in two counties
y
Independent random samples of means. Where the samples are large, skew does not matter; where they are small, as for the girls and boys, the book states the populations are normal.
n
Six and seven plots are small samples, and nothing is said about normality. Check a dot plot for strong skew or outliers before trusting the p-value.
w
Before and after on the same patients is matched pairs, Section 10.4. Proportions of voters are not means at all, Section 10.3.

The girls and boys study is the reminder that small samples are fine when normality is given. The fertiliser study is the warning: thirteen plots and no statement about shape means you must look at the data before you run the test.

The two wrong tests are the most common mix-ups in the chapter. Ask two questions in order: are the samples independent, and is the parameter a mean? Only two yes answers lead to this section.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10 introduction, pp. 511-512; §10.1, p. 512 — the chapter's list of tests and the conditions

57. Where two-sample comparisons show up

Real world

Outside the textbook, this test is everywhere that two versions of something are compared on a numerical outcome. The aspirin studies in the chapter introduction compare heart attack rates; a website might compare the time visitors spend on two page designs.

Discussion prompt

A company shows 400 randomly chosen visitors page design 1 and 400 others design 2. Visitors on design 2 stay a mean of 0.4 minutes longer, and the test gives p = 0.002. Write two sentences for the manager: one on whether the difference is real, one on what else should be reported before switching designs.

The example has the shape of the trap slide in Part 5. Large samples make the test sensitive, so significance is easy to reach; the effect size is what says whether the result is worth acting on.

Random assignment is what lets the manager claim the design caused the change. Had visitors chosen which design to see, the two groups might differ in other ways, and the first condition on the last slide but one would fail.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10 introduction, p. 511 — aspirin studies and other two-group comparisons

58. Putting it together

Section

Part 7

59. One sample against two

Comparison

Section 9 and this section side by side. Fill in the blanks; every entry on the right is the two-sample version of the entry on its left.

Comparison matrix

one sample (Chapter 9)two independent samples (§10.1)
parameterμmu1 - mu2
estimatex̄x̄1 − x̄2
null hypothesisμ = μ0mu1 - mu2 = 0
standard errors/√n√(s1²/n1 + s2²/n2)
degrees of freedomn − 1Aspin-Welch
test statistic(x̄ − μ0) ÷ SE(x̄1 − x̄2) ÷ SE

Read the table down the middle column first: everything in it should already be familiar. Then down the right: the parameter and estimate become differences, the standard error adds two variance terms, and the degrees of freedom need the Aspin-Welch formula.

What does not appear in the table is as important. The meaning of the p-value, the decision rule, the two types of error and the wording of the conclusion are all unchanged. Learning Chapter 9 well makes this section mostly a matter of two new formulas.

60. Match each quantity to its formula

Matching

Five quantities appeared in this section, and two of them look alike: the standard error used in the test and the pooled standard deviation used only for Cohen's d. Match each to its formula.

Match the pairs

  • a. the standard error of x̄1 − x̄2
  • b. the test statistic
  • c. the degrees of freedom
  • d. the pooled standard deviation
  • e. Cohen's d
  • v. √(s1²/n1 + s2²/n2)
  • w. (x̄1 − x̄2) ÷ SE
  • x. (A + B)² ÷ [A²/(n1 − 1) + B²/(n2 − 1)]
  • y. √{[(n1 − 1)s1² + (n2 − 1)s2²] ÷ (n1 + n2 − 2)}
  • z. (x̄1 − x̄2) ÷ sp

Why: The test uses the unpooled standard error and the Aspin-Welch degrees of freedom. The pooled standard deviation appears only inside Cohen's d, which describes the size of the gap rather than testing it.

Two of the formulas divide the gap by something, and the difference between them is the whole difference between testing and describing. Divided by the standard error, the gap becomes t, which grows with the sample sizes. Divided by the pooled standard deviation, it becomes d, which does not.

If you remember only one distinction from the section, make it that one. It explains the trap in Part 5 and the advice in the real-world slide: big samples make t large, but they leave d where it was.

61. Pattern: any two-sample test of means, in order

Pattern

Every problem in this section ran through the same sequence. The arithmetic is short; the decisions that come before it are where marks are won and lost.

Figure (svg): A flow diagram in two rows. Top row: two independent groups, means, sigmas unknown; then hypotheses about mu 1 minus mu 2 with the order of subtraction fixed; then the standard error, the square root of s1 squared over n1 plus s2 squared over n2. Bottom row: t equals the gap over the standard error; Aspin-Welch degrees of freedom, not pooled; the p-value in the tail the alternative names; then a decision and a conclusion in context, with Cohen's d for size.

The first decision, which group is group 1, controls the sign of everything after it. Make it once, write it down, and keep it.
  1. Design: independent groups, means, population σ's unknown? Otherwise a different section.
  2. Label: write group 1 and group 2. Keep that order through every line.
  3. Hypotheses: about μ1 − μ2; the claim's wording sets the tail.
  4. Standard error: add the two variance terms; check it lies between the larger group SE and their sum.
  5. t and df: gap over SE; Aspin-Welch df, between min(n) − 1 and n1 + n2 − 2. Do not pool.
  6. p-value and decision, then a conclusion naming the level, the direction and the populations.
  7. Size: Cohen's d with the pooled SD, reported beside the test, never instead of it.

\[ t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}}, \qquad d = \frac{\bar{x}_1 - \bar{x}_2}{s_p} \]

The diagram reads left to right and then down. Its first box is the one most often skipped: before anything else, confirm the design is two independent groups compared on a mean. A paired design or a proportion sends you to a later section, however familiar the numbers look.

Most wrong answers here are right calculations of the wrong thing: the order of subtraction flipped against the alternative, standard errors added instead of variances, the pooled df used because it is easier, or a large p-value read as proof of equal means. Each has a one-line check, and the list above puts each check where it belongs.

62. Check: the standard error and t

Check

Compute the two variance terms before you look at the options.

Write A and B separately, add them, and only then take one square root. Then divide the gap, in the order the groups are named.

Check your understanding

Group 1: n = 16, mean 52, s = 8. Group 2: n = 25, mean 47, s = 10. What is the two-sample t statistic for group 1 minus group 2?

  • A. t = 5 ÷ √(4 + 4) ≈ 1.77 (correct)
  • B. t = 5 ÷ (2 + 2) = 1.25
  • C. t = 5 ÷ √(64 + 100) ≈ 0.39
  • D. t = 5 ÷ √(4 − 4), undefined

Answer: A

Why: The variance terms are 64 over 16, which is 4, and 100 over 25, which is also 4. Their sum is 8, the standard error is the square root of 8, about 2.83, and the gap of 5 divided by 2.83 is about 1.77.

Why B tempts people
That adds the two standard errors, 2 and 2, instead of adding the variances under one root. It overstates the standard error.
Why C tempts people
That forgets to divide each variance by its own sample size, so the standard error is the spread of individuals, not of means.
Why D tempts people
Variance terms are never subtracted, even though the estimate is a difference. Independent variability adds.

In context: group 1's sample mean is about 1.8 standard errors above group 2's. That is borderline evidence for a one-tailed test at 5 percent and not enough for a two-tailed one, so the tail named by the alternative decides the outcome.

The bound check catches two of the wrong options at once. The standard error must lie between 2, the larger group standard error, and 4, their sum. The square root of 8 does; 4 sits on the upper bound, and the subtracted version gives zero.

63. Check: the tail and the conclusion

Check

Fix group 1 before you choose.

Ask which direction the claim points, with the groups in the order the question gives them, then which conclusion the p-value supports.

Check your understanding

A dietitian claims a new diet's group (group 1) loses more weight than a standard diet's group (group 2). The two-sample t test gives t = 2.31 on 21.4 df, p = 0.0155. At the 1 percent level, which conclusion is correct?

  • A. Not sufficient evidence at 1% that the new diet's mean loss is greater (correct)
  • B. Sufficient evidence at 1% that the new diet's mean loss is greater
  • C. The two diets' mean losses are equal
  • D. Sufficient evidence at 1% that the mean losses differ

Answer: A

Why: The claim is that group 1 loses more, so the test is right-tailed. The p-value of 0.0155 is larger than 0.01, so the null is not rejected at the 1 percent level, and the conclusion is that there is not sufficient evidence.

Why B tempts people
That would be right at the 5 percent level, but 0.0155 is larger than 0.01. The level belongs in the conclusion because it changes the decision.
Why C tempts people
Not rejecting never shows the means are equal; the data are only consistent with equality.
Why D tempts people
That changes the claim into a two-tailed one, which doubles the p-value, and still fails to reject at 1 percent.

In context: the diet study gives some evidence of a larger mean weight loss, enough at 5 percent, but not enough at the stricter 1 percent level the question set. A careful report would say both, with the level each time.

Notice how many separate decisions this one question needed: the tail from the wording, the comparison with the right alpha, and the refusal to turn not rejecting into equal. Each wrong option is one of those decisions made wrongly.

64. Check: pooling and degrees of freedom

Check

Use the bracket for the degrees of freedom.

Before any formula, work out the smallest and largest values the Aspin-Welch degrees of freedom can take for these two sample sizes.

Check your understanding

Two independent samples have sizes 8 and 30. A calculator running the test of this section reports df. Which value could it be?

  • A. df = 12.6 (correct)
  • B. df = 36
  • C. df = 6.2
  • D. df = 38

Answer: A

Why: The Aspin-Welch degrees of freedom lie between the smaller sample's size minus one, which is 7, and the total minus two, which is 36. Only 12.6 lies strictly inside that range and is not the pooled value.

Why B tempts people
Thirty-six is n1 plus n2 minus 2, the pooled test's degrees of freedom. The calculator was set to pool, which the book says not to do.
Why C tempts people
That is below 7, the smaller sample's own degrees of freedom, which the Aspin-Welch value can never fall below.
Why D tempts people
That is the total sample size, which is never a degrees-of-freedom value for this test.

A value as low as 12.6, when 36 was available, tells you something about the data: the group of 8 has the larger variance term, so the comparison rests mostly on those eight values.

The bracket is the quickest sanity check on calculator output in this whole section. Any df outside it means a mistyped input or the wrong test.

Each wrong option is a real calculator mistake: the pooled value from leaving Pooled on Yes, a value below the floor from typing a standard deviation into the sample-size box, and the total sample size from confusing n with degrees of freedom.

65. Explain why the variances add

Explain it to yourself

The single most surprising formula in this section is the standard error of a difference, with its plus sign. Explaining it in your own words is the best test of whether it has landed.

Discussion prompt

The estimate is x-bar 1 minus x-bar 2, yet its standard error adds the two variance terms. In two or three sentences, explain why subtracting the means does not subtract their variability.

A strong answer names independence and uses the word variance, not standard deviation. A very strong one adds the consequence: the standard error of the difference is larger than either group's own, and the noisier or smaller group contributes more of it.

If you found yourself wanting a minus sign, test that instinct on the simulation in Part 2: the differences there were visibly more spread out than either sample mean, not less.

66. Exit ticket: two battery brands

Exit ticket

Made-up data. Twelve brand 1 batteries last a mean of 32.0 hours, s 4.2; fifteen brand 2 batteries last a mean of 28.1 hours, s 5.9. Both lifetimes are normally distributed. Does brand 1 last longer, on average? Test at 5 percent, then give Cohen's d.

Discussion prompt

Write the hypotheses, compute the standard error, t, the Aspin-Welch df and the p-value, and state the conclusion and the effect size in context.

Check your degrees of freedom against the bracket: they must lie between 11 and 25, and 24.73 does. Check the statistic against the critical value too: 2.003 is just past the 5 percent cutoff of 1.709, which agrees with a p-value just under 0.05.

If your p-value came out near 0.056, you doubled the tail: the claim is directional, so only the right tail counts. If your standard error came out near 2.7, you added the standard errors instead of the variances.

67. Recap

Recap

Comparing two independent groups on a mean is Chapter 9's test with a new parameter, the difference μ1 minus μ2, and a new standard error. When the population standard deviations are unknown, the statistic follows Student's t distribution with Aspin-Welch degrees of freedom.

ideaformularead it as
hypothesesH0: μ1 − μ2 = 0fix group 1 first; the claim sets the tail
standard error√(s1²/n1 + s2²/n2)variances add; between the larger SE and the sum
statistict = (x̄1 − x̄2) ÷ SEthe gap in standard errors
degrees of freedomAspin-Welchrarely whole; between min(n) − 1 and n1 + n2 − 2
poolingdo not poolpooling assumes σ1 = σ2
conditionsindependent SRSssmall samples need normal populations
normal approximationn1 + n2 > 30allowed, but use t whenever possible
effect sized = (x̄1 − x̄2) ÷ sp0.2 small, 0.5 medium, 0.8 large

The work is mostly in the decisions: independent or paired, which group is group 1, which tail, and what the conclusion is allowed to say. Then check the standard error and the degrees of freedom against their brackets, and report the effect size beside the test result, never in place of it.

The Introductory Business Statistics text treats the same test in its §10.1, with business contexts, if you want more practice. Section 10.2 next takes up the rarer case where the population standard deviations are known, and the t becomes a z.

OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations §10.1, pp. 511-520 — the whole section

OpenStax Introductory Business Statistics 2e, §10.1 Comparing Two Independent Population Means §10.1 Comparing Two Independent Population Means — the same test in business contexts

Sources

  1. OpenStax Introductory Statistics 2e, §10.1 Two Population Means with Unknown Standard Deviations — Illowsky & Dean, OpenStax / Rice University, CC BY 4.0, pp. 511-520
  2. OpenStax Introductory Business Statistics 2e, §10.1 Comparing Two Independent Population Means — Illowsky & Dean, OpenStax / Rice University, CC BY 4.0
  3. OpenStax Introductory Business Statistics 2e, §10.3 Test for Differences in Means: Assuming Equal Population Variances — Illowsky & Dean, OpenStax / Rice University, CC BY 4.0

Want this taught 1-on-1? Alexander tutors Statistics — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108