SAT Math Type 19: Evaluating Statistical Claims

The rarest Math question type (0.7% of the bank) and the highest return per hour of study, because it reduces to a single distinction. Covers random assignment as the licence for causal claims and random selection as the licence for generalisation, how to tell an observational study from an experiment, why confounding variables defeat observational data, and how to match a conclusion to the study design that supports it — with six worked examples, four traps and three checks.

Subject: SAT Prep · 61 slides · applied lesson

Open the interactive version of this deck

What this lesson covers

The lesson, slide by slide

1. Evaluating Statistical Claims

Title

SAT Math · Type 19 of 19

0.7% of the question bank — 11 of 1675 questions

2. By the end of this deck you can

Objectives

A study is described in three or four sentences, and you are asked which conclusion it supports. There is no arithmetic at all. The entire type rests on one distinction, and once you hold it the questions take about fifteen seconds each — which is why the rarest type on the section is also the one most worth an hour of your time.

  1. Tell an observational study from an experiment by how subjects reached their groups.
  2. State that random assignment licenses causal claims, and explain why.
  3. State that random selection licenses generalisation, and to which population.
  4. Identify a confounding variable that could explain an observed association.
  5. Match a proposed conclusion to the study design that would support it.

Two sentences are the whole deck: random assignment licenses cause; random selection licenses generalisation. Neither one gives you the other.

Naruhodo Tutoring — SAT Math question bank export (sat-question-index.json) — 11 tagged questions of this type in the site's bank

3. Recognise It

Section

Section 1

4. What this question actually asks

Concept

A study is described in three or four sentences, and you are asked which conclusion it supports. There is no arithmetic at all. The entire type rests on one distinction, and once you hold it the questions take about fifteen seconds each — which is why the rarest type on the section is also the one most worth an hour of your time.

You will see it phrased in these ways:

Every one of these is answered by locating two words in the stem: whether subjects were randomly assigned, and whether they were randomly selected.

5. The card for this type

Picture it

The same three-panel card as the survey deck, so the shorthand carries over: what identifies it, what you write first, and what is built to catch you.

Figure (svg): Evaluating statistical claims: the tell, the move, and the trap

Evaluating statistical claims — 11 of 1675 bank questions (0.7%)

The green panel is the complete method. There is nothing else to learn on this type, which is why eleven questions in a bank of 1,675 still justify a deck.

6. Observational study or experiment?

Prediction

The distinction is about how subjects reached their groups.

Predict first

Which description is an experiment?

  • Researchers randomly assigned 200 volunteers to receive either the drug or a placebo
  • Researchers surveyed 200 people about whether they take the drug
  • Researchers compared people who already take the drug with people who do not
  • Researchers observed 200 patients over a year and recorded who improved

Correct: Researchers randomly assigned 200 volunteers to receive either the drug or a placebo

Why: An experiment is defined by the researcher ASSIGNING subjects to groups. In the other three, subjects arrived in their groups by their own choices or circumstances, and the researcher merely observed — which makes all three observational studies, however carefully conducted. The word assigned is the tell, and it is the single most important word in this type.

7. The faces this type wears

Concept

Three shapes, all decided by the same two questions.

variantwhat it wantsthe two questions
Which conclusion?the strongest supported claimassignment for cause, selection for scope
Compare two designswhich supports morethe one with assignment supports cause
Fix the studywhat would license the claimadd whichever form of randomness is missing

The third variant is the most instructive, because it forces you to say which kind of randomness was absent — which is exactly what the first two variants are testing implicitly.

8. Which randomness is which?

Definition probe

Two different words doing two different jobs.

Sort into buckets

What does each form of randomness license?

Licenses a causal claim
Subjects were randomly assigned to the treatment or control group
Licenses generalisation to a population
Subjects were randomly selected from all residents of the city
Licenses neither
Volunteers chose which group to join; Researchers surveyed whoever was in the shopping centre that day
cause
Random assignment balances every other factor between the groups, so a difference in outcome can be attributed to the treatment. It says nothing about who the subjects represent.
general
Random selection makes the sample representative of the population it was drawn from, so results extend to that population. It says nothing about cause.
neither
Self-selection and convenience sampling provide neither. Subjects who chose their group differ systematically, and shoppers at one centre on one day represent nobody in particular.

9. Which conclusion does each design support?

Discrimination

Six studies, three kinds of conclusion.

Sort into buckets

What can this study support?

Cause, within the group studied
Random assignment, volunteers recruited by advert; Random assignment among patients at one clinic
Generalisation, but only association
Random selection from a city, no assignment
Both cause and generalisation
Random selection AND random assignment
Neither
Neither: people who already exercise compared with those who do not; A voluntary online survey of 100,000 people
causeonly
Random assignment supports a causal claim about the subjects studied. But volunteers or one clinic's patients were not randomly selected from any wider population, so the finding may not extend beyond them.
genonly
Random selection makes the sample representative, so the finding generalises — but without assignment it establishes only an association.
both
Only a study with both forms of randomness supports a causal claim about a wider population. This is the gold standard and it is rare.
neither
Comparing self-selected groups gives neither, and a voluntary survey gives neither regardless of size.

10. Name the missing randomness

Warm-up

Try it before the rules.

Discussion prompt

Researchers randomly assigned 300 volunteers to a new exercise programme or a control group, and found the programme group had lower blood pressure. Can they conclude the programme lowers blood pressure? Can they conclude it would work for the general public?

Hint: There are two questions, and the answers differ.

Answer:

Yes to the first. Subjects were randomly ASSIGNED, which balances every other factor between the groups, so the difference in outcome can be attributed to the programme.

No to the second. The subjects were volunteers, not randomly SELECTED from the public. Volunteers differ — they tend to be more motivated and often healthier.

So the correct conclusion: the programme caused lower blood pressure among these volunteers.

What would license the wider claim: random selection from the general public, in addition to the random assignment already present.

That pair of answers — yes to cause, no to generalisation — is the most common configuration on this type.

11. The routine, every time

Pattern

Two questions and a match. There is no third step and no arithmetic.

Ask: were subjects randomly ASSIGNED to groups by the researcher?

Why: If yes, it is an experiment and a causal claim is available. If no, it is observational and only association is available, however carefully it was run.

Ask: were subjects randomly SELECTED from a wider population?

Why: If yes, the finding extends to that population. If no, it applies only to the subjects studied.

Match the conclusion to what those two answers permit.

Why: Reject any choice claiming cause without assignment, or claiming a wider population without selection.

Both questions are answered by single words in the stem. Underlining assigned and selected is genuinely the whole technique.

12. The Rules

Section

Section 2

13. Rule 1 · An experiment assigns; an observational study watches

Concept

The defining question is whether the researcher decided which group each subject went into.

experimentobservational study
How groups formthe researcher assignssubjects arrive by choice or circumstance
Typical wordingrandomly assigned, allocated tosurveyed, compared, observed, recorded
Supportsa causal claiman association only
Main weaknesssubjects may not represent anyoneconfounding variables

The distinction is entirely about the mechanism of group formation, never about the size or the care of the study.

14. Rule 2 · Random assignment licenses a causal claim

Concept

Assigning subjects randomly balances every other factor, known and unknown, between the groups.

This is why the phrase randomly assigned is worth circling the moment you read it. It is the only thing that licenses the word cause.

15. Which word matters?

Prediction

One word decides whether cause is available.

Predict first

Which phrase in a study description licenses a causal conclusion?

  • participants were randomly assigned to two groups
  • participants were randomly selected from the population
  • participants were surveyed carefully
  • the sample size was very large

Correct: participants were randomly assigned to two groups

Why: Random assignment is what balances other factors between groups and therefore licenses a causal claim. Random selection licenses generalisation instead, which is a different thing. Careful surveying and a large sample both improve the study without changing what kind of claim it can support.

16. Rule 3 · Random selection licenses generalisation

Concept

Selecting subjects randomly from a population makes the sample representative of that population.

The two randomnesses answer different questions, and a study can have either, both, or neither.

17. Assignment without selection

Prediction

One randomness, one licence.

Predict first

A study randomly assigns 200 volunteers to two groups and finds a real difference. What can be concluded?

  • The treatment caused the difference among these volunteers
  • The treatment causes the difference in the general population
  • There is only an association among these volunteers
  • Nothing at all can be concluded

Correct: The treatment caused the difference among these volunteers

Why: Random assignment licenses a causal claim about the subjects studied. But the subjects were volunteers rather than a random sample of any wider group, so the finding cannot be extended to the general population. The third choice understates: assignment does license cause, not merely association.

18. Rule 4 · A confounding variable explains an association without causing it

Concept

A third factor affecting both quantities can produce an association with no causal link between them.

When a question asks why a causal conclusion is unjustified, naming a plausible confounder is usually the expected answer.

19. Rule 5 · Self-selection is the commonest flaw

Concept

When subjects choose their own group, the groups differ before the study begins.

Self-selection defeats both randomnesses at once: the groups are not comparable, and the respondents represent nobody in particular.

20. Selection without assignment

Prediction

The other single randomness.

Predict first

A study randomly selects 1,000 residents of a city and finds that those who cycle have better health. What can be concluded?

  • Cycling is associated with better health among the city's residents
  • Cycling causes better health among the city's residents
  • Cycling causes better health in general
  • Nothing at all can be concluded

Correct: Cycling is associated with better health among the city's residents

Why: Random selection licenses generalisation to the city's residents, so the association extends to that population. But nobody was assigned to cycle — residents chose for themselves — so healthier people may simply be more likely to cycle, or a third factor such as income may drive both. Only association is available.

21. Rule 6 · A causal claim needs assignment, not size

Concept

No amount of data converts an observational study into evidence of cause.

This mirrors the bias point in type 18: size buys precision and never buys validity.

22. Rule 7 · The four combinations

Concept

A study has assignment or not, and selection or not, giving four cases with four different scopes.

assignment?selection?what it supports
yesyescause, in the wider population
yesnocause, among the subjects studied
noyesassociation, in the wider population
nonoassociation, among the subjects studied

Memorise this table and the type is finished. Every question on it is asking which of these four rows the described study occupies.

23. Name the confounder

Prediction

What could explain the association without causing it?

Predict first

Towns with more parks have lower obesity rates. Which is the most plausible confounding variable?

  • Town wealth, which affects both park provision and diet
  • The number of parks
  • The obesity rate
  • The number of towns studied

Correct: Town wealth, which affects both park provision and diet

Why: A confounder is a third factor influencing both of the quantities being compared. Wealthier towns can afford more parks and their residents tend to have better access to healthy food and healthcare — so wealth could produce the association with no causal link between parks and obesity at all. The other choices name the two variables themselves and a feature of the study rather than a third factor.

24. Three of these are true

Two truths and a lie

Three of these statements about study design are correct. The one left standing is false.

Eliminate the wrong options

Which statement is FALSE?

  • a. Random assignment allows a causal conclusion
  • b. Random selection allows generalisation to the population sampled
  • c. A large enough observational study establishes causation
  • d. A confounding variable can produce an association with no causal link

Survives elimination: c

Why: No sample size converts an observational study into evidence of cause. A larger sample narrows the margin of error, which is about precision, and does nothing about confounding. A study of a million self-selected exercisers still cannot separate the exercise from the income, diet and health that travel with it. Only random assignment can do that, because only assignment breaks the link between the treatment and everything else about the subject.

25. Check 1 · Which conclusion?

Check

Two questions, then match.

Check your understanding

Researchers randomly assigned 400 students at one university to either a new study technique or their usual method. The new-technique group scored higher. Which conclusion is best supported?

  • A. The technique improved scores among students at that university (correct)
  • B. The technique improves scores for all university students
  • C. There is an association between the technique and higher scores
  • D. Nothing can be concluded from this study

Answer: A

Why: Random assignment licenses a causal claim, so improved is justified. But the subjects were students at one university, not a random sample of university students generally, so the conclusion cannot extend beyond that institution.

Why B tempts people
This generalises beyond the sampled population. Random assignment happened within one university and licenses nothing about others.
Why C tempts people
This understates the finding. Random assignment supports a causal claim, so settling for association discards what the design actually established.
Why D tempts people
The study is well designed and supports a clear conclusion; rejecting everything is the over-correction.

Choices B and C are the two directions of error — too broad and too weak — and choice D is the over-correction. Locating the right row of the four-combination table avoids all three.

26. Worked Examples

Section

Section 3

27. Example 1 · An observational study

Worked example

Researchers surveyed 2,000 randomly selected adults in a region and found that those who eat breakfast weigh less on average. What can be concluded?

Figure (svg): A table applying the two questions to an observational study

Selection without assignment: generalisable association.

Ask about assignment: nobody was assigned to eat breakfast. Subjects chose for themselves.

Why: This makes it an observational study, so only association is available.

Ask about selection: 2,000 adults were randomly selected from the region.

Why: This licenses generalisation to the adults of that region.

Combine: there is an association between eating breakfast and lower weight among adults in that region.

Why: Selection gives the scope; the absence of assignment limits the claim to association.

This is the most common configuration on the test: a well-conducted survey supporting a generalisable association and nothing stronger.

Verify: name a confounder: people who eat breakfast may also have more regular schedules and more active jobs.

Why: A plausible third factor confirms that the causal claim is not available.

Answer: an association among adults in that region, not a causal claim

28. Example 2 · An experiment on volunteers

Worked example

150 volunteers were randomly assigned to a tutoring programme or a control group. The tutored group scored higher. What can be concluded?

Figure (svg): A table applying the two questions to an experiment on volunteers

Assignment without selection: cause, but narrow scope.

Ask about assignment: yes, the researchers randomly assigned the groups.

Why: This balances other factors, so the tutoring is the remaining difference between the groups.

Ask about selection: no, the subjects volunteered.

Why: Volunteers differ from non-volunteers in motivation, so they represent no wider population.

Combine: the tutoring caused higher scores among these volunteers.

Why: Cause is available; the wider generalisation is not.

This configuration is extremely common in real medical research, where volunteers are recruited and then randomised. The causal finding is solid; the generalisation is the contested part.

Verify: check what is missing for the wider claim: random selection from all students.

Why: Naming what is absent confirms exactly which half of the conclusion is unavailable.

Answer: the tutoring caused higher scores, among these volunteers

29. Example 3 · Neither randomness

Worked example

A company compared employees who chose to use its wellness app with those who did not, and found app users took fewer sick days. What can be concluded?

Figure (svg): Bars showing only the narrow association claim is supported

Neither randomness: the weakest available conclusion.

Assignment: no. Employees chose whether to use the app.

Why: Self-selection means the two groups differed before the study began.

Selection: no. These are the employees of one company, not a random sample of anyone.

Why: The findings apply to this workforce and no wider group.

So only an association among these employees is supported.

Why: Both randomnesses are absent, so both kinds of claim are unavailable.

This is the weakest of the four configurations and it is very common in workplace and marketing studies, which routinely report it as though it were causal.

Verify: name the confounder: health-conscious employees are more likely both to use the app and to be well anyway.

Why: A single plausible confounder is enough to defeat the causal reading.

Answer: an association among this company's employees, nothing more

30. What do you look for first?

Step zero

Before reading any answer choice.

Discussion prompt

A question describes a study and asks which conclusion is appropriate. What two words do you search the stem for?

Hint: Two phrases, doing two different jobs.

Answer:

Randomly ASSIGNED — if present, a causal claim is available.

Randomly SELECTED — if present, the finding generalises to the population named.

Underline whichever appears, and note which is absent. The absent one tells you what the correct answer will NOT claim.

If neither appears, the study supports only an association among the subjects actually studied — the weakest of the four conclusions.

Only then read the choices, and eliminate any that claim more than the two words permit. This usually leaves one standing.

31. Example 4 · The gold standard

Worked example

Researchers randomly selected 800 adults from a national register and then randomly assigned each to one of two diets. One diet produced greater weight loss. What can be concluded?

Figure (svg): A table showing both randomnesses present

Both randomnesses: the strongest available conclusion.

Assignment: yes, subjects were randomly assigned to the diets.

Why: Other factors are balanced between the groups, licensing a causal claim.

Selection: yes, subjects were randomly selected from a national register.

Why: The sample represents the national adult population.

So the diet caused greater weight loss, and the finding generalises nationally.

Why: Both randomnesses are present, so both claims are available.

This configuration is rare in practice because it is expensive, and it is rare on the SAT too. When it appears, the strongest conclusion offered is usually correct.

Verify: check that both words appear in the stem: randomly selected and randomly assigned.

Why: Both are explicitly stated, which is what distinguishes this from the previous examples.

Answer: the diet caused greater weight loss in the national adult population

32. Example 5 · Naming a confounder

Worked example

A study finds that students who take music lessons have higher mathematics scores. Why is the conclusion that music lessons improve mathematics unjustified?

Figure (svg): Two dot plots showing music students scoring higher on average

The difference is real; the explanation is not established.

Identify the design: nobody was assigned to take music lessons — families chose.

Why: This is observational, so a causal claim is unavailable from the outset.

Name a confounder: family income affects both the ability to pay for lessons and access to tutoring and resources.

Why: A third factor influencing both quantities produces the association without any causal link.

So the association is real and the causal explanation is not established.

Why: The data cannot distinguish music lessons from everything that travels with them.

Naming a specific plausible confounder is usually what these questions want, rather than the general statement that correlation is not causation.

Verify: check what would settle it: randomly assigning students to receive lessons or not.

Why: Assignment would break the link between lessons and family circumstances, isolating the effect.

Answer: because the study is observational, and confounders such as family income could explain the association

33. Complete the two rules

Faded example

From memory. These two sentences are the entire type.

Fill in the blanks

Random assignment licenses a claim about cause, because it balances other factors between the groups. Random selection licenses generalisation to the population sampled. A study is an experiment when the researcher assigns subjects to groups. And a confounding variable can explain an association without any causal link.

Why: The first two blanks are the whole deck. They are easy to confuse because both contain the word random and both sound like good practice — but they license completely different claims, and a study can have either without the other.

34. Example 6 · Fixing the study

Worked example

A researcher wants to conclude that a new fertiliser increases crop yield nationally. She currently compares farms that already use it with farms that do not. What must change?

Figure (svg): A line showing the two additions needed to reach the desired conclusion

Two claims require two separate changes.

The desired conclusion has two parts: a causal claim, and a national scope.

Why: Each part requires its own form of randomness.

For the causal part, she must randomly ASSIGN farms to use the fertiliser or not.

Why: Currently farms chose for themselves, so the two groups may differ in soil, climate and management.

For the national part, she must randomly SELECT the farms from a national list.

Why: Currently the farms are whichever ones she happened to compare.

The fix-the-study variant is the clearest test of whether you hold the distinction, because it requires naming which randomness serves which purpose.

Verify: check both changes are needed: assignment alone would give cause among those farms only.

Why: Neither change substitutes for the other, which is the central point of the type.

Answer: she must randomly assign farms to the fertiliser AND randomly select them nationally

35. Fill in the four combinations

Fill the middle

Two questions, four answers.

Fill in the blanks

Assignment yes and selection yes supports cause in the wider population. Assignment yes and selection no supports cause among the subjects studied. Assignment no and selection yes supports association in the wider population. And neither supports association among the subjects studied.

Why: This four-row table is the complete content of the type. Every question is asking which row the described study occupies, and the correct answer is the strongest conclusion that row permits — no stronger, and no weaker.

36. Judge the strength of a claim

Estimation

Rank the conclusions before looking at the study.

Predict first

Which conclusion is the STRONGEST claim, and therefore needs the most from a study design?

  • The treatment causes the effect in the general population
  • The treatment is associated with the effect in the study group
  • The treatment causes the effect in the study group
  • The treatment is associated with the effect in the general population

Correct: The treatment causes the effect in the general population

Why: Causation is stronger than association, and a claim about a general population is stronger than one about the study group. So the strongest claim needs both randomnesses — assignment for the causal part and selection for the population part. Ranking the choices by strength before examining the study is a useful habit, because the correct answer is the strongest claim the design actually supports.

37. Check 2 · Observational data

Check

Look for who chose.

Check your understanding

A survey of 1,500 randomly selected town residents found that those who own dogs walk more each day. Which conclusion is best supported?

  • A. Dog ownership is associated with more walking among the town's residents (correct)
  • B. Owning a dog causes people to walk more
  • C. Walking more causes people to acquire dogs
  • D. Dog ownership is associated with more walking among all adults

Answer: A

Why: Random selection from the town licenses generalisation to that town's residents. Nobody was assigned a dog, so only an association is available, and the scope stops at the town.

Why B tempts people
A causal claim requires random assignment, which is absent. People who like walking may be more likely to get a dog.
Why C tempts people
This is the reverse causal claim, equally unsupported by observational data.
Why D tempts people
This extends beyond the town, which is the only population the sample represents.

The two causal choices point in opposite directions, which is itself the argument: observational data cannot distinguish between them, so neither is available.

38. The Traps

Section

Section 4

39. Reading cause into an observational study

Trap

The trap

The trap. A careful, large, well-funded study finds that people who take a supplement live longer, and a choice concludes the supplement extends life.

Nobody was assigned to take it. People who choose supplements also tend to exercise more, eat better and visit doctors more often.

The study's quality and size are irrelevant to this: neither can separate the supplement from everything that travels with it.

The fix

The fix. Search the stem for randomly assigned before considering any causal choice.

If subjects arrived in their groups by their own choices, only association is available.

  1. Underline how subjects reached their groups.
  2. If they chose, reject every causal choice regardless of how the study is described.
  3. Prefer the choice using associated with, linked to, or tends to.

40. Confusing the two randomnesses

Trap

The trap

The trap. A study randomly SELECTS participants and a choice concludes the treatment caused the outcome.

Random selection makes the sample representative; it does nothing about which subjects received the treatment.

The word random appears in the stem, which makes the causal choice feel licensed when it is not.

The fix

The fix. Read the word after random: selected or assigned.

Selected licenses generalisation. Assigned licenses cause. They are not interchangeable.

  1. Circle the word immediately following randomly.
  2. Match it to what it licenses: assigned for cause, selected for scope.
  3. Note explicitly which one is missing, and reject choices requiring it.

41. Annotate an over-claim

Error analysis

A researcher's conclusion. The observation is sound and the inference is not.

Annotate

On: \( \text{observational: coffee drinkers report less fatigue} \;\Rightarrow\; \text{coffee reduces fatigue} \)

  • The observation is legitimate. If the sample was properly drawn, coffee drinkers really do report less fatigue, and that association is a genuine finding.
  • The inference to a cause is where it fails. Nobody was assigned to drink coffee — people chose for themselves.
  • So the two groups differed before the study. Consider what else distinguishes them: sleep patterns, working hours, age, and general health.
  • The direction is also unidentified. People who are less fatigued may simply be more inclined to drink coffee socially, rather than the reverse.
  • And a confounder could produce the pattern with neither causing the other: people with regular routines may both sleep better and drink coffee at fixed times.
  • What would license the claim: randomly assigning participants to drink coffee or not, which breaks the link between coffee and everything else about the person.

Note that the causal claim may well be true. The question is never whether a conclusion is plausible, but whether this study establishes it — and those are different questions.

42. Generalising an experiment on volunteers

Trap

The trap

The trap. A randomised experiment on 200 volunteers shows a real effect, and a choice concludes the treatment works for everyone.

Random assignment licenses the causal claim about those volunteers. It says nothing about who they represent.

Volunteers differ systematically from non-volunteers, often being healthier and more motivated.

The fix

The fix. Treat the two claims separately: cause is one question, scope is another.

Assignment answers the first; only selection answers the second.

  1. Split the conclusion into its causal part and its population part.
  2. Check each against the corresponding randomness.
  3. Choose the answer that keeps the causal claim and narrows the population.

43. Eliminate three by design

Elimination

Researchers randomly selected 500 adults from a city and found those who volunteer report higher life satisfaction.

Eliminate the wrong options

Which conclusion is supported? Three can be eliminated from the design alone.

  • a. Volunteering is associated with higher life satisfaction among the city's adults
  • b. Volunteering causes higher life satisfaction
  • c. Volunteering is associated with higher life satisfaction among all adults nationally
  • d. Higher life satisfaction causes people to volunteer

Survives elimination: a

Why: Random selection from the city licenses generalisation to the city's adults, and nothing wider. The absence of random assignment means only an association is available, in either direction. Notice how mechanical the eliminations are: two choices fail on the missing assignment and one on the scope of the selection, and none of it required reading the numbers.

44. Over-correcting to no conclusion at all

Trap

The trap

The trap. Having learned that correlation is not causation, you reject every conclusion and choose nothing can be determined.

An observational study genuinely does support an association, and a randomised experiment genuinely does support a causal claim about its subjects.

Rejecting everything is as wrong as accepting too much, and the SAT offers a nothing-can-be-concluded choice precisely to catch this.

The fix

The fix. Identify the strongest claim the design DOES support, rather than the strongest it does not.

Every one of the four combinations supports something.

  1. Work out which of the four rows the study occupies.
  2. State what that row permits, in full.
  3. Choose the answer matching it exactly — neither stronger nor weaker.

45. Find the counterexample

Counterexample

A claim about what large studies can show.

Discussion prompt

A student says: with enough data, an observational study proves causation. Give a concrete counterexample.

Hint: Think of an association everyone accepts is not causal.

Answer:

Counterexample: ice cream sales and drowning deaths. Across millions of records they rise and fall together with near-perfect consistency.

No amount of additional data would establish that ice cream causes drowning. The association is real and the explanation is elsewhere.

The confounder is hot weather, which independently increases both ice cream sales and swimming.

Why more data does not help: every additional record contains the same confounding. The sample gets more precise about an association that was never causal.

What would settle it: randomly assigning people to eat ice cream or not, and comparing drowning rates. Absurd here, and that absurdity is the point — the causal question requires an intervention the observational data never performed.

The general principle: size addresses precision, and only assignment addresses confounding.

46. Push the causation rule to its edge

Edge cases

Observational studies cannot establish cause. Is that the end of the matter?

Discussion prompt

Are observational studies useless for causal questions, and when is random assignment impossible?

Hint: Consider whether anyone could ethically run the experiment.

Answer:

They are not useless. Observational studies generate the hypotheses that experiments later test, and they are often the only evidence available.

Random assignment is frequently impossible or unethical. Nobody can randomly assign people to smoke for thirty years, or to grow up in poverty.

In such cases researchers strengthen observational evidence by controlling for known confounders, looking for dose-response relationships, and checking that the association holds across many different populations.

The smoking and lung cancer case is the classic example: the causal conclusion was eventually accepted on observational evidence, but it took decades, many independent studies, and a biological mechanism.

On the SAT, none of this applies. The test asks what a single described study supports, and the answer is governed strictly by the two randomnesses.

So the exam rule and the real-world picture differ, and it is worth knowing both — the exam rule is a simplification of genuine scientific caution, not an arbitrary convention.

47. Check 3 · Fixing the design

Check

Name which randomness is missing.

Check your understanding

A researcher compares patients who chose acupuncture with those who did not, and wants to conclude that acupuncture reduces pain for the general population. What must be added?

  • A. Random assignment to acupuncture, and random selection from the population (correct)
  • B. Random assignment to acupuncture only
  • C. Random selection from the population only
  • D. A larger sample of patients

Answer: A

Why: The desired conclusion has two parts. Reduces pain is causal and requires random assignment. For the general population is a scope claim and requires random selection. The current study has neither, so both must be added.

Why B tempts people
Assignment alone would license a causal claim about the patients studied, but not about the general population.
Why C tempts people
Selection alone would license generalisation of an association, but not a causal claim.
Why D tempts people
A larger sample improves precision and does nothing about confounding or scope.

This variant is the clearest test of the distinction, because it forces you to name which randomness serves which half of the conclusion.

48. Drill and Plan

Section

Section 5

49. Match each design to what it supports

Matching

Six designs, six conclusions.

Match the pairs

  • both. Random selection and random assignment
  • assign. Random assignment of volunteers only
  • select. Random selection, no assignment
  • neither. Comparing self-selected groups
  • voluntary. A voluntary online survey of 100,000
  • convenience. Surveying whoever was in one shopping centre
  • a. Cause, generalisable to the population
  • b. Cause, among the volunteers only
  • c. Association, generalisable to the population
  • d. Association, among those studied only
  • e. Association among respondents; no generalisation despite the size
  • f. Association among those shoppers; no generalisation

Why: The last two rows both have large or convenient samples and neither supports generalisation, because neither used random selection. That is the same point as the bias rule in type 18, and it is why the two types are worth studying together.

50. Sort six studies

Sorting

Each describes how subjects reached their groups.

Sort into buckets

Experiment or observational study?

An experiment
Participants were randomly allocated to two diets; Patients were assigned by coin flip to drug or placebo; Volunteers were randomly placed into a training group or a control group
An observational study
Researchers surveyed people about their existing diets; Students who already play sport were compared with those who do not; Records of 10,000 patients were analysed for patterns
exp
In each of these the RESEARCHER decided which group each subject entered — by allocation, by coin flip, or by random placement. That is what makes a study an experiment.
obs
In each of these subjects arrived in their groups by their own choices or by circumstance, and the researcher only recorded what happened. Item (f) is worth noting: analysing 10,000 records is large and careful and still observational.

The distinction is entirely about the mechanism of group formation. Size, care, cost and sophistication are all irrelevant to it.

51. The two randomnesses

Comparison

Fill the blanks from memory. This table is the whole type.

Comparison matrix

random assignmentrandom selection
What it decideswhich group each subject goes intowho is in the study at all
What it licensesa causal claimgeneralisation to the population sampled
What it does NOT giveany claim about a wider populationany causal claim
Absent whensubjects chose their own groupsubjects volunteered or were convenient

The third row is the one worth rehearsing. Each randomness gives exactly one thing and explicitly does not give the other, which is why a study can be rigorous in one dimension and useless in the other.

52. What each study feature buys

Trade off

Fill in what each does and does not achieve.

Comparison matrix

featurewhat it buyswhat it does not
Random assignmenta causal claim about the subjectsgeneralisation beyond them
Random selectiongeneralisation to the populationany causal claim
A larger samplea narrower margin of errorvalidity, cause, or freedom from bias
A careful, expensive studybetter data qualityany upgrade in what can be concluded

The bottom two rows are what students most often get wrong. Size and quality are real virtues that change nothing about which KIND of conclusion a design supports.

53. Where this shows up outside the test

Real world

One minute on why this distinction is worth more than 0.7 per cent.

Discussion prompt

Health headlines routinely report observational findings with causal language. How can a reader tell what a study actually showed?

Answer:

Look for the word assigned. A randomised controlled trial says participants were randomly assigned. An observational study says participants were followed, surveyed or compared.

The vocabulary of the report is a reliable tell. Observational findings are described as linked to or associated with; experimental findings say reduces or causes.

A famous reversal: observational studies found hormone replacement therapy was associated with lower heart disease. A later randomised trial found the opposite — the association came from healthier, wealthier women being more likely to receive it.

That case cost real lives, and it is the standard example of why the distinction is not academic pedantry.

The practical skill: when you read that something is linked to an outcome, ask who chose. If the participants chose, a confounder is always available as an explanation.

Which is exactly the SAT question, asked about a study described in three sentences.

54. Order these designs by the strength of conclusion they support

Ranking

Order from weakest to strongest.

Put in order

  1. Comparing self-selected groups at one workplace
  2. Random selection from a city, no assignment
  3. Random assignment among volunteers
  4. Random selection nationally and random assignment

Why: The weakest supports only an association among those studied. Adding random selection extends that association to a population. Random assignment instead gives a causal claim, which is a stronger kind of claim even though its scope is narrow. And both randomnesses together give a causal claim about a population, which is the strongest conclusion any study can support. Note the judgement in ranking (b) below (c): causation is treated as the stronger claim, since a narrow causal finding usually tells you more about mechanism than a broad association does.

55. How to practise this type

Concept

This type is 0.7 per cent of the section — about one question — and it is a single distinction, which makes it the best return per hour on the entire test.

sessionwhat you dowhy
1Learn the two sentences: assignment licenses cause, selection licenses generalisation.This is genuinely the whole type; twenty minutes is enough.
2Classify fifteen described studies as experiment or observational, naming the deciding word.Builds the reflex of searching for assigned.
3For ten studies, write which of the four combinations it occupies and what it supports.The four-row table is the answer to every question on this type.
4Ten conclusion questions, eliminating choices on scope and on causal language before reading further.Trains the mechanical elimination that makes these fifteen-second questions.

Naruhodo Tutoring — SAT Math question bank export (sat-question-index.json) — filter the bank to this skill tag — 11 questions, roughly a third of each difficulty

56. Explain the distinction from memory

Explain it to yourself

Close the deck.

Discussion prompt

Without looking, state what random assignment licenses and why, what random selection licenses and why, and what a study with neither can support.

Hint: Two sentences and a consequence.

Answer:

Random assignment licenses a causal claim, because it balances every other factor — known and unknown — between the groups, so the treatment is the only systematic difference left.

Random selection licenses generalisation to the population the sample was drawn from, because it makes the sample representative of that population.

Neither gives the other. An experiment on volunteers supports cause among volunteers; a random survey supports association in a population.

With neither, a study supports only an association among the subjects actually studied — the weakest of the four conclusions, and still a real one.

If you also said that size and care change nothing about which kind of claim is available, you have the version that resists every distractor on this type.

57. Teach the two randomnesses

Explain it

Two minutes, out loud.

Discussion prompt

A friend sees the word random in a study description and assumes any conclusion is fine. How do you separate the two meanings for them?

Answer:

Tell them there are two different randomnesses, and they do completely different jobs.

Random SELECTION decides who is in the study. It makes the participants representative, so findings extend to the population they came from. It says nothing about cause.

Random ASSIGNMENT decides which group each participant goes into. It makes the groups comparable, so a difference in outcome can be blamed on the treatment. It says nothing about who they represent.

Give the two examples side by side. A perfect national survey with no assignment: generalisable, but only an association. A perfect experiment on volunteers: causal, but only about those volunteers.

Then the practical instruction: read the word immediately after randomly, and note which of the two is missing. The missing one tells you what the answer will not claim.

58. How confident are you on the distinction?

Commit first

Commit before you check.

Predict first

A study randomly selects 2,000 people nationally and observes that gym members have lower cholesterol. What is the strongest supported conclusion?

  • Gym membership is associated with lower cholesterol nationally
  • Gym membership causes lower cholesterol nationally
  • Gym membership is associated with lower cholesterol among these 2,000 people only
  • Nothing can be concluded

Correct: Gym membership is associated with lower cholesterol nationally

Why: Random selection is present, so the finding generalises to the national population. Random assignment is absent — nobody was assigned a gym membership — so only an association is available. The third choice understates by ignoring the random selection, and the second over-claims by ignoring the missing assignment. This is the selection-without-assignment row of the four-combination table.

59. Draw the whole type on one page

Connect it up

Blank paper — and this one genuinely fits.

Draw it

Draw a two-by-two grid. Label the rows random assignment yes and no, and the columns random selection yes and no. Fill each of the four cells with what that combination supports: cause in the population, cause among the subjects, association in the population, association among the subjects. Beside the grid write the two sentences that generate it: assignment balances other factors so it licenses cause, and selection makes the sample representative so it licenses generalisation. Underneath, draw an example of a confounder — three circles, with an arrow from a third factor to each of the two observed quantities and no arrow between them. At the bottom write: size and care change nothing about which cell you are in.

60. Exit ticket

Exit ticket

One question before you close the deck.

Predict first

A study reports that participants were randomly selected but not randomly assigned. Which claim is available?

  • An association, generalisable to the population sampled
  • A causal claim about the population sampled
  • A causal claim about the participants only
  • No claim at all

Correct: An association, generalisable to the population sampled

Why: Random selection makes the sample representative, so whatever is found extends to the population it was drawn from. Random assignment is absent, so the groups formed themselves and confounders cannot be ruled out — meaning only an association is available. Both causal choices require assignment, and rejecting everything is the over-correction: a well-drawn observational study genuinely does establish a generalisable association.

61. What to take away

Recap

One type, one distinction, two sentences.

never do thisdo this instead
Conclude cause from an observational studySay associated with, and name a possible confounder
Treat random selection as licensing causeRead the word after randomly: assigned or selected
Generalise an experiment on volunteersKeep the causal claim and narrow the population
Argue a huge study proves causationSize buys precision, not validity
Answer nothing can be concluded by defaultIdentify the strongest claim the design supports
Judge a study by how careful it soundsJudge it by how subjects reached their groups

Naruhodo Tutoring — SAT Math question bank export (sat-question-index.json) — and the site's SAT pages to drill this type in isolation, then mixed

Sources

  1. College Board — Digital SAT Suite: test description and format
  2. College Board — Digital SAT Suite Assessment Specifications, Math section: domain weightings and skill definitions — College Board, 2023
  3. Naruhodo Tutoring — SAT Math question bank export (sat-question-index.json) — 1675 questions carrying official domain, skill and difficulty tags; the frequency figures in this deck are counted from this file
  4. Khan Academy — Official Digital SAT Prep, Math

Want this taught 1-on-1? Alexander tutors SAT Prep — $55/session, free consultation.

Book on Wyzant · Text (657) 465-8108