MCAT content library / Research & data / Observational and experimental studies: what can we conclude?
Match a causal claim to group formation, measurement timing, and the alternatives the design leaves open.
About 14 minutes, including the worked examples and checks.
Read a study and its figures · Lesson 3 of 6 · Continue in this path
On this page
- Concept: classify what the researchers did
- Pattern: the same research question, three designs
- Rule: random assignment changes the comparison, not every limitation
- Application: distinguish the final score from the change
- Application: read a randomized result with its missing data
- Make the conclusion fit the design
- Check your reasoning
- Continue learning
- Sources
Concept: classify what the researchers did
Two studies can report the same difference and support different conclusions. The distinction begins before the results: did researchers observe an exposure that participants already had, or assign an intervention? In an observational study, they measure naturally occurring differences. In an experiment, they manipulate an explanatory variable and measure an outcome. An experiment may use random assignment, but manipulation and randomization are separate features.1
Imagine that students who spread practice across several days score higher on a delayed recall test than students who complete practice in one sitting. That pattern could reflect an effect of the schedule. It could also reflect differences in prior knowledge, motivation, total practice time, or the circumstances that made each schedule possible. The association itself does not identify which explanation generated it.
For MCAT research questions, match the conclusion to three features: how groups were formed, when variables were measured, and what else differed between groups. AAMC explicitly includes reasoning about temporality, random assignment, confounding, and the limits of generalization.2
Pattern: the same research question, three designs
Consider three hypothetical studies of practice schedule and delayed recall. Their designs, rather than any claimed real-world result, are the focus.
| Design | What happens | What the design adds |
|---|---|---|
| Single-time observational study | Students report their usual practice pattern and complete a recall test during one visit. | Describes an association in the measured sample; ordering of underlying differences remains uncertain. |
| Prospective observational study | Researchers record students’ chosen schedules before a later recall test. | Establishes that the recorded schedule precedes that outcome measurement; selection differences can remain. |
| Randomized experiment | Students are assigned to schedules with matched materials and total practice time, then tested after the same delay. | Breaks systematic selection into schedules and strengthens a causal comparison of the assigned interventions. |
Following people over time answers an ordering question, but it does not automatically answer a confounding question. A student’s prior knowledge may influence both the schedule they select and their later score. Measuring schedule first does not remove that common cause. Nor does a prospective design guarantee that an earlier, unmeasured version of the outcome played no role in the exposure choice.
A potential confounder is a variable that can provide an alternative explanation for the exposure–outcome relationship because of how it relates to both. It is not simply any extra variable mentioned in a passage. Here, a difference in total study time matters if it accompanies schedule choice and affects recall. A difference unrelated to either schedule or recall would not explain the association.
Rule: random assignment changes the comparison, not every limitation
Proper random assignment makes treatment selection independent of participants’ characteristics through the allocation process. It balances those characteristics in expectation across repeated randomizations; it does not guarantee identical groups in one finite sample. Chance imbalances can occur. The CONSORT explanation of randomized trials makes this distinction explicit.3
Random assignment also does not recover missing outcomes, enforce adherence, or ensure that the groups receive equal attention after assignment. If the spaced group receives daily coaching while the one-session group receives none, the intervention is a package of schedule plus coaching. A difference could support an effect of that package without isolating the schedule itself.
Random sampling and random assignment answer different questions. Sampling concerns how participants enter the study and helps assess which population they represent. Assignment concerns how enrolled participants enter comparison groups. A study can have random assignment among volunteers from one campus and still have limited grounds for generalizing to all learners. The sampling frame, eligibility criteria, and nonresponse matter for that broader claim.4
Application: distinguish the final score from the change
In this illustrative observational dataset, students choose their own schedule. The same students complete both tests, using a common 0–100 score scale.
| Chosen schedule | Students | Mean baseline score | Mean follow-up score | Mean change |
|---|---|---|---|---|
| Spaced practice | 48 | 72 | 81 | +9 |
| One-session practice | 52 | 60 | 67 | +7 |
The follow-up difference is 81 − 67 = 14 points. The baseline difference was already 72 − 60 = 12 points. Comparing changes gives 9 − 7 = 2 points. Those are three different quantities, not competing calculations of the same quantity. Because the same participants have both measurements, each group’s difference between means equals its mean individual change.
The statement “spaced practice raised scores by 14 points” attributes the entire follow-up gap to the schedule even though most of that gap was present beforehand. The more precise descriptive statement is that the spaced group finished 14 points higher and improved 2 points more on average. Neither statement, by itself, establishes a causal schedule effect in this self-selected sample.
Subtracting baseline scores addresses the visible starting difference numerically. It does not show that motivation, prior exposure, measurement error, or changes in study time were equivalent. Baseline adjustment can be useful, but its existence is not a substitute for examining how the data were generated. Also, the table supplies no within-group variability, so it does not tell us how uncertain the observed difference is.
Application: read a randomized result with its missing data
Now suppose 80 students are randomly assigned, 40 per schedule. Only 20 spaced-practice students and 38 one-session students complete the outcome test. The completers average 78 and 70, respectively. The 8-point difference describes the students whose outcomes were observed. It does not automatically preserve the original randomized comparison.
If lower-performing participants preferentially left the spaced group, the remaining group could look stronger even without a beneficial treatment effect. That is a possible explanation, not a conclusion that dropout definitely caused the difference. To evaluate it, researchers need reasons for missingness, information on those missing, and an analysis that addresses missing outcomes under explicit assumptions. Reporting only completers hides the evidence needed to assess the claim.
Make the conclusion fit the design
Use a sentence with a named comparison and a boundary: “Among the observed students, the chosen schedules were associated with different follow-up scores,” or “Under this randomized, matched-time protocol, assignment to one schedule affected the measured outcome,” when implementation and analysis support that inference. Then separately ask whether the magnitude is precise and whether the result extends to other people, materials, or time intervals. Causal support, uncertainty, and generalizability are related questions, but answering one does not settle the others.
ACTIVE PRACTICE · RESEARCH INFERENCE
Separate the Observed Difference From a Causal Claim
The original observational dataset compares students who chose spaced practice versus a one-session schedule. Baseline means are 72 and 60; follow-up means are 81 and 67. In a separate randomized study, only 20/40 and 38/40 assigned students complete follow-up.
Spaced: 72 → 81 (change +9). One-session: 60 → 67 (change +7). The groups were self-selected.
Randomized study attrition: 40 assigned per group; observed outcomes for 20 spaced and 38 one-session participants. Completer means: 78 versus 70.
Check your reasoning
Choose an answer before opening its feedback. Each option explains the reasoning behind that choice.
Check 1
Researchers record weekly work hours at the start of a semester and measure sleep duration at its end. Students working more hours later report less sleep. Which conclusion is best supported by the design described?
A. The association is free of confounding because work hours were recorded first.
Review this choice. Measuring exposure first improves the temporal description. It does not make students with different work hours equivalent in financial strain, caregiving, baseline sleep, or other relevant characteristics.
B. The difference estimates the sleep change caused by assigning additional work hours.
Review this choice. No work schedule was assigned, and no within-person change was specified. This statement adds both an intervention and a change estimate that the design does not provide.
C. The recorded exposure precedes the measured outcome, while common causes may remain.
Correct. The design establishes the order of these measurements. It remains observational, so variables such as financial strain could influence both work hours and sleep.
D. The result establishes that earlier sleep duration could not influence work-hour choice.
Review this choice. Earlier sleep or related circumstances were not ruled out. Recording work hours before the final sleep measurement does not show that the underlying sleep pattern began afterward.
The principle to carry forward
Prospective measurement strengthens the temporal account without erasing selection differences or earlier unmeasured states.
Check 2
Volunteers are randomly assigned to two reading interfaces. One group later has a higher mean comprehension score. Which additional design detail most directly limits generalization to older adults, rather than introducing another explanation for the difference between the assigned groups?
A. Every participant was between 18 and 22 years old.
Correct. The causal comparison can be sound for the studied participants, yet older adults are absent from the sample. Extending the result requires an assumption that the effect carries across age and related characteristics.
B. Each interface group is assessed by a different examiner using discretionary scoring.
Review this choice. Examiner differences can help explain the observed score difference within the study. They affect interpretation of the comparison itself, rather than specifically defining the population to which it might extend.
C. Only participants who prefer their assigned interface complete the outcome test.
Review this choice. Selecting outcomes according to post-assignment preference can change the composition of the observed groups. This threatens interpretation of the randomized comparison, rather than merely limiting its age range.
D. The two groups receive different reading passages at the outcome test.
Review this choice. Different passage difficulty could help explain the score difference. This changes the comparison within the study rather than specifically restricting generalization to older adults.
The principle to carry forward
A credible causal comparison in a narrow sample can coexist with uncertainty about whether the effect extends to another population.
Continue learning
Read a study and its figures
Lesson 3 of 6.
- Previous in Read a study and its figures: Experimental Controls: What a Comparison Can Show
- Next in Read a study and its figures: Reading graphs: axes, scales, and the comparison that matters
- Return to this path’s overview
See every step in Read a study and its figures
Read a study and its figures
Follow a question through its design, graph, normalization and statistical description.
- Hypotheses, predictions, and competing explanations
- Experimental Controls: What a Comparison Can Show
- Observational and experimental studies: what can we conclude? · You are here
- Reading graphs: axes, scales, and the comparison that matters
- Normalized data, percent change, and fold change
- Mean, median, and spread: describing a dataset
Related lessons
- Experimental Controls: What a Comparison Can Show
- Hypotheses, predictions, and competing explanations
- Mean, median, and spread: describing a dataset
Browse research & data · MCAT content library
MCAT preparation and roadmap · Aim High, Score Higher
Sources
1. OpenStax Introductory Statistics 2e: Experimental Design and Ethics
2. AAMC: Scientific Inquiry and Reasoning Skill 3
3. SPIRIT–CONSORT: Item 25, Baseline Data
4. OpenStax Introductory Statistics 2e: Data, Sampling, and Variation