Random assignment, confounding and causal claims

An association is an observation

Two variables are associated when their values vary together in a pattern. That pattern can be informative, but it does not by itself establish that one variable caused the other. A third variable may influence both, the direction of influence may be different from the proposed one, or the study's selection and measurement processes may affect the result. A causal question asks what would happen under a change in a condition, not simply which observations occurred together. Study design helps make that question answerable. In an experiment, an investigator assigns a condition; in an observational study, the investigator observes conditions without assigning the exposure in that way.

Assignment and sampling do different jobs

Random assignment allocates participating units to conditions by a chance process. It helps balance potential confounders between conditions in expectation, although a particular small sample can still be imbalanced. Random sampling concerns how units are selected from a population. The former mainly supports comparisons among conditions; the latter concerns representation of a target population. Randomly assigning volunteers does not make those volunteers a random sample of all learners. Similarly, a representative survey does not automatically become an experiment. Keep the selection step and the assignment step separate when evaluating what a result can support.

Worked example: a plausible confounder

A fictional survey finds that students who use a review application score higher on a practice test. The survey also finds that application users, on average, studied longer before the test. It would be premature to attribute the entire score difference to the application. Study time is a plausible alternative contributor because it differs between the groups and can relate to performance. Prior preparation, motivation and access to study materials could also matter. The survey's association may justify further investigation, but the phrase “the application caused the increase” goes beyond what this comparison alone establishes.

Worked example: a randomized comparison

Suppose 40 volunteers are randomly assigned to two fictional study activities, with 20 in each group. Both groups receive the same practice time and then take the same scoring-blinded assessment. Group A's average is 72 and group B's is 66, a difference of six score points. The design addresses several alternative explanations more directly than the survey: allocation was random, exposure time was matched, and scorers did not know the assigned condition. Still, the observed difference has sampling uncertainty. It does not show that every participant benefited by six points or that the result automatically generalizes to all Philippine examinees. Statistical analysis and replication would strengthen interpretation.

Control does not mean perfection

Holding conditions comparable and blinding scoring can reduce particular sources of bias. They do not eliminate every limitation. Participants may not follow instructions, outcome measures may cover only part of the intended skill, and missing follow-up scores may differ between groups. Report those features rather than treating the word “randomized” as a guarantee. Another common error is interpreting a group average as an individual rule. Two groups can differ in average score while their individual scores overlap substantially. A study can support an estimated average effect under its conditions without showing that a particular learner must experience that exact change.

Sources & further reading