Have All Possible Influences on the Findings Been Identified and Controls Instituted?

A good research design controls or minimizes bias, the influence of factors other than the experimental ones on the results of the study. For example, a researcher plans to study the effect of toothpaste claiming a new chemical agent on plaque and gingivitis with the hypothesis that it will reduce plaque formation and gingival bleeding upon probing. In this case, the researcher wants assurance that, if the hypothesized results occur, they are due to the effect of the toothpaste and not some other factor, such as another mouthwash or drug. If such controls could not be imposed, then the researcher could not conclude with any certainty that the reduction in plaque formation and gingival bleeding was due to the chemical agent under study. The strategy is to identify as many factors as possible up front that could influence the results and systematically impose controls for them, such as preventing the subjects from using other drugs. For all the unidentified factors (confounders) that could possibly influence the results, of which there could be many, additional design strategies are employed.

One of these strategies is random assignment of subjects to treatment groups. There is usually an experimental group that receives the experimental therapy and a control group that receives no treatment, a placebo, or the currently recognized standard of therapy. Both groups are needed so comparisons can be made. With random assignment, every subject has an equal chance of being assigned to either group. The assumption can then be made that the unidentified influences are equal for both groups, and they differ from each other only in terms of the therapy being studied. Subjects can be randomly assigned to groups by generating random numbers from a computer, by using a random number table, or simply by drawing names from a hat. Selecting every nth person on a list is nota random assignment. Also, assigning subjects into groups based on the day they present themselves or the time of day may bias results in some unknown way. Such systematic assignment of subjects, though arbitrary, does not meet the requirement of randomness.

An additional control strategy is the use of a double-blind design (Figure 4). In this design, the subjects should not be able to determine whether they are in the control or the experimental group; they are "blind" to their assigned group. Additionally, the members of the research team who work directly with the subjects will not know (be “blind”) which of the subjects is receiving the experimental treatment and which is receiving the standard or placebo treatment. In the case of the subjects, knowledge of their group assignment may cause them to behave differently than normal, which would introduce new and uncontrolled influences on the results. For instance, individuals may perform oral hygiene better if they know they are using the newly designed toothbrush. This phenomenon of subjects performing better when they know they are in an experiment is known as the Hawthorne effect.2 Researchers may also perform differently with the knowledge of subject assignment, such as under assessing the level of gingival disease in an experimental group using a new antiplaque toothpaste.

Figure 4.

Double Blind Research Design

Investigators are concerned about identifying cause and effect relationships between variables so prevention and treatment strategies for disease can be developed. The extent to which investigators can conclude causation between the variables will depend on how well extraneous variables are controlled. For instance, in an experimental study of an antiplaque/antigingivitis mouthwash, where brushing habits, diet, and other key factors are carefully controlled across study groups, it may be valid for the authors to conclude the use of this mouthwash has the effect of reducing plaque deposits and gingivitis. Drawing conclusions of causation are more difficult in retrospective studies and especially in cross-sectional studies. For example, if a study showed a positive correlation (a statistical measure of association) between the amount of sugary beverages consumed and caries incidence in a sample of children, the investigators would need to be more cautious in making statements such as “sugary beverages cause caries.” Other unknown factors that may systematically associate with the beverage consumption may actually be the cause of the caries. Well-controlled prospective studies, in which the same subjects are evaluated at two or more time points, have the greatest potential for determining causation. In general, being absolutely certain of cause and effect remains one of the most vexing problems for scientists.