7.5 Design issues: Overview

In summary, issues to consider when designing a study, when possible, include:

• Minimising confounding (and lurking variables);
• Minimising the carryover effect;
• Minimising the Hawthorne effect;
• Minimising the observer effect;
• Minimising the placebo effect.

Ways to minimize the impact of these have been discussed (Fig. 7.7), but is not always possible. These effects are important to understand, so studies can be designed to manage or minimise their influence. This ensures that the results and conclusions from our studies are correctly interpreted (that is, noting, for example, how the Hawthorne effect may have influenced the conclusions).

Often, however, some (or all) of these issues cannot be well managed. For instance, individuals often know they are involved in an experimental study (Hawthorne effect). In these cases, the impacts should be minimized as far as possible, and then the likely impact that these issues have on our conclusions discussed. The impact of these issues are often reported as limitations in a journal article (Chap. 9), perhaps part of the Discussion section.

Example 7.7 (Study limitations) A study of alcohol use in college females reported these limitations of their study:

The present study has several limitations. First, data were collected over 15 years ago […] Second, only college females were assessed and findings may not generalize to college males or to broader groups of young adults […] Third, alcohol and caffeine consumption variables were all self-reported…

— Kelpin et al. (2018), p. 3

FIGURE 7.7: Design considerations. Note: Lurking variables become confounding variables when measured, observed, assessed or recorded in the study, and then they can be managed. The arrows mean that the design issue can be partially managed by the indicated means