Homepage: Topic Group 5

Appropriate and valid study design is crucial for the valid conduct of observational studies (Elm 2007¹). Observational studies can make a key contribution to establishing causal relationships, in combination with other types of evidence (e.g. mechanistic studies, animal studies) (Hill 1965²). They may be particularly valuable when randomized trials are impractical and/or unethical, e.g. for exposures such as smoking and lung cancer (Hill 1965²). Some observational studies observe and measure associations without any attempt to infer causality, for example prognostic studies relating to biomarkers. Other studies may seek solely to get unbiased estimates of some phenomenon, such as the prevalence of some condition in a community.

The appropriateness of any study design thus depends on the research question, in the context of the current state of theory and knowledge, the availability of valid measurement tools, and the proposed uses of the information to be gathered. Although, in theory, certain study designs are better than others, in practice the validity of a study design is highly topic and context-specific (Pearce 2011³). Hierarchies of observational study designs are often proposed (usually with cohort studies at the top, followed by case-control studies, cross-sectional studies, etc). However, their relative validity represents a continuum, rather than a dichotomy, and ‘less valid’ study designs may yield valuable information in some instances (Pearce 2011³).

It is highly unusual for a single observational study to deliver definitive results. Assessing the epidemiologic evidence almost always involves a process of triangulation across studies in different populations, using a variety of study designs, investigators, and methods. No individual study can be perfect, or can deliver a definitive answer. Rather, the aim of a particular study is to contribute to the pool of knowledge for a particular issue.

With these considerations in mind, there are a number of important considerations with regards to design of observational studies:

1. What is the question that one wants to answer? What does this imply about the basic design?

2. What is the most efficient study design?

3. What is the most appropriate (and available) study source population and risk period?

4. How can the design assist in an effort to control for potential confounding?

5. How can design assist in assuring reliable exposure assessment?

6. How can the design assist in assuring reliable and complete disease ascertainment?

7. What are the best methods to assure completeness of the data from sampled subjects?

8. What is the role (if any) of subgroup analysis, and what does this imply for the study design?

9. What are the specific issues involved in the design of studies of prognosis (prognostic factors and prognostic modeling studies) and diagnosis?

Appropriate and valid study design involves a context-specific balance between these competing considerations (Rothman 2008⁴). Whatever study design is used, it is also important to have the capacity to conduct sensitivity analyses, e.g. by gathering information on the likely extent of uncontrolled confounding and measurement error.

This topic group will review these issues, and will produce guidelines (but not inflexible rules) for weighing up these considerations when designing an observational study.