Choosing the right design/study

Dr Anne Bernard (a.bernard@qfab.org)

How to choose the right study design

Choosing the right study design is an important point and depends on different criteria: the question being asked in the study, related information and data previously published in the literature, which research approach is appropriate, existing regulations, time urgency, money, etc. As seen in General concepts of statistical trial design and methodology, different types of designs exists, some are observational (cohort studies, case control studies or cross sectional studies for example), some others are experimental (randomised controlled trials or non randomised ones). Pros and cons of these designs will be presented in this section and some guidelines on how to choose the most appropriate one will be introduced (Mann, 2003).

The right study design for a given clinical research question is dependent upon the nature of the question being asked. Questions pertain to a variety of central clinical issues such as therapy, diagnosis and prognosis. Each of these issues belongs in a distinct research category, for which a range of study designs is possible (Bragge, 2010). In the setting of melanoma, questions often relate to a type of therapy and these questions are best explored using an experimental study design.


Strength and weaknesses of Experimental studies

In clinical trials, one has to define the treatment and control groups, administer exposure to the treatment group, but not the control group, and follow both groups over a period of time in order to collect data that allows for appropriate comparisons to be made between the groups. The comparisons made will depend upon the defined outcomes of the clinical trial, such as the overall survival of the subjects in the treatment group compared with the control group, or perhaps the safety and tolerability of the treatment.

Pros

Accuracy and validity
The clinical trial has the best measure of exposure and accuracy because it is the only design in which the investigator administers a measured exposure to the trial groups. Because of the treatment and control groups, and group random assignment, experimental studies address more threats to internal validity than any other type of study. They give the most reliable indication as to whether or not a treatment or intervention had a causal effect on the treatment group.
Similarities between groups
In randomised trial designs the study groups are created to be similar. Randomisation guarantees that the groups will be very similar with regard to personal characteristics (e.g., sex, age) and other factors. This ensures that there is little difference between the groups except for their exposure to the treatment being studied. Therefore it is a controlled experiment and the differences seen can be attributed to the experimental treatment being administered. A clinical trial is the closest an epidemiologist can get to a controlled experiment involving humans. In all other types of study designs such as in observational ones, it is more difficult to be sure that study groups are similar.
Scientifically sound
A well thought out and designed clinical trial allows there to be confidence in the results. Randomised clinical trials are considered to be the gold standard because they reduce bias and allow tight control regarding treatment exposure and measurements of the impact of that exposure.

Cons

Time consuming
Trials involve the additional steps of randomisation, giving a treatment to the experimental group over time, and periodically determining and recording a number of factors such as disease status, adverse effects, health system costs and the well being of the subjects. The investigator observes the entire journey from treatment exposure to a defined endpoint e.g. occurrence of new disease or death, and this is time consuming.
Expensive
The long course of the study and the attention paid to obtaining accurate and complete information on exposure and outcome is costly.
Ethics
In a clinical trial subjects are intentionally exposed to an experimental treatment. If the exposure is hypothesised to be harmful, it would not be ethical to intentionally expose people to the treatment. In observational studies, harmful exposures can be studied because the investigator just observes what people are exposed to in their daily lives.

Strength and weaknesses of Observational studies

Cohort Studies

In cohort studies, the researcher has to select a healthy study sample, observe who is exposed and who is not, follow both groups over a period of time in order to collect data that allows for appropriate comparison of the risk of disease in the exposed group with that in the unexposed group. The investigator is involved during the entire time between exposure and disease. These are the best method for determining the incidence and natural history of a condition.

The studies may be prospective or retrospective and sometimes two cohorts are compared. Cohort studies look forward in time by following up each subject.

Pros

Accuracy
Exposure is being measured at the time of the study. The cohort design follows study subjects more closely than other observational study designs throughout the journey between exposure and disease (since it starts out with healthy people, it is possible to establish that exposure occurred first, then disease). That makes the investigator more confident in the findings compared to other observational studies.
Per variable effect assessment
Permit calculation of the effect of each variable on the probability of developing the outcome of interest (see Choosing the right Test).

Cons

Time consuming
The investigator could have to wait several months or years for disease to develop or not.
Expensive
This design involves large numbers of study subjects and often repeated measures of exposures and disease over a long study period. At the end, it also takes time and/or money to trace everyone's disease status.

Case Control Studies

In case control studies, the researcher has to select a group of people with the disease of interest, such as melanoma (cases) and a similar group without the disease (controls). Both groups are questioned regarding past behaviours, perhaps particularly focusing on known melanoma risk factors, so that these can be compared between the two groups. Case-control studies look back at what has happened to each subject and as such relies on the memory of the participants.

Pros

Rare disease
Case control designs allow study of rare diseases because the first step is to identify cases. In contrast, the cohort design starts out with a group of healthy people, some of who have, and do not have, the exposure of study interest.
Time consuming
Less time consuming than cohort studies but takes time to select cases and controls and meet with each study participant to ask questions about past exposures. Even if the questions are asked in a mailed questionnaire, it will take time to gather in all the replies.
Expense
Less expensive than cohort studies but spends time and/or money on defining cases and controls and in obtaining a detailed history of past exposures. Calculate odds ratios, which in turn, usually approximate to the relative risk.

Cons

Accuracy
Case control studies are vulnerable to serious error as it relies on subjects recalling past exposures and so the information provided may not be accurate. Furthermore, people who have the disease may recall their exposures more completely or in an exaggerated way, compared with people who do not have the disease, because they are searching for explanations for their illness.
Time order
The investigator does not start out with a group of healthy people, measure exposure, and then measure disease. Therefore, the time order of the exposure and disease cannot be identified with certainty (If you see people for the first time only after they already have the disease, it is not possible to know which came first, the exposure or the disease).
Similarity of groups
Case control studies pose a particular challenge in trying to find a comparison group of people (controls) who are similar to the group with the disease of interest (cases) but who do not have the disease. This is a particular vulnerability of the case control design.

Cross sectional studies

In cross sectional studies the researcher has to select a study sample, ask each person about both exposure and disease at that point in time (a "snapshot") and compare the disease risk in the exposed group with that in the unexposed group. Cross sectional studies look at each subject at one point in time only.

Pros

Time and money
Cross sectional studies are quick and remain the least expensive of the study designs. As there is no follow up (there is no waiting for disease or other outcome to develop), less resources are required to run the study. Relative to the other designs, the investigator does not spend as much time with study subjects or as much time and effort measuring exposure and disease, therefore there are not as many data points to document. They can be seen as a snapshot of the exposures and diseases of a group of people at one point in time. They are the best way to determine prevalence of a disease of interest.

Cons

Accuracy
As in case control studies, the investigator does not start out with a group of healthy people, measure exposure, and then measure disease. So the most significant problem with this type of study is that they do not themselves differentiate between cause and effect or the sequence of events (the time order of the exposure and disease cannot be identified with certainty). Furthermore because they are simply a measure of exposure at one point in time, they do not provide enough detail to be very helpful in figuring out relationships between exposures over time and disease.
Figure 1: Summary of the strength and weaknesses of Experimental and Observational studies