Choosing the right design/study
How to choose the right study design
Choosing the right study design is an important point and depends on different criteria: the question being asked in the study, related information and data previously published in the literature, which research approach is appropriate, existing regulations, time urgency, money, etc. As seen in General concepts of statistical trial design and methodology, different types of designs exists, some are observational (cohort studies, case control studies or cross sectional studies for example), some others are experimental (randomised controlled trials or non randomised ones). Pros and cons of these designs will be presented in this section and some guidelines on how to choose the most appropriate one will be introduced (Mann, 2003).
The right study design for a given clinical research question is dependent upon the nature of the question being asked. Questions pertain to a variety of central clinical issues such as therapy, diagnosis and prognosis. Each of these issues belongs in a distinct research category, for which a range of study designs is possible (Bragge, 2010). In the setting of melanoma, questions often relate to a type of therapy and these questions are best explored using an experimental study design.
Strength and weaknesses of Experimental studies
In clinical trials, one has to define the treatment and control groups, administer exposure to the treatment group, but not the control group, and follow both groups over a period of time in order to collect data that allows for appropriate comparisons to be made between the groups. The comparisons made will depend upon the defined outcomes of the clinical trial, such as the overall survival of the subjects in the treatment group compared with the control group, or perhaps the safety and tolerability of the treatment.
- Accuracy and validity
- The clinical trial has the best measure of exposure and accuracy because it is the only design in which the investigator administers a measured exposure to the trial groups. Because of the treatment and control groups, and group random assignment, experimental studies address more threats to internal validity than any other type of study. They give the most reliable indication as to whether or not a treatment or intervention had a causal effect on the treatment group.
- Similarities between groups
- In randomised trial designs the study groups are created to be similar. Randomisation guarantees that the groups will be very similar with regard to personal characteristics (e.g., sex, age) and other factors. This ensures that there is little difference between the groups except for their exposure to the treatment being studied. Therefore it is a controlled experiment and the differences seen can be attributed to the experimental treatment being administered. A clinical trial is the closest an epidemiologist can get to a controlled experiment involving humans. In all other types of study designs such as in observational ones, it is more difficult to be sure that study groups are similar.
- Scientifically sound
- A well thought out and designed clinical trial allows there to be confidence in the results. Randomised clinical trials are considered to be the gold standard because they reduce bias and allow tight control regarding treatment exposure and measurements of the impact of that exposure.
- Time consuming
- Trials involve the additional steps of randomisation, giving a treatment to the experimental group over time, and periodically determining and recording a number of factors such as disease status, adverse effects, health system costs and the well being of the subjects. The investigator observes the entire journey from treatment exposure to a defined endpoint e.g. occurrence of new disease or death, and this is time consuming.
- The long course of the study and the attention paid to obtaining accurate and complete information on exposure and outcome is costly.
- In a clinical trial subjects are intentionally exposed to an experimental treatment. If the exposure is hypothesised to be harmful, it would not be ethical to intentionally expose people to the treatment. In observational studies, harmful exposures can be studied because the investigator just observes what people are exposed to in their daily lives.
Strength and weaknesses of Observational studies
In cohort studies, the researcher has to select a healthy study sample, observe who is exposed and who is not, follow both groups over a period of time in order to collect data that allows for appropriate comparison of the risk of disease in the exposed group with that in the unexposed group. The investigator is involved during the entire time between exposure and disease. These are the best method for determining the incidence and natural history of a condition.
The studies may be prospective or retrospective and sometimes two cohorts are compared. Cohort studies look forward in time by following up each subject.
- Exposure is being measured at the time of the study. The cohort design follows study subjects more closely than other observational study designs throughout the journey between exposure and disease (since it starts out with healthy people, it is possible to establish that exposure occurred first, then disease). That makes the investigator more confident in the findings compared to other observational studies.
- Per variable effect assessment
- Permit calculation of the effect of each variable on the probability of developing the outcome of interest (see Choosing the right Test).
- Time consuming
- The investigator could have to wait several months or years for disease to develop or not.
- This design involves large numbers of study subjects and often repeated measures of exposures and disease over a long study period. At the end, it also takes time and/or money to trace everyone's disease status.
Case Control Studies
In case control studies, the researcher has to select a group of people with the disease of interest, such as melanoma (cases) and a similar group without the disease (controls). Both groups are questioned regarding past behaviours, perhaps particularly focusing on known melanoma risk factors, so that these can be compared between the two groups. Case-control studies look back at what has happened to each subject and as such relies on the memory of the participants.
- Rare disease
- Case control designs allow study of rare diseases because the first step is to identify cases. In contrast, the cohort design starts out with a group of healthy people, some of who have, and do not have, the exposure of study interest.
- Time consuming
- Less time consuming than cohort studies but takes time to select cases and controls and meet with each study participant to ask questions about past exposures. Even if the questions are asked in a mailed questionnaire, it will take time to gather in all the replies.
- Less expensive than cohort studies but spends time and/or money on defining cases and controls and in obtaining a detailed history of past exposures. Calculate odds ratios, which in turn, usually approximate to the relative risk.
- Case control studies are vulnerable to serious error as it relies on subjects recalling past exposures and so the information provided may not be accurate. Furthermore, people who have the disease may recall their exposures more completely or in an exaggerated way, compared with people who do not have the disease, because they are searching for explanations for their illness.
- Time order
- The investigator does not start out with a group of healthy people, measure exposure, and then measure disease. Therefore, the time order of the exposure and disease cannot be identified with certainty (If you see people for the first time only after they already have the disease, it is not possible to know which came first, the exposure or the disease).
- Similarity of groups
- Case control studies pose a particular challenge in trying to find a comparison group of people (controls) who are similar to the group with the disease of interest (cases) but who do not have the disease. This is a particular vulnerability of the case control design.
Cross sectional studies
In cross sectional studies the researcher has to select a study sample, ask each person about both exposure and disease at that point in time (a "snapshot") and compare the disease risk in the exposed group with that in the unexposed group. Cross sectional studies look at each subject at one point in time only.
- Time and money
- Cross sectional studies are quick and remain the least expensive of the study designs. As there is no follow up (there is no waiting for disease or other outcome to develop), less resources are required to run the study. Relative to the other designs, the investigator does not spend as much time with study subjects or as much time and effort measuring exposure and disease, therefore there are not as many data points to document. They can be seen as a snapshot of the exposures and diseases of a group of people at one point in time. They are the best way to determine prevalence of a disease of interest.
- As in case control studies, the investigator does not start out with a group of healthy people, measure exposure, and then measure disease. So the most significant problem with this type of study is that they do not themselves differentiate between cause and effect or the sequence of events (the time order of the exposure and disease cannot be identified with certainty). Furthermore because they are simply a measure of exposure at one point in time, they do not provide enough detail to be very helpful in figuring out relationships between exposures over time and disease.
- Bragge, P. (2010). Asking good clinical research questions and choosing the right study design. Injury, 41, S3-S6.
- Mann, C. J. (2003). Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emergency Medicine Journal, 20(1):54-60.
- Beller, Elaine (2013). Clinical Trials Protocol Design. [PowerPoint presentation]
- Coggon D., Rose, G., and Barker, D. J. P. Epidemiology for the uninitiated. British Medical Journal. [Online content]