Melanoma-specific trial design and methodology

Justin Scott (


Other sections pertaining to this statistical guide are quite generic, dealing with statistics applicable to a wide variety of research fields. This section introduces the statistical concepts (and pitfalls) specific to clinical, medical, or public health trial design and analysis within the field of melanoma research, specifically highlighting recruitment and retention, socio-demographic bias, subjective measurements (e.g. biopsies), longitudinal studies, gene expression or protein analysis (bioinformatics) and interim analyses.

Recruitment and Retention

Recruitment and retention strategies can dramatically influence the quantity and quality the final data set. Planning, communication, liaison, and management all impact on the recruitment and retention of patients, volunteers, clinical staff, or investigators. Poor implementation translates into longer recruiting times, more withdrawals and drop-outs, poorer quality data, and either an over-time or underpowered study.

The type of study undertaken will dictate the ease or difficulty of recruitment and retention. For example, retrospective studies usually do not require patient consent, just ethical approval, whereas finding twin siblings to use as controls for patients with a rare disease will be problematic.

Aitken et al (2003) identified a number of principles to guide the processes of recruitment and retention:

  1. Selection of an appropriate population
  2. Establishment of a sampling process that represents the population
  3. Creation of systematic and effective recruitment mechanisms
  4. Implementation of follow-up mechanisms that promote participant retention
  5. Maintenance of ethics and privacy regulations

The following are some suggestions to maximize recruitment:

  • Emphasize the benefits to the community and individual
  • Outline steps to protect confidentiality
  • Provide an estimate of how long it will take to participate
  • Identify the senior researcher’s position and organisation

The following are some suggestions to maximize retention:

  • Contact the participants at regular intervals
  • Communicate the status or results of the study

The recruitment or retention rates should be reported in any publication, as this gives an indication as to the success of the trial and how robust the final data set might be.



Are the trial patients representative of their background population? If not, how do they differ? Does the final cohort differ from the drop-outs cohort? Are these differences biasing the results observed?

The objective is to record and report on genetic, environmental and ethnic or cultural background information that might influence clinical outcomes. Socio-demographic factors include: Age, Gender, Marital Status, Country of birth or Language spoken at home, Ethnicity: e.g. Australian Aboriginal, Torres Strait Islander, or Maori background; Post code, Metropolitan versus Regional and Remote, Income and Socioeconomic status.

As an example as to why some of the above data is collected; post codes by themselves are meaningless as numbers but can be sub-grouped into metropolitan versus regional/remote, socio-economic groupings or translated into median salaries using data from the Australian or New Zealand government’s statistical or taxation bodies. It is important to identify how far-reaching a trial is and if it is representative of regional/remote areas.

Being aware of a patient’s ethnicity may allow for data to be viewed in a range of perspectives, as it may become apparent that a drug under trial has a greater impact amongst a certain sub-population.

Quantitative and qualitative reports suggest clinical trials are prone to recruitment and drop-out biased socio-demographic cohorts. There is a concern that those of a lower socio-economic background are less likely to participate in a clinical trial and more likely to withdraw, thus introducing a bias into the results. Therefore the associated background population socio-demographic breakdowns should also be documented and commented upon.

It is recommended that any control arm be balanced for age and gender within the treatment arm(s) but this is an added complication to a simple randomized control trial. For more information on balancing techniques refer to the article "General concepts of statistical trial design and methodology: Randomisation".


Subjective Measurements

An objective measurement will always be the same regardless of the circumstances, assessor, or instrument. Examples include: age, gender, height and weight. A subjective measurement is one that is open to interpretation or estimated. Some characteristics may not be measured with precision and there may be a disagreement between experts on what is being observed. For example, the presence, extent or severity of melanoma identified in a body area, under a microscope, by photograph or radiograph could be described as "borderline" and open to interpretation and differing of opinions even by experts.

Care will need to be taken in deciding how best to measure a subjective observation. The following are some suggestions to consider when designing a clinical trial with subjective observations:

  • Review the literature for an appropriate assessment technique(s).
  • Run a small pilot study with multiple assessors to assess their agreement and variability. Additionally compare the results with a more expensive "gold standard".
  • Using expert input, ensure the trial protocol or other document such as the Study Operations Manual outlines exactly how something should be measured and assessed, describing in detail the techinque to be used.
  • Use at least two, preferably three expert raters.
  • If a third rater is too expensive for the entire trial, include a third rater for a random sample of subjects.
  • Workshop or train the raters as a group prior to the trial. This will allow for a common reference or baseline and guidance on "borderline" cases.
  • Decide prior to the trial on a protocol for handling disagreements in the assessment(s) e.g. consensus, majority

In the case of multi-site investigations, the options are:

  • Have the experts/rating teams from each site trained collectively,
  • Have experts/rating team visit each site,
  • Have a centralized team with the measures sent to them,
  • Methodically check the first results for any confusion or misunderstanding

As part of the report, you (or your Biostatistician) will need to derive the agreement and/or correlation between the assessors.


Longitudinal Study

A longitudinal study is one where all or some of the measurements made at the beginning are repeated over time and is common in melanoma clinical trials. Note that this differs from a repeated measures study where each subject receives more than one treatment.

Common clinical measurements taken over time in melanoma studies include weight, ECOG performance status, mental well being and change in the spread and/or severity of disease.

Longitudinal studies are at risk of being substantially underpowered unless additional characteristics are factored into its sample size calculation. Drop-outs are no longer an overall 10% or 20% but becomes a rate. The longer the trial, the greater the number of drop-outs. Participants become more fatigued with the trial as time goes on, especially if there is a large number of tests or of a complex, invasive, or painful nature. They may move interstate, overseas, they may die, or they may seek alternative therapies. The drop-out rate can increase as time goes on as the initial enthusiasm wanes. Even if a participant remains in the trial, they may not be available at every time point or for every measurement and then there are naturally occurring errors. These produce missing values. Over time these missing values accumulate. A standard statistical software package will remove all cases that contain missing values in the variable of interest which can leave only a fraction of overall cases. These missing values can be imputed or estimated by a Mixed Effects model or Generalised Estimating Equation (GEE) model but then there is a question mark over their representativeness especially if the missing values constitute a substantial proportion of the overall data.

Calculating sample size in longitudinal studies

In a longitudinal study the sample size calculation is based on the number of participants still participating at the end-point not at recruitment or baseline. This must then be inflated to take into account the likely causes of attrition and missing values that occur after recruitment.

N\prime = \frac{N}{[(1 - MVP).(1-DR)^{t}]}

  • N is the number of patients
  • MVP is the estimated Missing Value Percent
  • DR is the Dropout Rate
  • t is the number of time points
Example: calculating sample size in longitudinal studies

This example study begins with 100 subjects with a dropout rate of 10% per time point and a missing value rate of 10% per time point. After 12 time points there are only 25 subjects for which there is full data available.

TimeStartDropoutsMissing valuesFull Data
Baseline 100.00 10.00 9.00 81.00
1 90.00 9.00 8.10 72.90
2 81.00 8.10 7.29 65.61
3 72.90 7.29 6.56 59.05
4 65.61 6.56 5.90 53.14
5 59.05 5.90 5.31 47.83
6 53.14 5.31 4.78 43.05
7 47.83 4.78 4.30 38.74
8 43.05 4.30 3.87 34.87
9 38.74 3.87 3.49 31.38
10 34.87 3.49 3.14 28.24
11 31.38 3.14 2.82 25.42

N\prime = \frac{N}{[(1-MVP) \times (1-DR)^{t}]}

  • N = 100
  • MVP = 0.1
  • DR = 0.1
  • t = 12

N\prime = \frac{100}{[(1-0.1)(1-0.1)^{12}]}

N′ = 318

In this example, 318 subjects must be recruited for an end-point cohort of 100.

Deciding on the number of patients to recruit prior to the trial can be very challenging. As an alternative design your study such that there will be sufficient patient retention to the endpoint of the study.

It is recommended that all patients be observed for the same minimum amount of time to neutralize any time bias and that this be some multiple of 365 days to neutralize any seasonal bias.

It may not be possible to recruit all the desired patients immediately, only as they present. This produces an additional time lag from the date the last patient is recruited through to their minimum observation time or end-point. A proposed ‘five year’ study, therefore, may take longer than five years to complete and care must be taken in correctly assessing the over-all time and costs involved. Additional time must be factored in for data quality control, analysis and write-up.


Gene Expression or Protein Analysis (Bioinformatics)

Roxane Legaie (

Gene expression is the process by which information from a gene (DNA) is used to synthesise functional gene products. These products are usually proteins, which go on to perform essential functions in the cells as enzymes, hormones, receptors, etc. These molecules are essential to every cell in an organism, and vary in quantity and quality along the life cycle of that organism.

The study of a gene expression profile in a cell or tissue at a particular moment gives an insight into the state of that cell: multiplication, growth, stress, death. Recent technological advances make it possible to analyse the expression of the entire genome (all the genes) in a single experiment. These “gene expression assays” complement or replace previous assays which measured the gene expression of only one gene, or a selected group of genes at a time.

Because gene expression is regulated by many pre and post transcriptional events which may result in truncated or degraded products, protein analysis provides a higher level of insight into the real status of a cell. In particular quantitative proteomics can be used to determine the amount of proteins in a sample under a particular condition and can be compared to other conditions.

In most cases gene expression studies are conducted in order to identify genes and pathways dysregulated in a particular disease or in response to a specific treatment. As the study of genomics advances, the application of genomic information is expected to enhance the diagnosis, prognosis, treatment and prevention of many different diseases.

While genetic tests have long been used to determine a patient risk or hereditary predisposition for cancer (and other inherited diseases), recent advances in the field have permitted the development of genomic tests which look specifically at a patient’s tumor genes and try to identify their drivers. The use of gene expression testing to help design a tailored treatment plan is commonly called personalized medicine and is growing fast.

In particular, it is becoming more common in melanoma trials to collect information on mutation status of a tumour. Common mutations seen in melanoma involve BRAF which is involved in the cell signalling pathway, however expression of NRAS and c-kit mutations are also looked for as sometimes it is possible to target these mutations with specific drugs.


Interim Analysis

When developing a clinical trial it is important to consider an appropriate time at which to perform an interim analysis, i.e. a point in the trial at which the data will be reviewed to ensure that recruitment is sufficient, that the trial is safe and that the results do not yet carry favour with one experimental arm or another.

In the case of testing a new drug with potential hazardous complications it is necessary to monitor the treatment arm periodically so as to ensure that the complication or mortality rate does not exceed that of the control or gold standard treatment group. It must be decided if this will be a test for difference, equivalence, or non-inferiority between the two or more treatment arms. Also be aware of what the standard clinical hazardous complications rate is just in case there has been a spike in all treatment arms for some unknown reason. Should an increased hazard rate be detected then the study is either terminated or suspended. As well as the overall statistical investigation, each incident should also be investigated in itself.