Introduction to Sample Size in Clinical Trials

Sample size definition

In research, sample size (typically denoted as n) refers to the number of units of investigation involved in a study. In a research study aiming to reveal the average number of trees across 30 major cities, the sample size (n) is 30 (cities). To calculate the average age of all residents of a building that houses 1,500 people, the sample size would be 1,500 (people).

What does sample size mean in clinical trials?

In clinical research, sample size refers to the number of participants enrolled in a study. Sample size calculation is one of the central components of the design of clinical trials, and has a major influence on the study and its results. Sample size must be chosen in consideration of a few different factors, which we will discuss in this article, along with why sample size is important and how it can be calculated.

How does sample size affect clinical trials?

An adequate sample size is crucial for generating reliable and clinically significant findings in clinical trials. Larger sample sizes lead to improved precision by reducing sampling variability, as well as enhanced generalizability, while also enabling the detection of smaller yet potentially meaningful effects between groups. However, sample size can be limited by the number of people who have a given condition, who meet the eligibility criteria, or who live within range of the study sites for traditional site-based trials. Each additional participant represents increased resource requirements, and thus the optimal sample size is large enough to impart sufficient statistical power without wasting resources or exposing an unnecessarily large number of people to the potential risks of a study.

Does sample size affect reliability?

Sample size greatly impacts the reliability of clinical trial results. A larger sample size generally allows for the generation of more reliable and accurate findings. With a larger sample, the distribution of participants is more likely to represent the target population as a whole, reducing the impact of random variation and increasing the generalizability – or external validity – of the study's conclusions.

What are the disadvantages of small sample size in clinical trials?

Utilizing a sample size that is too small increases the risk of random error or sampling variability, and can make it difficult to determine whether any observed differences are statistically significant or simply due to chance. This situation can lead to results that may not be reliable outside of the specific group studied.

Does sample size affect clinical significance?

Sample size also influences the determination of clinical significance – whether the observed effects have potential real-world implications for patient care. Larger samples provide greater statistical power (more on this below), which allows researchers to detect smaller but meaningful differences between study groups. A large sample size promotes confidence in determining whether or not a treatment has substantive benefits over alternatives.

With smaller samples, even substantial treatment effects may not reach statistical significance due to limited power. It's important to note that while statistical significance is essential for interpreting the results of any research study, it doesn't necessarily mean that a result is of clinical importance. That’s because clinical significance depends on additional factors beyond just sample size. These include effect size (the magnitude of treatment benefit), considerations such as side effects and costs associated with a given intervention, and expected treatment outcomes from the perspective of both patients and healthcare professionals.

What is a good sample size for a study?

When it comes to selecting a sample size that’s appropriate for a study, many factors come into play. We will first explore these before providing a more detailed description of the sample size calculation clinical trial step.

Factors that play into selecting sample size for a clinical trial

Variability

The variability of a sample describes the extent of difference (or range) of a given variable or characteristic in the study population. Part of the reason for eligibility criteria is to limit extreme variability in the study population. However, capturing a certain level of variability is important for achieving results that are generalizable to the broader population, i.e., for achieving a representative sample.

If a study population has very high variability in a potential confounding variable, say for example including adults aged 19 through 85, then a larger sample size will be needed to detect a difference between study treatments. If a more homogenous population is enrolled (in this example, let’s say only including people aged between 30 and 35), the smaller variability in the population means that it might be possible to uncover a statistically meaningful difference with a smaller sample size. It’s not always straightforward to know what factors may confound or influence study outcomes, and thus it’s not always possible to pinpoint sample variability in a useful way. However, it’s important to consider variability in factors that are known or have a potential to impact study endpoints/outcome measures, so that they can be treated appropriately during randomization and in data analysis.

Type 1 error and level of significance (P value)

Type I errors, also known as false positives, occur when a treatment effect is identified when in reality there was no effect. The probability of type I error occurring is also known as the level of significance, or the P value, and is denoted by ⍺. The significance level is typically set to 5%, which means that it’s determined to be acceptable to have a 5% chance of incorrectly rejecting the null hypothesis – i.e., concluding that there is a treatment effect when there was actually no difference between groups.

Type 2 error and statistical power

ANother factor is the probability of type II error, also known as a false negative, and is usually expressed as ꞵ. Type II errors occur when it is concluded that there was no treatment effect, despite there actually being one. The term “power” refers to the statistical power of a study design to reveal a treatment effect, and is calculated as 1-ꞵ. As sample size increases, the probability of type II error decreases and the statistical power of the study increases. Thus, increasing sample size is a way to increase the power of a study. Power is generally set to either 80% or 90% when calculating sample size, which means that there is a 20% (or 10%, respectively) chance of failing to reject the null hypothesis even though it is false (i.e., determining that there was no treatment effect when there actually was).

Effect size

Effect size refers to the minimal detectable difference between study arms/treatments which would indicate that a novel treatment is safer and/or more effective than an existing treatment or in comparison to placebo or another control. The minimal detectable difference is usually expressed in a percentage, and is determined based on prior knowledge about treatment efficacy, clinical experience, or other factors. For example, if there is already a fairly effective treatment available for a given condition, then it might be necessary to demonstrate that a new treatment is 30% more effective than the standard – an effect size of 30%. For a life-threatening condition with no known treatments, a novel drug that improves quality of life even by 10% relative to control (no treatment) may represent a huge advance. In that case, demonstrating an effect size of 10% might be deemed clinically significant.

How do you determine sample size in clinical research?

Q: “What is the appropriate sample size for a clinical trial?”

A: It depends on multiple factors!

A full answer to this question requires a few pieces of information about the study design, hypothesis, and the condition being studied and the population affected by it. For example, a randomized clinical trial sample size calculation for a diabetes lifestyle intervention study will differ from a controlled trial sample size calculation for depression medication.

The simplest way to begin determining sample size for a clinical trial is to determine the factors described above, in consideration of the specific research hypothesis (hypotheses), and the available information about the population of interest, and then use a clinical trial sample size calculator or a basic sample size formula. There are some great resources and tools available online from reputable clinical research institutes, which we link to below:

Clinical trial sample size calculator and sample size formula links:

Sample-size.net - Various calculators and useful resources for understanding sample size calculations, provided by the UCSF Clinical & Translational Science Institute

Sample size calculator - Great resource by the Cleveland Clinic with Clinical research sample size estimation for various study design types

Sample Size Calculators by the Massachusetts General Hospital (MGH), a teaching hospital of Harvard Medical School

ClinCalc Sample Size Calculator

Raosoft sample size calculator

The first three of these resources also provide a somewhat deeper dive into the statistical theory behind sample size calculations and the factors that play into it. For those who are interested, understanding the theoretical background can help you become more comfortable with performing sample size calculations in clinical trials. If you are not working with an experienced statistician, it’s important to dedicate time to calculating sample size, as the success of the study depends on it.

Conclusion

Calculating sample size is a complex yet vital part of designing a successful clinical trial that balances limiting resource use and patient exposure with sufficient statistical power and scientific reliability. This article is meant to serve as a general introduction to the factors that play into clinical trial sample size calculations, to support you in your journey of optimizing clinical trial design. Take a look at the resources linked in the final section of the article for sample size calculators for various clinical trial types and for more detail about the statistical basis for sample size calculation.