The unique position of pragmatic trials in clinical research

Randomized controlled trials (RCTs) are largely considered the “gold standard” for robust clinical research studies. However, RCTs imply significant costs and often take years to complete. Further, the results produced are, unfortunately, often difficult to implement in real-world practice. In fact, it was found that after an average of 17 years, merely 14% of research findings and medical discoveries will have resulted in widespread changes in patient care.1

Due in part to the strict eligibility criteria and controlled settings characteristic of RCTs, results from such studies do not translate smoothly into clinical practice or policy-making decisions, which involve broader and more varied populations in uncontrolled settings. Even for novel interventions that do get adopted in regular clinical practice, a dramatic decrease in its effectiveness is often noted under real-world conditions.2 Moreover, systematic reviews consistently note that there is not sufficient practical evidence from such trials to effectively inform the healthcare decisions that patients and healthcare providers must make; essentially, we need more practice-based evidence for more evidence-based practice.1

That's where pragmatic trials enter the picture. Pragmatic clinical trials effectively address the disconnect between clinical research and real-world settings by assessing interventions under real-world conditions. Pragmatic trials might even test the same intervention as explanatory or randomized controlled trials, but are conducted in real-world clinical practice settings.2

Pragmatic trials aim to include diverse, representative patient populations and are conducted by qualified clinicians, who may not necessarily have a research background. The results of pragmatic trials are, in theory, more applicable to healthcare and policy decisions.

What makes a clinical trial pragmatic?

A pragmatic trial aims to determine an intervention's effectiveness under real-world settings, i.e., in routine clinical practice, and are conducted from everyday care settings such as hospitals and community clinics.3,4 Pragmatic trials are designed to provide real-world evidence that supports policy-making and practical improvements in clinical practice.

Pragmatic trials involve collaboration between patients, practitioners, communities, and health systems at large. They often involve the use of electronic health records (EHRs) to support cost-effective and efficient recruitment, data collection, and trial monitoring and follow-up.1 The participants enrolled in a pragmatic trial are similar to - and as diverse as - the population receiving the intervention in real-world practice. A primary goal of pragmatic trials is to maximize the likelihood that the tested interventions can be practically rolled out in clinical practice if results are promising.3

What is the difference between pragmatic trials and explanatory trials (efficacy trials)?

Explanatory trials focus on evaluating the efficacy of an intervention under ideal (controlled or semi-controlled) conditions, whereas a pragmatic trial evaluates its effectiveness in routine clinical practice, i.e., non-controlled settings.2,5

Often, positive results from explanatory studies turn out to be less effective in real-world settings because of several confounding factors that are prevalent in routine clinical practice but which are excluded from or controlled for under the idealized settings of explanatory trials.

Explanatory trials typically involve strict recruitment criteria, narrowing down the study population and making it difficult for elderly people or those with certain risks or conditions to enroll. While this allows for more precise results by achieving some level of homogeneity in the study sample (controlling for extraneous factors), it tends to make the results less applicable to the broader population. On the other hand, pragmatic trials tend to define more relaxed eligibility criteria and accept participants from diverse populations, thus testing the intervention on a sample that is more representative of the subset of patients who may need the intervention in the real world.2

What is an example of a pragmatic trial?

The SIREN (SARS-CoV-2 Immunity and Reinfection Evaluation) study, which provided evidence of the efficacy of COVID-19 vaccines in preventing recurring infection, was a pragmatic trial. It involved the recruitment of more than 40,000 diverse healthcare workers in their regular work environments.6 It involved an extended follow-up period to monitor the long-term efficacy of the vaccines, the results of which were eventually used to widen the booster vaccination campaign.7

The SIREN study presented real-world evidence (RWE) to support the efficacy of the COVID-19 vaccines and boosters. The data it produced has since been valuable in informing healthcare policy decisions and vaccination strategies.

What are the advantages of pragmatic clinical trials?

Pragmatic research is well-suited for studies that tackle questions relating to how best to get new health interventions out into regular practice. Pragmatic clinical trials can also provide insights into how a given intervention works differently in various healthcare settings.2

In general, pragmatic trials offer a multitude of other benefits, such as the following:1

Practicality

Pragmatic clinical trials are designed to gain insights into how various interventions work in everyday care settings. By studying treatments under conditions that closely reflect those under which they will eventually be used, pragmatic trials provide practical evidence of effectiveness that can be incorporated directly into policy and healthcare decisions.

Inclusivity

Pragmatic clinical trials are generally inclusive, with diverse trial populations providing the context for robust real-world evidence of the effectiveness of an intervention in the general population. As opposed to controlled explanatory trials, pragmatic trials can often include patients with complex medical histories and comorbidities, including those that are most in need of treatments and interventions.

Engagement and relevance

Pragmatic trials engage various stakeholders throughout the health system, such as healthcare providers, hospitals and clinics, patients, and policymakers. The increased collaboration and engagement of diverse actors relative to explanatory trials promotes the formulation of research questions that are most relevant to day-to-day patient care. The results of pragmatic trials are designed to be relevant for informing the decision-making undertaken by healthcare policymakers, administrators, healthcare providers, and patients. By demonstrating the effectiveness of treatments rooted in real-world evidence, pragmatic trials provide data that can lead to practical improvements in healthcare.

Challenges and disadvantages of pragmatic research studies

Pragmatic clinical trials can be more costly than explanatory trials due to the larger samples, and may sometimes require more complex study designs. Such designs are also not suited for early-phase, exploratory studies that seek to understand if an experimental intervention causes a biological effect.3

Below we provide an overview of potential challenges associated with pragmatic clinical trials, along with potential ways to overcome them:

Study participant recruitment

Clinical trial participation has declined over the years. For instance, amongst people without established disease, less than 10% respond to a screening invitation.4 In order for the study sample to appropriately represent the subset of patients who might receive the intervention in routine care, large sample sizes are needed.

We have published various articles on the art of recruitment, for example relating to enrollment statistics and recruitment techniques, recruitment via social media, common recruitment challenges to be aware of, tools to assist with recruitment, and general tips for recruitment strategies regardless of the specific method used. As a brief synthesis, potential ways to enroll more participants include:

• Offering financial compensation
• Avoiding restrictive eligibility criteria
• Designing simpler, patient-centric study protocols
• Minimizing study visits through the use of remote technologies and decentralized designs.

For pragmatic trials, the study should be designed to directly address a need that is evident amongst the population with the condition being studied or a gap in healthcare in general - if the trial concerns a relevant issue, it’s more likely that people will understand its importance and be willing to participate.

Another option is to employ designs such as large simple trials, or (when it is ethically sound to do so) cluster-randomized trials (those randomized above the individual level, for example by clinic or even city/region). Studies with such unique designs lend themselves to larger sample sizes without the degree of complexity that would be demanded by an RCT with the same sample size.

Investigator recruitment

Many healthcare professionals outside of research and academic centers don't participate as investigators in clinical trials due to time constraints, lack of awareness, or a lack of research experience or familiarity with applicable regulations. In order to capture the heterogeneity amongst healthcare providers, a diversity of investigators should also be sought after for pragmatic trials.

Efforts should be made to include healthcare professionals from a variety of hospitals and clinics, and specialists from various backgrounds. Trials run by a central authority can also achieve higher investigator participation rates; for instance, a hospital could insist that all of its wards participate in a given trial. Providing incentives to investigators for participation may also help, as most healthcare providers already face demanding jobs and may be reluctant to assume additional responsibilities.

Selection of endpoints

The endpoints chosen in pragmatic trials should be practical and should matter to patients. Further, they should be relatively easy to record - for example, major concepts such as disability, overall symptoms, and quality of life. This facilitates simpler trial designs and lower burden for participants as they can input less data and with a lower frequency. It’s also easier to collect such data remotely, i.e., through online surveys, eliminating the need for study visits. However, the nature of self-reporting requires additional considerations, as there can be unrealistic levels of variability in responses if forms are not well-designed with constraints and thorough instructions.

Trial follow-up

Unified electronic healthcare records serve as a great source of data for follow-up in trials, potentially reducing the follow-up burden placed on participants; unfortunately, such records are not available in many countries.4

An attractive alternative to using electronic health records is to enroll patients who are already enrolled in intervention- or diseases-specific registries. Such registries contain detailed data about patients’ long-term follow-up and distinct disease characteristics, and may represent a cost-effective and efficient approach for conducting pragmatic trials.

Clinical trial design: The randomized-pragmatic trials continuum

Pragmatic and explanatory or randomized controlled trials are not entirely separate approaches; most trials contain elements of both, and the two approaches can be seen as existing on a continuum. The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) tool is a useful method for assessing the degree of pragmatism when designing a clinical trial.3

Leaving the nuances aside, the overlapping and non-exclusive nature of these designs tells us that adopting one design in some aspects of the trial does not mean entirely abandoning the other; both RCTs and pragmatic clinical trials have led to valuable medical breakthroughs and advances in healthcare.1

Pragmatic clinical trials and real-world evidence (RWE)

Real-world data, or RWD, refers to patient data that is collected via the delivery of standard care in real-world settings. RWD can come from:

• Electronic health records (EHRs)
• Case report forms (CRFs)
• Medical claims and bills
• Government databases
• Patient-generated data

Real-world evidence, or RWE, refers to the clinical evidence and insights generated from such data. RWE can include information on the use, therapeutic effectiveness, risks, and side effects of an intervention. RWD and RWE can also lead to the suggestion of new therapeutic indications for existing/approved treatments.

Pragmatic trials help generate real-world evidence due to their resemblance to real-life medical practice. Certain pragmatic trial designs represent an opportunity for comprehensive health outcome conclusions to be elucidated with relatively lower resource use as compared to controlled trials.8

Since they have relatively relaxed inclusion criteria, they can use vast RWD to produce effective RWE for swift and successful intervention implementation.

Pragmatic trials: History and future

Pragmatic trials have a history of successes, beginning with the polio vaccine studies in the 1950s and early studies on aspirin as an acute treatment for heart attack in the 1980s and 1990s.1

Pragmatic trials have since evolved over time, driven by the need for practical research results that can be implemented successfully in real-world clinical practice. As discussed above, pragmatic trials played an important role in solidifying the effectiveness of COVID-19 vaccines. Pragmatic clinical trials have been conducted in broad fields of healthcare, including mental health, cardiovascular disease, and cancer.

The future of pragmatic trials seems bright, thanks to the continuing advancement of AI-driven technology, big data, and analytics, as well as the increasing attention given to the importance of RWD and RWE. Electronic health records and real-world data have helped sponsors of pragmatic trials recruit diverse participants in developed countries. Technological advancements have enabled various concepts that open the doors to new possibilities for clinical trials, such as:

• Real-time monitoring
• Precision medicine
• Adaptive trial designs
• Integration of real-world evidence

For example, precision medicine uses AI and ML to identify patient subgroups that may respond differently to certain interventions. Researchers can also use large datasets from varying sources to identify suitable trial participants. However, there are challenges associated with the use of these new technologies that are important to address in order to maintain high standards for patient rights and scientific integrity:

• Upholding data quality
• Mitigating biases
• Addressing privacy concerns
• Adapting regulatory frameworks and standards

Conclusion

Pragmatic clinical trials are essential to the development and implementation of new healthcare interventions based on robust and practical evidence. Pragmatic trials aim to answer relevant questions about the effectiveness of different therapies in truly representative populations, and to overcome limitations of controlled trials related to the controlled conditions which do not reflect those of real-world practice. Although they come with their own unique considerations and challenges, pragmatic trials seem poised to play an increasingly important role in guiding healthcare policy and decision-making alongside the increased use of RWE/RWD and advances such as big data and AI that allow for much larger data sets to be analyzed than ever before.

References

[1] https://dcricollab.dcri.duke.edu/sites/NIHKR/KR/Introduction%20to%20pragmatic%20clinical%20trials.pdf

[2] https://www.nia.nih.gov/research/blog/2017/06/pragmatic-clinical-trials-testing-treatments-real-world

[3] https://core.ac.uk/download/pdf/82468026.pdf

[4] https://www.nejm.org/doi/full/10.1056/NEJMra1510059

[5] https://www.qmul.ac.uk/pctu/research/what-is-a-pragmatic-clinical-trial/

[6] https://ukhsa.blog.gov.uk/2021/03/11/the-siren-study-answering-the-big-questions/

[7] https://ukhsa.blog.gov.uk/2022/06/20/the-siren-study-two-years-on/

[8] https://www.withpower.com/guides/from-data-to-insights-real-world-evidence-in-clinical-research