Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that allows research into pragmatic trials. It shares clean trial data and ratings using PRECIS-2, allowing for multiple and diverse meta-epidemiological research studies to evaluate the effect of treatment on trials that have different levels of pragmatism, as well as other design features.
Background
Pragmatic studies are increasingly recognized as providing real-world evidence for clinical decision making. The term "pragmatic" however, is not used in a consistent manner and its definition and measurement need further clarification. Pragmatic trials should be designed to guide clinical practice and policy decisions, rather than confirm a physiological or clinical hypothesis. A pragmatic trial should aim to be as similar to real-world clinical practice as possible, such as the recruitment of participants, setting and design of the intervention, its delivery and implementation of the intervention, as well as the determination and analysis of the outcomes, and primary analysis. This is a major distinction between explanation-based trials, as described by Schwartz & Lellouch1, which are designed to confirm the hypothesis in a more thorough way.
Truly pragmatic trials should not blind participants or the clinicians. This can result in bias in the estimations of the effects of treatment. Practical trials should also aim to enroll patients from a wide range of health care settings so that their results can be applied to the real world.
Additionally, pragmatic trials should focus on outcomes that are vital for patients, such as quality of life or functional recovery. This is particularly relevant when it comes to trials that involve the use of invasive procedures or potentially serious adverse events. The CRASH trial29, for example focused on the functional outcome to evaluate a two-page case report with an electronic system for the monitoring of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 utilized urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these characteristics pragmatic trials should reduce the trial's procedures and requirements for data collection to reduce costs. Additionally, pragmatic trials should seek to make their findings as applicable to real-world clinical practice as is possible by making sure that their primary analysis follows the intention-to treat approach (as described in CONSORT extensions for pragmatic trials).
Despite these requirements, a number of RCTs with features that defy the notion of pragmatism were incorrectly labeled pragmatic and published in journals of all kinds. This could lead to misleading claims of pragmaticity, and the usage of the term needs to be standardized. The creation of the PRECIS-2 tool, which offers an objective and standard assessment of pragmatic characteristics, is a good first step.
Methods
In a pragmatic study the aim is to inform clinical or policy decisions by demonstrating how an intervention can be integrated into routine care in real-world contexts. from this source test hypotheses about the cause-effect relation within idealized conditions. In this way, pragmatic trials may have lower internal validity than explanation studies and be more prone to biases in their design as well as analysis and conduct. Despite their limitations, pragmatic studies can provide valuable data for making decisions within the healthcare context.
The PRECIS-2 tool scores an RCT on 9 domains, ranging between 1 and 5 (very pragmatic). In this study, the domains of recruitment, organisation and flexibility in delivery, flexible adherence and follow-up scored high. However, the principal outcome and the method for missing data were scored below the practical limit. This suggests that it is possible to design a trial that has good pragmatic features without harming the quality of the results.
It is difficult to determine the amount of pragmatism within a specific trial since pragmatism doesn't have a binary characteristic. Certain aspects of a study may be more pragmatic than others. Moreover, protocol or logistic modifications during the course of a trial can change its pragmatism score. Koppenaal and colleagues discovered that 36% of 89 pragmatic studies were placebo-controlled, or conducted prior to the licensing. Most were also single-center. Therefore, they aren't quite as typical and can only be described as pragmatic if their sponsors are tolerant of the lack of blinding in these trials.
A common feature of pragmatic research is that researchers try to make their findings more meaningful by analyzing subgroups within the trial sample. This can lead to imbalanced analyses and less statistical power. This increases the chance of missing or misdetecting differences in the primary outcomes. This was the case in the meta-analysis of pragmatic trials as secondary outcomes were not corrected for covariates' differences at the time of baseline.
Additionally practical trials can have challenges with respect to the gathering and interpretation of safety data. It is because adverse events are typically self-reported and are susceptible to errors, delays or coding errors. It is essential to increase the accuracy and quality of the outcomes in these trials.
Results
While the definition of pragmatism does not require that all clinical trials are 100% pragmatic there are benefits to including pragmatic components in trials. These include:
By incorporating routine patients, the results of trials are more easily translated into clinical practice. But pragmatic trials can have their disadvantages. The right kind of heterogeneity, like, can help a study generalise its findings to many different settings or patients. However the wrong type of heterogeneity could decrease the sensitivity of the test, and therefore decrease the ability of a study to detect even minor effects of treatment.
A variety of studies have attempted to categorize pragmatic trials, with various definitions and scoring systems. Schwartz and Lellouch1 created a framework to distinguish between research studies that prove a clinical or physiological hypothesis, and pragmatic trials that aid in the selection of appropriate therapies in the real-world clinical setting. The framework was composed of nine domains that were evaluated on a scale of 1-5 with 1 being more lucid while 5 was more practical. The domains covered recruitment, setting up, delivery of intervention, flexible adhering to the program and primary analysis.
The original PRECIS tool3 was built on the same scale and domains. Koppenaal et. al10 devised an adaptation of the assessment, known as the Pragmascope which was more user-friendly to use for systematic reviews. They found that pragmatic systematic reviews had a higher average score in most domains but lower scores in the primary analysis domain.
The difference in the primary analysis domains can be explained by the way that most pragmatic trials analyze data. Some explanatory trials, however, do not. The overall score for systematic reviews that were pragmatic was lower when the domains of organisation, flexible delivery and following-up were combined.
It is important to remember that a study that is pragmatic does not mean a low-quality trial. In fact, there are increasing numbers of clinical trials that employ the term 'pragmatic' either in their abstracts or titles (as defined by MEDLINE, but that is not precise nor sensitive). The use of these words in abstracts and titles could indicate a greater understanding of the importance of pragmatism however, it is not clear if this is reflected in the contents of the articles.
Conclusions
As the importance of real-world evidence becomes increasingly popular the pragmatic trial has gained traction in research. They are clinical trials randomized that evaluate real-world alternatives to care instead of experimental treatments in development. They include populations of patients that are more similar to the ones who are treated in routine care, they employ comparators that are used in routine practice (e.g. existing drugs), and they rely on participant self-report of outcomes. This approach can overcome the limitations of observational research such as the biases associated with the reliance on volunteers, and the limited availability and codes that vary in national registers.
Pragmatic trials have other advantages, including the ability to use existing data sources, and a greater chance of detecting significant differences from traditional trials. However, these trials could be prone to limitations that compromise their validity and generalizability. The participation rates in certain trials could be lower than anticipated due to the health-promoting effect, financial incentives, or competition from other research studies. A lot of pragmatic trials are limited by the need to recruit participants on time. Practical trials aren't always equipped with controls to ensure that observed differences aren't caused by biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatic and that were published up to 2022. They assessed pragmatism by using the PRECIS-2 tool, which consists of the eligibility criteria for domains as well as recruitment, flexibility in adherence to intervention and follow-up. They discovered 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or above) in at least one of these domains.
Trials that have a high pragmatism score tend to have more expansive eligibility criteria than traditional RCTs which have very specific criteria that aren't likely to be used in clinical practice, and they include populations from a wide range of hospitals. These characteristics, according to the authors, can make pragmatic trials more useful and relevant to everyday clinical. However, they don't ensure that a study is free of bias. The pragmatism characteristic is not a definite characteristic and a test that does not possess all the characteristics of an explanatory study could still yield reliable and beneficial results.