Real-World Studies & Randomized Controlled Trials: Strengths & Limitations of These Complementary Designs

00:00:00: You are listening to an ADIS Journal podcast.

00:00:12: Hello and welcome to part one of three of this podcast series on practical approaches to understanding real-world evidence for oncology and therapy.

00:00:20: My name is Adam Browski and I am a professor of medicine at the University of Pittsburgh, co-director of the Comprehensive Cancer Therapeutics Program of the UPMC Hillman Cancer Center in Pittsburgh, Pennsylvania.

00:00:31: Thank you, Chris, for joining me in this important discussion of real-world evidence and how it can inform everyday clinical practice.

00:00:38: Thank you, Adam.

00:00:39: It's great to be here.

00:00:41: My name is Christos Vaclavas and I'm an associate professor practicing breast medical oncologist and the physician leader in breast cancer for the Hansman Cancer Institute of the University of Utah.

00:00:51: Today, we will be discussing the strengths and limitations of randomized controlled trials, or RCTs, as well as studies using real-world data.

00:00:58: Of the two, our audience is likely to be more familiar with RCTs, which remain the gold standard for evaluating the efficacy and safety of a drug, medical device, or any therapeutic intervention.

00:01:09: But as we will explore, real-world studies have emerged in recent years as an important source of complementary information that can support decision-making in the clinic and beyond.

00:01:18: To provide context for understanding real-world studies, we will first review some key aspects of RCTs.

00:01:25: Adam, given that you are a member of the University of Pittsburgh community, you are uniquely positioned to provide some background on the origins of the modern clinical trial.

00:01:35: Perhaps you can provide some brief history on this important step in the evolution of evidence-based medicine.

00:01:41: Modern clinical trials, such as RCTs, especially in breast cancer, were first pioneered in the nineteen sixties by Dr.

00:01:47: Bernie Fisher and his colleagues at the University of Pittsburgh.

00:01:50: He noted that most breast cancer therapies at the time were primarily based on expert opinion case studies and basic retrospective analyses.

00:01:58: To introduce more scientific rigor, he started advocating for randomized prospective multi-centered trials which led to the development of the modern RCT.

00:02:07: He went on to use his newly developed studies designed to successfully demonstrate the efficacy of tamoxifen for the treatment of patients with breast cancer.

00:02:14: We of course now understand that for RCTs to demonstrate causality between a treatment and an outcome, patients must be randomly assigned to each arm.

00:02:22: and they often stratify by certain demographics and disease characteristics to ensure balance between arms.

00:02:28: Results from these RCTs form the basis for treatment guidelines and standard of care that are in use today and have improved the outcomes for patients worldwide.

00:02:37: That is a great perspective.

00:02:39: Over the past fifty years, we have made advances not just in our understanding of disease biology but also in how we go about asking research questions.

00:02:49: In that spirit, it is important to recognize that clinical trials have a number of important limitations.

00:02:54: First, in order to reduce variability and increase confidence in the findings, clinical trials incorporate strict inclusion and exclusion criteria for participation.

00:03:04: As a result, clinical trial participants are generally younger and healthier than those seen in routine clinical practice.

00:03:11: For example, patients with comorbidities and those taking concomitant medications are often excluded.

00:03:17: Other patient groups that may be excluded are pregnant women, those with prior or concurrent malignancies, and those with brain metastasis.

00:03:25: Secondly, certain patient groups are often underrepresented in clinical trials, such as socially disadvantaged patients and certain racial and ethnic groups.

00:03:35: Finally, for cancers and other diseases that are rare and geographically diffuse or uncommon in one sex or the other, RCTs are often not practically feasible.

00:03:45: We should point out that efforts are ongoing to improve the diversity of recruitment in clinical trials, but those limitations you just mentioned are valid and may limit the utility of RCT findings for guiding treatment of patients excluded from or underrepresented in these trials.

00:04:00: In fact, I know I've had patients whose clinical characterizes do not match those typically senior RCTs, and I am sure that is the case for you too, Chris.

00:04:09: You are absolutely right, Adam.

00:04:11: Fortunately, data from real-world studies can fill knowledge gaps that are difficult to address via clinical trials.

00:04:17: Real-world data is defined as the medical information routinely collected outside of the confines of a clinical trial, such as data found in medical records or insurance claims databases.

00:04:29: In contrast to the small, single-site studies of the past, more than real-world databases and registries can contain data from millions of patients in the geographically diverse regions.

00:04:40: As such, these sources capture more diverse patient populations including those who are often excluded from or underrepresented in clinical trials.

00:04:49: Rural data collection is also continuous and may capture data over long periods of time.

00:04:55: Analysis may therefore reveal temporal trends in outcomes, safety profiles and treatment patterns.

00:05:01: With large databases, it may be possible to capture rare outcomes or events.

00:05:06: real-world studies may be particularly well suited to studying rare diseases, for which RCTs may not be practical.

00:05:12: with the COVID that data quality and consistency in limited sample size may limit conclusions from such analysis.

00:05:19: Adam, maybe you can share an example from your research that has leveraged real-world data.

00:05:25: Sure.

00:05:26: For diseases with complex treatment landscapes, real-world studies can shed light on treatment patterns and their impact on clinical outcomes.

00:05:33: We did this a few years ago using a large population of nine hundred and seventy-seven patients with her two positive metastatic breast cancer to describe treatment patterns in routine clinical practice.

00:05:43: When these patients were further characterized by clinical receptor status, we identified a subpopulation of patients with low tumor hormone receptor positivity that had relatively poor survival outcomes, potentially due to undertreatment with endocrine therapy.

00:05:57: This example also highlights another strength of real-world data.

00:06:01: It allows for the assessment of how individual therapies are being prescribed to specific groups.

00:06:06: Our listeners should also be aware of a range of limitations inherent in real-world studies.

00:06:12: You're right, Chris.

00:06:13: One key limitation is that in real-world data sets, patients have not been randomly assigned to treatment groups, possibly resulting in treatment selection bias.

00:06:22: An example of this bias would be a situation where elderly patients were typically given a less effective therapy with fewer side effects, while younger patients were given a more effective therapy but with more side effects.

00:06:33: This example illustrates why causality cannot be established between treatment and outcome in real-world studies, because factors other than treatment such as patient age may be influencing the outcome.

00:06:44: In contrast to the term efficacy, which implies causality in RCTs, the term effectiveness is used to describe the association between treatment and clinical outcome in real-world studies.

00:06:55: We should not hear that there are validated statistical methods for mitigating treatment selection bias in real-world studies and those will be covered in the next part of this series.

00:07:05: Another limitation of real-world studies is that endpoints may rely on the treating physician's judgment and not on standardized criteria, as would be used in an RCT.

00:07:15: There might be more variability in treatment adherence and follow-up than in RCTs.

00:07:20: Finally, there's the potential for missing or erroneous data in real-world databases.

00:07:25: Overall, RCTs and real-world studies pursue the same objective, to understand which therapies work best for patients through evidence-based approaches.

00:07:35: Both types of studies have evolved beyond expert opinion, case studies, and small single-institution observations and have evolved into modern, rigorous inquiries that can guide both clinicians and regulatory bodies.

00:07:48: That said, Due to the differences in strengths and limitations of RCTs and airworld studies, it is important to exercise caution in directly comparing the results of these two study types.

00:07:58: RCTs remain essential for establishing the efficacy of new treatments, while airworld evidence reflects their effectiveness and plays a complementary role.

00:08:08: All great points, and as you just mentioned, regulatory agencies in the United States and around the world are increasingly looking to rear-world evidence to supplement knowledge gained from RCTs.

00:08:20: The US Food and Drug Administration has recently launched a rear-world evidence initiative that notes the history of rear-world data in monitoring post-market safety of approved drugs and recognizes recent advances in the generation of robust rear-world evidence that can support regulatory decisions.

00:08:37: Similarly, the European Medicines Agency is working to integrate rear-world evidence into regulatory decision-making.

00:08:44: Importantly, guidance for best practices in real-world study design has been shared by various groups, such as the ISPE, ISPOR Joint Task Force, and real-world evidence reporting guidelines have been provided by ESMO's guidance for reporting oncology real-world evidence, also known as ESMOGROW.

00:09:04: Looking ahead, I think we will continue to see innovation in trial design, such as hybrid trials that leverage both real-world data and randomization to evaluate new therapies in an efficient and cost-effective manner.

00:09:16: I am excited to watch developments in this space and will continue to use real-world evidence in support of treatment recommendations from our patients.

00:09:24: Thanks again for joining me today, Chris, for this important discussion on the relationship between clinical trials and real-world studies in the modern medical research landscape.

00:09:33: We hope this has helped our audience gain familiarity with real-world studies and an appreciation for how real-world evidence can fill in knowledge gaps in support of treatment decision-making.

00:09:44: I also hope our listeners will join us for part two of this series where we'll explore methodologies and statistical approaches commonly used in real-world studies.

00:10:15: For a full list of declarations, including funding and author disclosure statements and copyright information, please visit the article page on the journal website.

00:10:26: The link to the article page can be found in the podcast description.