Greater Rigor Is Needed To Enable Real-World Evidence To Reach Its Full Potential
By Dan Riskin
Biopharmaceutical manufacturers are increasingly incorporating real-world evidence (RWE) into their regulatory strategies. The collection and use of data from diverse sources — and the use of innovative analytical tools and modeling methodologies to glean important insights from such data — have created unprecedented opportunity for stakeholders to better understand how medications are being used and are performing under messy real-world conditions. Today, the calls for increased regulatory guidance and more harmonized industry agreement on how real-world data (RWD) should be gathered and analyzed continue to grow.
Efforts to impose a higher level of rigor and scrutiny should be welcomed by all stakeholders. The goal is to enable “regulatory-grade” RWE to be developed, so that the trends or conclusions developed by analyzing RWD can be viewed as being both scientifically valid and statistically relevant.
Specifically, regulatory-grade RWE is defined as being sufficiently valid to support clinical assertions in terms of the accuracy of the data and population generalizability. The goal is to produce trustworthy RWE that can inform and support regulatory decision making, including those related to label revisions. In addition to using RWE to create reliable, regulatory-grade clinical assertions, it also can be used to inform and support drug-pricing and reimbursement negotiations.
Today, all stakeholders should hold regulatory-grade RWE to the highest possible standards, in terms of data validity. Too often, different sources of RWD related to medication use and other healthcare interventions — including EHR, insurance claims, pharmacy dispensing records, and patient registries — are simply assumed to be sufficient. In reality, disparate sources of data were developed in parallel, with little overarching consistency from source to source in the absence of rigorous, unifying frameworks or protocols. This results in tremendous heterogeneity, inconsistency, and incompleteness in the data.
HOLDING RWD AND RWE TO HIGHER STANDARDS
It is incumbent upon all stakeholders in life sciences to recognize and understand the strengths and limitations of today’s complex sources of RWD and RWE. Stakeholders must measure accuracy in terms of recall and precision as well as work to improve the quality and accuracy of data sources, so that they are sufficient for the clinical assertion and to develop more consistent and rigorous frameworks and protocols. Accuracy measurement requires evaluating not only whether a patient included in a cohort should be (precision), but also whether patients that should be included in a cohort are (recall).
As part of the 21st Century Cures Act, the FDA will be publishing guidance to help improve the consistency and rigor of key healthcare-related data sources, improve methods for information extraction, create protocols for consistent cohort definition, and propose frameworks to improve overall study design. Increased scrutiny and the development and application of more-consistent protocols should be viewed as a positive step forward for all life sciences stakeholders, as such efforts will ultimately ensure that RWE used to support regulatory and reimbursement decision making has been held to the highest standards of scientific validity and statistical relevance.
The development of regulatory frameworks and protocols aimed at minimizing data variability and maximizing data quality should provide greater confidence for all, in terms of the veracity and value of the analytical insights revealed by the RWE. As the FDA receives increasing submissions containing RWE, it will continue developing visibility into high-quality versus low-quality submissions, and such experience will ensure the most credible assertions using the safest approach.
DAN RISKIN, M.D., MBA is founder and CEO of Verantos, an RWE-specialized firm that runs regulatory-grade EHR-based studies.