Direct To Data - An Electronic Solution To Patient Diaries
By Wayne Koberstein, Executive Editor, Life Science Leader
Follow Me On Twitter @WayneKoberstein
Regulators have two main requirements for clinical-trials data: accuracy and data integrity from patient to investigator to regulatory review. For patient-reported outcome (PRO) information, paper diaries have long been the standard instruments for recording responses to treatment by the patients themselves.
But following a study published in the British Medical Journal (May 18, 2002), there has been general recognition that paper is a poor protector against patient noncompliance with diary keeping — timely and accurate entry of meaningful information — and that electronic alternatives (ePRO) can improve data accuracy and integrity by comparison. As with any technological solution in drug development, however, the use of so-called eDiaries or ePRO instruments requires especially careful design and engineering, integration with trial protocols and endpoints, and security measures.
FROM PARKING LOT TO ON THE SPOT
The 2002 study focused on patient compliance with the requirements of diary-keeping, comparing the accuracy, timeliness, and completeness of paper and electronic diary entries. Researchers traditionally used paper and pencil, but for decades researchers have grown ever more aware that patients often incorrectly entered information, and they have become more uneasy about the lack of means to verify compliance. When the key measures are patient responses to treatment, such uncertainty becomes an even more acute problem.
The study’s lead author was Dr. Arthur Stone, professor at Stony Brook University and Chairman of the Scientific, Clinical, and Regulatory Advisory Panel at invivodata (now part of ERT). “One of the major issues with paper and pencil diaries had been known for a long time but hadn’t really been quantified before our 2002 paper,” Stone says. “Just how often were people doing it the way they were supposed to do it? Anecdotally, researchers knew people were not highly compliant; there was already a term for it: parking lot compliance.”
At some point, someone on a research team looked out the window and witnessed a patient sitting in a parked car furiously catching up on neglected diary entries before turning it in. In many cases, researchers found that a patient would fill out an entire diary in that way.
Stone’s study used a hybrid of electronic and paper technology to monitor diary entries. One group of patients entered its reports in a paper binder that, unbeknownst to them, was outfitted with a computer chip and a light sensor that recorded when the binder was opened and for how long.
Most patients (79%) routinely fudged their inputs, reporting entries they had not actually made. In comparison, the study gave another patient group a fully electronic diary that prompted, accepted, and confirmed all entries at the required time intervals. Compliance — reported entries matching actual ones — rose to almost 100%.
The study’s revelations came at about the same time clinical research was turning to ecological momentary assessment (EMA), also called experience sampling method (ESM), based on the assumption that proper timing of patient-reported treatment responses elicits the most accurate data. Paper diaries offered no way to verify actual compliance in time. So, clinical researchers began to adopt ePRO instruments over paper as the standard.
Meanwhile, the FDA developed the PRO Guidance, issued as final in 2009, that avoids endorsing any technology but arguably makes ePRO an obvious choice to satisfy the agency’s criteria for patient-recorded outcomes data. The guidance strongly recommends short-time intervals between reports so that patients rely less on memory, and it calls for verification that patients have entered their reports at the proper times.
The agency does state some reservations about ePRO instruments, however — mainly related to security in how researchers “create, modify, maintain, archive, retrieve, or transmit clinical data to the FDA,” as directed in the final rule 21 CFR part 11.7,8. The normal restrictions apply: Lead investigators, not the sponsors or CROs, should have sole access to the raw ePRO data on the developer side, yet FDA investigators must be able to inspect, verify, and copy the data at any study site. Security measures must be sufficient to ensure that data is not altered, prematurely unblinded, or otherwise compromised by bias, data loss, high error rates, or misdirection.
PRO ACHIEVEMENTS
Since the preliminary draft PRO Guidance issued in 2006, the first drug to claim FDA approval of labeling that includes ePRO symptom data was Incyte’s Jakafi (ruxolitinib) for myelofibrosis. One of the two pivotal studies used eDiaries to collect patient reports on six symptoms: night sweats, itching, abdominal discomfort, pain under the ribs on the left side, early satiety, and bone or muscle pain. Based on considerable back-and-forth with the FDA, the trial design used the PRO data to measure a “total symptom score” as the secondary endpoint, with a physical measure (spleen-volume reduction) as the primary endpoint.
Incyte CEO Paul Friedman, speaking at the 2012 JP Morgan Healthcare Conference, credited the symptom data for clinching Jakafi’s FDA approval. “Looking at the change in total symptom score for each individual patient at week 24, most patients receiving Jakafi experience reductions in symptom burden, while the majority of patients receiving no treatment continued to see their symptoms worsen.”
A more recent ePRO-driven approval is for Subsys, a sublingual spray form of fentanyl from specialty pharmaceutical company Insys Therapeutics. In January 2012, the FDA approved Subsys for breakthrough cancer pain. The Phase 3 efficacy trial used eDiaries to collect patient-reported response scores at timed intervals tied to corresponding endpoints. Unlike the Jakafi trial, the Subsys trial used the ePRO tool specifically for the collection of its primary efficacy data.
“We were looking to prove early efficacy,” says Neha Parikh, senior director, clinical operations, at Insys. “The most valuable thing to us in a clinical trial is the data and then being able to use the data we collect. We needed a reliable and validated tool to measure patient response at specific time points. We considered paper diaries, but electronic collection of patient-reported outcomes fit our needs, and the FDA was moving in the same direction.”
The primary endpoint for Subsys was the summed pain-intensity difference at 30 minutes after dosing, with secondary endpoints of 5, 10, 15, 45, and 60 minutes. Patients rated their pain intensity on a visual analog scale of 0 to 100. In addition to changes in pain intensity after dosing, patients rated their level of pain relief at time points of 5, 10, 15, 30, 45, and 60 minutes and their global satisfaction with the study’s medication at 30 and 60 minutes postdose. Both pain relief and global satisfaction were measured on five-point categorical scales.
The eDiary instrument had the added effect of regulating and recording treatment compliance. Patients initiated each dose when they had breakthrough pain and were then prompted by the eDiary to do their entries at the given time points. Their records were correlated with the measured-dose sublingual spray.
“Patients were trained, once they were feeling a breakthrough pain episode, to enter their pain intensity into the diary and then dose with the medication. Five minutes later, it would prompt them for their pain intensity again, then at ten minutes, and so on, for sixty minutes,” says Parikh. Every patient in the study went through an open-label period where they were titrated to an effective dose. Once they achieved their successful dose of the study’s medication, patients entered the double-blind period of the study. Patients utilized the eDiary in both periods of the study.
Parikh says Insys began designing the study in 2007, in a “collaborative approach” with its supplier. Starting with the time-based endpoints, she says it was then a matter of drawing on the supplier’s experience with similar trials to match the eDiary configuration for the data-collection needs. “We worked hand-in-hand with the development team to design the eDiary for the study.”
ePRO EXPANSION
Although eDiaries most often come into use in Phase 3 trials, the instruments are also employed in earlier phases of clinical development and prior to that, starting with patient selection as well. A typical trial might use eDiaries during the baseline area of the trial, prior to the administration of placebo or drug or multiple drugs, and then at other points later in the trial to help define efficacy outcomes.
“There has also been considerable progress in integrating eDiaries with all of the trial data, including biological data,” says Stone. “As eDiary data is collected, it can be integrated into a database that also contains data from other aspects of the trial, including all kinds of biological data, which can be sent back to clinicians and researchers.”
The same qualities even out the expense calculations of ePRO versus paper PRO, says Stone. “eDiary data can be integrated and analyzed very quickly because, in particular applications, the data is sent to central servers, often immediately. And one of the big advantages of using eDiaries is that you don’t have the extra step of translating from paper diaries into an electronic form which is very expensive, in addition to other problems with paper, such as incorrectly completed questionnaires.”
Besides applying ePRO at more stages of clinical development, researchers are finding new applications unique to the technology. Some portable eDiary units are fitted with monitors for cardiac function or glucose level, for example, and may someday make DNA or other biomarker scans. Connectivity might be via the internet, Bluetooth, smartphone, or Wi-Fi options.
Most eDiaries are dedicated units — specially configured smartphones powered with custom software. Other systems use laptops or computer chips in custom-built units. But the vision of eDiaries with “apps” for monitoring patients’ bodies begs the question of whether ePRO will become just an extension of smartphones, which already offer features such as heart-rate monitoring. Stone doubts such a future.
“A lot of folks think that creating an eDiary is a straightforward scheme. But years of research and practice have suggested that this is not just a simple technology issue. There is a host of human factors. It’s a matter of understanding how people use the device, where they foul up with the device, and so on. Over the years, suppliers have made a tremendous effort at making these devices compatible with people’s lifestyles. That’s what yields high compliance.”
ePRO ADVANTAGES
Both Stone and Parikh echo other researchers in their observation that patients find ePRO anything but a burden. They say patients generally have a positive experience using eDiaries, preferring them over other modes of data collection — an intangible but logically important benefit of the “direct-to-data” PRO approach, translating to its demonstrated high rates of patient compliance.
Parikh also stresses the superior effects of ePRO in obtaining critical data that flows more efficiently through the regulatory process: “The electronic database allowed for a hassle-free transfer of data to the FDA.”
Perhaps the icing on the cake for Parikh and the Subsys investigators is the ability to access and integrate PRO data in real time. Direct-to-data in that sense means untold savings in time, data cleanup, and validation on a daily basis during the trial. Perhaps that explains why, nearly every time a drug is approved based on ePRO, its sponsor gives a large share of the credit to the electronic solution.
COST & COMPLEXITY
Small companies doing small studies have pioneered eDiary technology, despite the general expectation that such companies would be the last to afford it. Most reports of the Jakafi approval, for example, emphasized little Incyte’s brave tradeoff of expense for the advantages patient-reported outcomes brought to the table in strengthening the approved labeling.
Even on paper, PRO (patient-reported outcomes) involve more trouble and expense than biological data collection. The “e” for electronic in ePRO typically adds the cost of equipment, software development through testing and validation, vendor services, patient and personnel training, and data security.
Once a company decides it needs PRO, however, the equation of whether or not to use ePRO or paper is simple: Given available funds and assuming a high value of symptomatic labeling for the product in development, researchers need to balance ePRO’s added cost against its relative efficiency and verifiability. Despite the high cost of clinical trials in general, or maybe because of them, buying the extra confidence in patient compliance, data accuracy, and regulatory conformance may be money well-spent.
Some evidence exists that, despite higher initial costs, eDiaries save money in execution through the reduced cost of data cleaning and the efficiencies of well-designed ePRO instruments. Obviously, having PRO data with the high compliance factor of ePRO is much more efficient than finding and correcting large-scale corruption from noncompliant paper reports. Researchers at the Department of Psychosomatic Medicine Charite, Humbolt University, Berlin found that, when compared with paper methods, ePRO methods reduced trial preparation time by 67% and data management time by 78% (Hair 2006).
In a Phase 3 study of a treatment for overactive bladder in which the primary efficacy endpoint included a count of the patients’ daily number of micturitions, use of eDiaries reduced error variance by 33.5%. The sponsor, Sepracor, translated the 33% drop in error variance into a 50% decrease in the number of patients who would have been needed to detect the drug’s effects.
A further element in the cost equation is the trial’s complexity and number of endpoints. Subsys’ Phase 3 trial handled multiple endpoints in time and symptomatic variables, though it measured only one symptom: pain. Jakafi’s pivotal study was even more complex with its multiple endpoints and symptom measurement. But again, ePRO appears not only more efficient, but more rewarding than paper reports.
In complex studies, eDiaries have some advantages over paper diaries, including the ablity to make “smart” queries that route patients to different question sets depending on their answers. Another option with eDiaries is to include cognitive tests, such as having the patient track something around on the screen. Such actions are integrated into patients’ everyday experience, again encouraging compliance and reducing overall costs.
GLOBAL PRO
Beyond measures by the FDA to encourage patient-reported outcomes (PRO) measures in clinical trials, the agency’s own criteria for PRO data and an electronic patient data consortium in Europe are pushing investigators and sponsors toward greater use of patient eDiaries, or ePRO, in clinical development. The FDA’s PRO Guidance sets the standards for patient data including appropriate security of ePRO data. The Critical Path Institute (C-Path), in cooperation with the FDA and the medical products industry, has formed the Patient-Reported Outcome (PRO) Consortium “for the purpose of developing, evaluating, and qualifying PRO instruments with the FDA for use in clinical trials designed to evaluate the safety and efficacy of medical products.” Meanwhile, Europe’s Clinical Data Interchange Standards Consortium (CDISC, www.cdisc.org) has created SHARE, a global, accessible electronic library to enable “precise and standardized data element definitions that can be used in applications and studies to improve biomedical research and its link with healthcare, based on the principles behind computable semantic interoperability, i.e. the ability for computer systems to be able to exchange information while retaining the meaning of this information.” SHARE subscribes to a common information model, the Biomedical Research Integrated Domain Group (BRIDG) and is a key catalyst in encouraging integration of all patient-data types, from the clinical to the practice setting. Among the aims of the CDISC, SHARE, and other initiatives is the general adoption of eDiary technology to collect PRO data in clinical research.