The FDA 's Focus On Metrics, Performance, And Quality
By Ellen Leinfuss, SVP and Life Science practice leader, UL EduNeering
At the PDA Quality Metrics Conference in December 2013, FDA CDER (Center for Drug Evaluation and Research) Director Janet Woodcock set out her goals for the conference: “to shift [the FDA’s] focus to performance and away from compliance.” Under the FDA’s lead, the life sciences community will have no choice but to follow suit in its own operations and approaches to quality.
Product quality is the inherent goal of life sciences companies. It is their mission in developing and producing products, delivering those products to patients, and ensuring the futures of their organizations. Historically, the standard of quality was compliance, but highly publicized recalls, product bans, and drug shortages have forced companies to rethink quality. At the CDER, QbD (quality by design) has nudged the industry to build in quality controls at the beginning of a drug’s life cycle. At the Center for Devices and Radiologic Health (CDRH), the rallying cry has been around the case for quality. And the FDA is seeking industry input about the metrics essential to quality control in order to make riskbased decisions, ranging from inspection scheduling to assessing the potential for drug shortages because of product quality issues. The FDA wants objective measures of product quality, site operations, and site systems performance.
The Value of Metrics
The FDA challenged life sciences companies to identify the key objective metrics that indicate product and site health. The agency wants to see product and site metrics for trend comparison across the industry.
The use of metrics would seem to be self-evident to quality and compliance managers, yet product recalls, plant shutdowns, enforcement actions, and product shortages continue. At the PDA conference, attendees agreed about the main goals and benefits of using metrics: to eliminate subjectivity, provide a benchmark and visibility for continuous improvement, and ensure side-byside training across multiple operational functions, including quality assurance, operations, and production. With such agreement, the questions have to be asked, “Why are so few organizations successfully integrating metrics into their operational systems?” and “Why are we still seeing too-common product recalls, plant closures, enforcement actions, and drug shortages?”
Metrics Vs. Data
Life sciences organizations have data — hundreds of thousands of pieces of data, often unorganized, inaccessible, and inconsequential as a quality tool. At the conference, participants voted on preferred site metrics, ranking the top five as confirmed out-of-speculation (OOS) rates, CAPA (corrective and preventive actions) effectiveness, batch-failure rates, critical investigation rates, and environmental monitoring grades. The same question was provided for product metrics, with the top five listed as complaint rates, OOS rates, process capability, critical investigation rates, and batch-rejection rates.
Interestingly, these identified metrics can spark initial red lights, or they can form the resulting action plan. Consider, for example, OOS rates compared month over month for one facility. A deviation — often very small — is noted, setting a benchmark and warranting attention. If the trend increases, questions must be asked. Is the deviation local or enterprisewide? Did the deviation begin with the installation of new equipment or systems? Has a new workforce been hired or the existing workforce been downsized? Is a new manager in place? Do training metrics show any drop in completion rates or levels? Have budgets been cut for maintenance of production equipment?
The same questions should occur in reverse with the right metrics. Consider training metrics that show a drop in successful completion rates for a specific topic. Is the decrease limited to one facility, production line, or product? Is it common across production lines at multiple plants? Does it correlate with quality issues such as OOS rates or batch-rejection rates? Based on those answers, what is the action plan? Remedial training for a defined group of learners? Manager training? A renewed commitment to an enterprisewide culture of quality? New training materials? Additional new-hire onboarding?
The answers and subsequent actions based on useful, accurate metrics will determine a company’s quality, compliance, and financial profile. With the FDA’s evolving approach to quality, how a company collects and uses metrics will also identify a company’s risk for quality failures — and whether or not those risks warrant added scrutiny and inspections by the FDA.