FDA Seeks Public Comment On Quality Metrics Reporting Program

On March 9, 2022, the FDA announced the establishment of Docket No. FDA-2022-N-0075 to solicit comments on changes to FDA's previously proposed Quality Metrics Reporting Program (QM Reporting Program). The purpose of the docket is to potentially refine the QM Reporting Program based on lessons learned from industry-based pilot programs and draft guidance for industry titled Submission of Quality Metrics Data.

The FDA defines pharmaceutical manufacturing quality metrics as an objective means of measuring, evaluating, and monitoring the product and process life cycle to proactively identify and mitigate quality risks, thereby managing operations at higher levels of safety, efficacy, delivery, and performance.

The FDA intends to use data reported under the QM Reporting Program to calculate quality metrics for each product and covered establishment. The FDA will analyze the information with statistical and machine learning methods to provide useful insights for inspection resource allocation. Specifically, the FDA intends to establish a signal detection program as one factor in identifying establishments and products that may pose significant risk to consumers, identify situations in which there may be a risk for drug supply disruption, improve the effectiveness of establishment inspections, and improve FDA’s evaluation of drug manufacturing and control operations.

The FDA utilized the Quality Metrics Site Visit Program to tour the operations of 14 domestic and international establishments to gain an in-depth understanding of how quality metrics data collected, analyzed, communicated, and reported are used in industry. The FDA also obtained feedback on industry’s anticipated challenges in applying the approach described in FDA’s revised draft guidance QM Reporting Program.

The three primary metrics the FDA intends to collect data on include:

The FDA has identified the following four general categories for the QM Reporting Program:

1. Manufacturing Process Performance

• Process Capability/Performance Indices (C_pk/P_pk)
• LAR
• Right-First-Time Rate
• Lot Release Cycle Time

2. Pharmaceutical Quality System (PQS) Effectiveness

• CAPA Effectiveness
• Repeat Deviation Rate
• Change Control Effectiveness
• Overall Equipment Effectiveness
• Unplanned Maintenance

3. Laboratory Performance

• Adherence to Lead Time
• Right-First-Time Rate
• IOOSR
• Calibration Timeliness

4. Supply Chain Robustness

• On-Time In-Full (OTIF)
• Fill Rate
• Disposition On-Time
• Days of Inventory On-Hand

The FDA is specifically soliciting comments on the following aspects of the QM Reporting Program. The FDA encourages stakeholders to provide the rationale for their comments, including available examples and supporting information. The FDA is also interested in any other pertinent information that stakeholders and any other interested parties would like to provide regarding the QM Reporting Program.

Reporting Levels

1. Do you agree that reporting should be aggregated at an establishment level?
2. Would reporting at an establishment level facilitate submission of quality metrics data by contract manufacturing organizations?
3. If you normally assess metrics by product family at an establishment, what are useful definitions of “product family” from your industry sector?

Practice Areas and Quality Metrics

1. If you think the general practice areas listed in section II of this notice would not meet the objectives of FDA QM Reporting Program, what other practice areas should FDA consider?
2. If FDA were to consider Quality Culture as one of the general practice areas, what are the critical components of a robust quality culture and can any of these components be measured quantitatively? If so, how do you recommend quality culture information be captured as a quantitative metric (e.g., near misses, APR on-time, binary response to Quality Culture survey, or other numerical metrics/KPIs)?
3. Do you think that any of the examples of quality metrics proposed by FDA would not be an appropriate measure for the designated practice area?
4. What other metrics should FDA consider for a designated practice area?
5. FDA is interested in an establishment’s experience with implementing process capability and performance metrics. For example, how would you report C_pk and/or P_pk to FDA as part of the QM Reporting Program (e.g., reporting C_pk and/or P_pk for certain products, aggregated at the establishment level)?
6. A metric may need to be changed or adjusted by an establishment to better monitor PQS effectiveness, inform appropriate business strategy, or capture insightful trends, thereby driving continual improvement behaviors. What criteria should be applied to justify changing or modifying a quality metric (by either the establishment or by FDA)? How frequently would you expect changes or modifications to be needed?
7. When would you rely on multiple metrics versus a single metric as an indicator when assessing a particular practice area (e.g., two metrics are considered in combination because one metric influences the other)? What combination of metrics have been meaningful and useful?

Other Considerations

1. Are there considerations unique to specific product categories (e.g., generic drug products, OTC drug products, or biological products) that should be addressed in the QM Reporting Program?
2. What would be the optimal reporting frequency for quality metrics data submissions (e.g., monthly, quarterly, or yearly, and segmented by quarter or month)?
3. In instances where a manufacturer is not able to extract domestic data and its submission to FDA contains both U.S. and foreign data, how can these data be submitted to FDA in a manner that would still be informative?
4. Are there any other aspects of FDA’s proposed direction for the program that FDA should address in future policy documents?

Analysis

The QM Reporting Program should be viewed as a collaborative endeavor between the pharmaceutical industry and the FDA. However, the pharmaceutical industry should be cautious about overcommitting to the data that is required to be submitted. Smaller organizations may not have the resources necessary to collect, summarize, and submit the information. The pharmaceutical industry and the FDA need to fully understand which information is required to meet the FDA’s expectations and requirements for the QM Reporting Program to ensure the least burdensome approach is maintained.

Many of the metrics contained in the four general categories for the QM Reporting Program can and will be beneficial if applied properly and can and will be beneficial to firms to use as CAPA triggers. However, the issue is, as always, does the firm have the proper channels to easily gather, summarize, and report the data? Any metrics that are utilized should become part of the annual product review (APR) process as well as inputs to management review.

Make sure you are heard; submit written comments before June 7, 2022, to the Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852 or electronic comments to https://www.regulations.gov. Please reference docket number FDA-2022-N-0075 with all comments.

About The Author:

Mark Allen Durivage has worked as a practitioner, educator, consultant, and author. He is Managing Principal Consultant at Quality Systems Compliance LLC, an ASQ Fellow and SRE Fellow. Durivage primarily works with companies in the FDA regulated industries (medical devices, human tissue, animal tissue, and pharmaceuticals) focusing on quality management system implementation, integration, updates, and training. Additionally, he assists companies by providing internal and external audit support as well as FDA 483 and warning letter response and remediation services. He earned a BAS in computer aided machining from Siena Heights University and an MS in quality management from Eastern Michigan University. He holds several certifications, including CRE, CQE, CQA, CSSBB, RAC (Global), and CTBS. He has written several books available through ASQ Quality Press, published articles in Quality Progress, and is a frequent contributor to Life Science Connect. You can reach him at mark.durivage@qscompliance.com with any questions or comments.