The Value Of Experiential Data In Making Informed CMO Selection Decisions

By Kate Hammeke, Market Research Director, Industry Standard Research

As part of Life Science Leader’s commitment to bringing the most valuable decision-making tools to its readership, the methodology and research supplier for the Leadership Awards has changed. Industry Standard Research, a full-service market research provider to the pharma and pharma services industries, collected experiential data on Sponsor-CMO service encounters that have transpired in the past 18 months to serve as the foundation for the 2016 CMO Leadership Awards.

This approach, which provided feedback from recent customers across 26 performance metrics on more than 80 CMOs, more closely corresponds with the level of qualification important to the CMOs receiving the awards and the Sponsors that engage contract manufacturing services.

There are five core CMO Leadership Award categories: Quality, Reliability, Capabilities, Expertise, and Compatibility. Several performance metrics factor into each award category in order to help both Sponsors and CMOs to understand a quantified evaluation and accurately assess a supplier’s skills within a category that can span a range of definitions. Have you ever asked your customers or vendors how they define quality? ISR has. And it’s tricky to find coherent through lines (i.e., ongoing themes).

The change from perception-based data to experiential data is important because it shifts the focus of the awards from a CMO’s reputation — a product of marketing, public relations spin, and any “industry buzz” about a company known by professionals in the industry — to how a CMO has performed for its current and recent customers relative to their expectations. How well a CMO executed its contracted responsibilities for customers is valuable information when it comes to making informed CMO selection decisions, particularly because industry perceptions don’t always match reality.

This can be especially true when it comes to identifying industry leaders. ISR’s CMO benchmarking surveys ask respondents to list up to three CMOs they think of as leaders in either small molecule, biologic, or drug product manufacturing services. This is an unprompted question where respondents are not provided with a list of CMOs from which to choose. As a result, sometimes a CMO was listed as a perceived leader for services the business does not offer — this can mean that the company is benefitting from the halo effect, a cognitive bias where an observer’s overall impression of a company influences the observer’s thoughts about the company’s properties. But the halo effect is not always beneficial. When it backfires (so to speak), it can influence an individual to hold a company to higher expectations than warranted or expect a range of services and support in which the supplier has no expertise.

The data shows the 185 different suppliers were proffered by respondents when asked to name leaders in contract manufacturing. The three businesses receiving the highest number of mentions also are three of the largest CMOs and the CMOs that topped the list for proposal volume. In fact, CMO size (and the well-funded marketing budget that often accompanies size) appears to have a greater influence on perceived leadership than anything else, including how a CMO performed according to its customers. Alas, it’s no surprise that customer experience can be drowned out by the marketing and PR machines; that is the purpose, after all.

But are the marketing communications designed to influence perception potentially harming the business by fueling high expectations? The in-depth results from ISR’s research point toward a challenge of scale faced by service providers and vendors from all types of industries — service excellence can be diluted by large customer bases. Or, it’s impossible to please everyone all the time. This impossibility is further supported by the absence of a consensus on the “most important attribute” for a CMO to possess — even for a particular type of project.

In addition to asking research participants to evaluate each CMO that they have recently done business with, the survey asks respondents to prioritize the 26 performance metrics as “Top 5” and the “Most Important” when it comes to selecting a CMO. The metric deemed most important — strong regulatory track record — by the largest percentage of respondents, only captured 14 percent of the vote; 40 percent of respondents included this metric in the Top 5. A track record of meeting quality performance metrics and a proven ability to manufacture API tied for second position, with 10 percent of respondents stating one of these metrics is the most important attribute. Yet only one-quarter of respondents included these metrics in their Top 5, suggesting that these particular metrics carry significant weight to a fraction of buyers of outsourced services, but are less significant in CMO selection among a larger portion of the outsourcing audience. Reliable on-time delivery was the only other metric to be deemed the most important criterion by roughly one in 10 respondents (and among the Top 5 of 41 percent of respondents). Then, there is a steep drop-off in agreement on the most important selection attribute, and the next five metrics are each considered the most important to one in 20 respondents.

Some of the challenges of making an informed CMO selection decision can be alleviated by knowing which CMO performance metrics have positively contributed to successful outsourcing relationships among peers and then which CMOs have performed best on these metrics for their customers. The data in Figure 1 displays how the performance metrics are ranked by buyers of outsourced services and which award categories correspond to the performance metric. This data can be coupled with the 2016 CMO Leadership Awards winners list to help identify best matching CMOs for upcoming outsourced work. A mix of contract manufacturers with respect to both size/capacity and scope of offering won CMO Leadership Awards in 2016 based on how well these businesses performed for real clients in recent history. Using this data to support your company’s CMO vetting process (or to understand your company’s relative position to its CMO peers) will put you on the path toward a successful partnership.

Survey Methodology: Industry Standard Research’s Contract Manufacturing Quality Benchmarking research is conducted annually via an online survey. For the 2016 CMO Awards data, more than 80 contract manufacturers were evaluated on 26 different performance metrics. Research participants were recruited from biopharmaceutical companies of all sizes and are screened for decision-making influence and authority when it comes to working with contract manufacturing suppliers. Respondents only evaluate companies with which they have worked on an outsourced project within the past 18 months. This level of qualification ensures that quality ratings come from actual involvement with a business and that companies identified as leaders are backed by experiential data.

If you want to learn more about the report or how to participate, please contact Andrew Schafer, president, or Kate Hammeke, market research director, at Industry Standard Research by sending an email to andrews@isrreports.com or kateh@isrreports.com.