How Some Sponsors Internally Evaluate Their CDMOs

By Louis Garguilo, Chief Editor, Outsourced Pharma

Effective and consistent measurement of service provider performance is no easy task in today’s biopharmaceutical industry, for Big Pharma or small/virtual organizations. To learn from some biopharma professionals how they’ve managed this difficult task, Outsourced Pharma San Francisco Conference & Exhibition assembled a panel of supply chain experts:

• ALI JAVADIAN, Director of Biotherapeutics & Vaccines Outsourcing, Pfizer
• ULRICH ERNST, SVP of Tech Operations, Amunix Pharmaceuticals
• THOMAS SUN, VP of Nonclinical Development, Nuredis
• DAVID UPCHURCH, VP of Supply Chain, Jaguar Animal Health
• JOHN MCSHANE, Managing Partner, Validant

What follows is a portion of that Q&A discussion, in which the panelists shared best practices and perspectives on what parameters to use in evaluating CMOs/CDMOs, how to address low performance partners, and how to meet growing FDA expectations.

Q: How effective are your CDMO partners on a scale of 1 to 10?

ULRICH ERNST, AMUNIX PHARMACEUTICALS: It varies. Some are probably a little bit lower than 8 to 9, maybe in the 5 to 6 range — but the decisions to go with them were strategic and financial in nature. The thinking was that we would bring the rest of what was needed to the table in making those selections. You don’t need the perfect CMO; you need one that works to meet your needs, especially during early-stage development. If you can foresee gaps in a CMO’s capabilities, and you fi ll those gaps with your own expertise, you can close the distance on the point scoring.

That is how we, as a small company, approach it, because time and cost are two very important variables that we want to control. That approach is the opposite — perhaps diametrically opposite — of what large pharma companies do. But when you put yourself in the shoes of the ownership structure we have to deal with, the mandate nowadays is to work lean and fast. It’s “damn the torpedoes” sometimes. Ownership is willing to take risks — risks that perhaps Big Pharma isn’t — to get results. That is the tradeoff.

I’m not saying one is right and the other wrong. I have worked on both sides. I know the paradigm, and sometimes it’s very, very different.

THOMAS SUN, NUREDIS: It’s hard to put a number on it, but in my experience it is very heterogeneous. There are certainly some differences between a small company looking to outsource one or two projects, and a Gilead or a Pfizer. Large companies have much more leverage than small companies.

As a small company, what you can do to make your partner effective is develop long-term trust. Help them to do the work. Try to put yourself in the issues — be more understanding. Once you develop a good relationship, you develop trust. Next time around, even if your project is small, you can expect them to put capable people on your project. Overall, we just try to help partners work as effectively as we can.

I would put a number at 5 to 6.

DAVID UPCHURCH, JAGUAR ANIMAL HEALTH: Reflecting a lot of what the others have said, from the large pharma perspective I would say 6 to 9.9. The 6s are not because of a failure in technical capability or understanding of the industry. A 6 is because you didn’t communicate adequately up front about your needs or their capabilities. As Ulrich indicated, you can backfill gaps in capabilities to help the supplier. The question is: What is the fit between what you need and what that supplier can provide?

Moving to the small pharma side, the picture is not quite as rosy, unfortunately. I would go a little bit rosier than Thomas did — into the 6 to 7 range — but again I’ll put a lot of the blame on the sponsor side. Many times, the people making partnership decisions do not fully understand or communicate their needs. Most small pharma companies need just about everything. So, if they decide to go with a CMO or CDMO that can’t provide 100 percent of that, there’s going to be disappointment.

Q: What parameters do you use to measure partners?

ALI JAVADIAN, PFIZER: We measure on quality, technical, cost, deliverables, and absolutely cultural fit. We form partnerships with our CMOs. They are our extended family — our extended lab, extended manufacturing — so we have to establish that relationship. Eventually, we want them to be part of us, so cultural fit is extremely important.

ERNST: We don’t measure our suppliers in the traditional sense, like a large pharma would in a very studious manner. As I mentioned previously, we need to move relatively adroitly, and we are not given the time to adjudicate multiple partnering opportunities.

First and foremost, we look at technical fit. Can they do the processing that is required? Do they have the necessary skillsets and capabilities? We look at maybe one or two batches early on, depending on how much time we can get in the facility. Ideally, we don’t want to invest in equipment, and we don’t want to wait for the equipment to show up, either. I’m not saying we wouldn’t make investments — I have done so in the past — but we would be very selective, and it would be in a time frame we could foresee would meet our goals.

The other things in the mix are cost and quality. They are never out of the mix, but they’re certainly not the primary driver.

SUN: I don’t have a matrix to score them from 1 to 10; I develop a measurement in my head over the years. The two things I really care about and measure are technical competence and communication. Those are the most important things. If a contractor can’t demonstrate these traits, then I would not use them.

After that, I always look at how quickly a contractor can get started on my project — where I rank in terms of priority on their end. As a result, I tend to go with smaller contractors more than larger ones.

UPCHURCH: My response would be similar. In a small company, we are looking at technical innovation. Getting access to the scientist who will be handling your project can give you a feel for cultural fit and trust. And frankly, price comes in as a total cost, which is directly related to technical innovation capability. They also need to recognize that we have other basic needs as a small pharma. We need input in terms of quality and regulatory, and that factors into the total package.

Q: What do you do with those that rank the lowest? Do you take any action?

JAVADIAN: No, we don’t take any direct action, but we do work with the people at the top to resolve the issue. In some cases, CDMOs are issued a 483 that is not related to all projects at the facility, and we will work with their QA group to help resolve it. In our internal monthly compliance meeting, we evaluate these sorts of issues and discuss what we can do to bring the CDMO’s level up. We don’t leave them stranded there.

JOHN MCSHANE, VALIDANT: I was in external quality with Genentech for about three years. We had our performance matrices, but we rarely kicked any CMO out. Instead, we would start moving volume away from people who were not performing and had work plans to get their performance back up to our expectations. There were certainly some tense conversations along the way.

ERNST: I would echo that. In my experience at J&J, we would work with the underperforming folks, but would also look at moving capacity to sites we felt were more amenable to meeting our needs. We would rarely cut off a CDMO entirely, and it’s always a slow and very measured process in these large organizations.

UPCHURCH: I’d add that when I was at Gilead, we would open up our rating systems to the CMO. That way, we could have very open and honest conversations — it actually made it a little bit easier to talk through those issues. The result, as we previously discussed, is giving them either more projects/volume or less.