By Kenneth Wu

When you boil it down, there are five common methods being used for forecasting and budgeting clinical studies. These are:

• bid-generated budgets
• internally developed software
• resource management software
• cost benchmarking software
• activity-based costing software

Often, more than one method is utilized within the same company or even the same department, which demonstrates the general lack of planning sophistication in the life sciences industry. There are advantages and disadvantages to each of these methods. What follows is based on 20 years of experience both working at and consulting to leading biopharmaceutical and medical device companies.

#1: Bid-Generated Budgets
This method relies on getting bids from CROs to cost the study. One advantage of this method is that for sponsors, it’s basically free. It also leverages the CRO’s costing tools and management expertise. In addition, you have continuity if one of the CROs is selected to conduct the study, and the method can provide a detailed breakdown of study costs.

However, there are many disadvantages with this method. First, many CROs are reluctant to provide a ballpark bid when many of the study assumptions aren’t known. But, in order to ensure you have an adequate budget to conduct the studies in the future, that’s exactly what you need them to do. In addition, building program-level budgets from a series of individual study bids is a very time-consuming process.

It’s hard to place an average accuracy percentage number on the CRO bid method, since CRO estimates will vary significantly if underlying assumptions are not yet defined. There are also large cost variances between CRO estimates because of different underlying assumptions and CRO size/perspective. For example, site payments are a significant cost driver (30% to 50%) of total study budgets. CRO labor estimates differ between CROs according to their organization and billing practices. Small differences in fully burdened labor costs will be magnified if, for example, enrollment rate is slower than planned.

#2: Internally Developed Software
Most companies have internal software that consists of Microsoft Excel spreadsheets or Access databases for costing clinical studies. It will typically have some clinical and institutional knowledge “baked in,” and can provide a detailed breakdown of study costs.

On the other hand, internal software has significant disadvantages — which is disconcerting since spreadsheets remain the most widespread costing tool for clinical studies in the industry. As spreadsheets evolve over time, there is rarely a robust process in place to validate that all formulas and links continue to work as intended, in addition to the fact that large spreadsheets can become corrupted and no longer function properly.

Although the clinical and institutional knowledge that is built into these tools can be very accurate for studies in the same indications and locations, their accuracy can vary when dealing with new indications or complex, global studies. Variances of up to 40% from plan to actual costs are not unheard of with spreadsheets.

Finally, because spreadsheets can be widely disseminated, it can be difficult to identify which version (if any) represents the single version of “the truth.” And finally, the knowledge of how they work is often limited to one or two individuals in the organization, which means, if these people leave the company, that knowledge leaves with them.

#3: Resource Management Software
This refers to commercial software that is customized with clinical algorithms to provide study costs. Advantages include costs being driven by time and materials of study resources. This method has the ability to estimate internal as well as external costs and can provide a detailed breakdown of study costs.

However, using resource management software for costing is a very time- and resource-intensive approach — especially with the ongoing maintenance of custom algorithms to stay current with the organization and the industry as a whole. It’s dependent on available internal IT and clinical subject matter experts and often dependent on third-party vendor services and consultants.
And while generally accurate to around 80% for resources, this method can create wide variances if the fully burdened costs of each resource are not carefully developed and maintained. Finally, turnaround times for initial costs and revisions are often problematic. This method is usually found in large biopharmaceutical and medical device companies due to the level of internal and external resources required.

#4: Cost Benchmarking Software
Cost benchmarking software is commercial software featuring a database of cost data from previously contracted studies. It’s generally accurate to 80% of actual costs, and typical vendors have thousands of top-line clinical study costs in their databases to pull from. The databases also contain studies across virtually all therapeutic indications — although some indications have a smaller sample size because fewer studies have been conducted for those indications.

Cost benchmarking software represents an improvement over an Excel-based tool, but there are still several disadvantages. As a lagging indicator based on average costs in historical service provider bids, cost benchmarking software does not adequately reflect new developments such as adaptive studies or the trend toward more complex, global studies. And because it’s a broader, top-down approach, cost benchmarking can’t reflect the unique qualities of your own study, which adds to the potential for meaningful variance. Sponsor companies and CROs vary significantly in processes and resources. Global sponsors and CROs will yield a higher cost estimate than small start-up sponsors and small CROs, and it can be difficult to control for these variances in a benchmarking approach.

#5: Activity-Based Costing Software
Activity-based costing software is commercial software that builds study budgets and forecasts from the “bottom up,” based on the clinical assumptions you enter. Activity-based costing is a methodology widely used in other industries that have already experienced the higher scrutiny on costs and resources now facing the life sciences.

One of the key advantages of activity-based costing software is that it is the most accurate: within 90% of actual spend when few clinical assumptions are known, and up to 97% accurate when detailed assumptions are entered. It also gives transparency to the clinical assumptions on cost drivers and enables better forecasting of costs and FTE demand based on when the work will actually occur. Thus, activity-based costing software moves beyond being a simple costing tool to providing accurate cost projections, controls, and negotiation leverage.

The most powerful benefit with activity-based costing software is its ability to quickly generate “what-if” scenarios for planning purposes. This type of software enables rapid testing of different underlying assumptions to view their impact on study cost, timelines, and resource demand. The optimal scenario can be modeled for your business objectives in the context of a competitive landscape. Previously, scenario analyses required a team of subject matter experts and financial analysts weeks to simulate.

Of all of the methods available, activity-based planning software will provide the most accurate budget and resource estimates for clinical studies. However, in order to be effective, the software cannot be generic — it needs to have built-in intelligence that adjusts estimates based on user input, therapeutic area and indication, number and locations of study sites, and a host of other factors. Activity-based planning software should also be updated regularly to ensure it adapts to changing practices and conditions.

Determine Your Requirements
In order to choose the clinical study costing method that’s right for your organization, carefully weigh the different approaches against your own requirements. For example, how accurate do your ballpark estimates need to be? Do you need to develop and revise budgets quickly? Do you consider scenario planning critical for creating better study plans? Is transparency to assumptions, cost drivers, and resource demand critical to planning and making decisions? Is alignment with service providers on tactical plans, timeline, resources, and costs imperative in the planning and implementation stages? You’ll want to choose the level of sophistication that best meets your needs.

One Sponsor's View: Vertex Pharmaceuticals, Inc.

Vertex Pharmaceuticals recently purchased an activity-based costing system from ClearTrial (see box) to replace its bid-based study costing method. We spoke with Peter DiBiaso, Senior Director of Clinical Operations at Vertex, to get his views on the results so far.
“What we’ve found is that activity-based costing (ABC) software supports planning best practices. Since it takes a ‘bottom-up’ approach, it forces the discipline at the sponsor level to really think through the study’s operational design and protocol objectives early in the planning process, avoiding potentially costly errors associated with an unforeseen final contract amount. We’ve developed a collaborative process that relies on the study teams and our budget analyst using the software to create a detailed study plan before engaging outside vendors for bids.

“Prior to implementing our planning processes and adopting the use of the software, we were relying primarily on bid responses from CROs for the study budgets. These estimates could vary as much as 15% to 20% from our original assumptions. With ABC software, based on our initial deployment, we find we’re consistently within 5% variance from plan (bid baseline) to actual (final contracted amount) — for studies as well as change orders — and being within 1% to 3% is well within our expectations.

“Another advantage of ABC software is its ability to accurately calculate internal as well as outsourced study costs. We’re using the software to run different insource vs. outsource scenarios, which lets us evaluate and choose the approach that best matches our study outsourcing strategies.

“The software also enables effective change management by providing visibility to prospective change orders once the study is under way. By taking actual data from the study in progress – for example, screen failures – and using these to refine the original study assumptions, we’re able to easily create a reforecast that shows us the impact on cost, timeline, and resource demand.

“We’ve been taking advantage of the ability to configure the software to create study templates based on the nuances of the therapeutic indications we work in, as well as building in the specific clinical knowledge we’ve gained over the years.

“A couple of caveats: the system we’re using focuses on later-phase development, and doesn’t calculate some external costs such as central labs. We’ve taken a cue from the software, however, and are actually using an activity-based costing method to calculate central labs and other costs and including these totals in the software so they are included in the total study costs. And our vendor is launching its early-phase product this summer, so we’re looking forward to expanding activity-based costing to our Phase 1 study planning.”
 

• ClearTrial (www.cleartrial.com)
• Medidata Solutions (www.mdsol.com)
• Planisware (www.planisware.com)
• TTC (www.ttc-llc.com)

About The Author
Kenneth Wu, principal consultant of Kenneth Wu & Associates, LLC, is a 20-year veteran in clinical operations at medical device and biopharmaceutical companies. He worked in several successful early stage start-ups, midsize, and large biopharmaceutical companies.