An Assessment Of 5 Clinical Study Costing Methods
By Kenneth Wu
When you boil it down, there are five common methods being used for forecasting and budgeting clinical studies. These are:
- bid-generated budgets
- internally developed software
- resource management software
- cost benchmarking software
- activity-based costing software
Often, more than one method is utilized within the same company or even the same department, which demonstrates the general lack of planning sophistication in the life sciences industry. There are advantages and disadvantages to each of these methods. What follows is based on 20 years of experience both working at and consulting to leading biopharmaceutical and medical device companies.
#1: Bid-Generated Budgets
This method relies on getting bids from CROs to cost the study. One advantage of this method is that for sponsors, it’s basically free. It also leverages the CRO’s costing tools and management expertise. In addition, you have continuity if one of the CROs is selected to conduct the study, and the method can provide a detailed breakdown of study costs.
However, there are many disadvantages with this method. First, many CROs are reluctant to provide a ballpark bid when many of the study assumptions aren’t known. But, in order to ensure you have an adequate budget to conduct the studies in the future, that’s exactly what you need them to do. In addition, building program-level budgets from a series of individual study bids is a very timeconsuming process.
It’s hard to place an average accuracy percentage number on the CRO bid method, since CRO estimates will vary significantly if underlying assumptions are not yet defined. There are also large cost variances between CRO estimates because of different underlying assumptions and CRO size/perspective. For example, site payments are a significant cost driver (30% to 50%) of total study budgets. CRO labor estimates differ between CROs according to their organization and billing practices. Small differences in fully burdened labor costs will be magnified if, for example, enrollment rate is slower than planned.
#2: Internally Developed Software
Most companies have internal software that consists of Microsoft Excel spreadsheets or Access databases for costing clinical studies. It will typically have some clinical and institutional knowledge “baked in,” and can provide a detailed breakdown of study costs.
On the other hand, internal software has significant disadvantages — which is disconcerting since spreadsheets remain the most widespread costing tool for clinical studies in the industry. As spreadsheets evolve over time, there is rarely a robust process in place to validate that all formulas and links continue to work as intended, in addition to the fact that large spreadsheets can become corrupted and no longer function properly.
Although the clinical and institutional knowledge that is built into these tools can be very accurate for studies in the same indications and locations, their accuracy can vary when dealing with new indications or complex, global studies. Variances of up to 40% from plan to actual costs are not unheard of with spreadsheets.
Finally, because spreadsheets can be widely disseminated, it can be difficult to identify which version (if any) represents the single version of “the truth.” And finally, the knowledge of how they work is often limited to one or two individuals in the organization, which means, if these people leave the company, that knowledge leaves with them.
#2: Internally Developed Software
Most companies have internal software that consists of Microsoft Excel spreadsheets or Access databases for costing clinical studies. It will typically have some clinical and institutional knowledge “baked in,” and can provide a detailed breakdown of study costs.
On the other hand, internal software has significant disadvantages — which is disconcerting since spreadsheets remain the most widespread costing tool for clinical studies in the industry. As spreadsheets evolve over time, there is rarely a robust process in place to validate that all formulas and links continue to work as intended, in addition to the fact that large spreadsheets can become corrupted and no longer function properly.
Although the clinical and institutional knowledge that is built into these tools can be very accurate for studies in the same indications and locations, their accuracy can vary when dealing with new indications or complex, global studies. Variances of up to 40% from plan to actual costs are not unheard of with spreadsheets.
Finally, because spreadsheets can be widely disseminated, it can be difficult to identify which version (if any) represents the single version of “the truth.” And finally, the knowledge of how they work is often limited to one or two individuals in the organization, which means, if these people leave the company, that knowledge leaves with them.
#3: Resource Management Software
This refers to commercial software that is customized with clinical algorithms to provide study costs. Advantages include costs being driven by time and materials of study resources. This method has the ability to estimate internal as well as external costs and can provide a detailed breakdown of study costs.
However, using resource management software for costing is a very time- and resource-intensive approach — especially with the ongoing maintenance of custom algorithms to stay current with the organization and the industry as a whole. It’s dependent on available internal IT and clinical subject matter experts and often dependent on third-party vendor services and consultants. And while generally accurate to around 80% for resources, this method can create wide variances if the fully burdened costs of each resource are not carefully developed and maintained. Finally, turnaround times for initial costs and revisions are often problematic. This method is usually found in large biopharmaceutical and medical device companies due to the level of internal and external resources required.
#4: Cost Benchmarking Software
Cost benchmarking software is commercial software featuring a database of cost data from previously contracted studies. It’s generally accurate to 80% of actual costs, and typical vendors have thousands of top-line clinical study costs in their databases to pull from. The databases also contain studies across virtually all therapeutic indications — although some indications have a smaller sample size because fewer studies have been conducted for those indications.
Cost benchmarking software represents an improvement over an Excel-based tool, but there are still several disadvantages. As a lagging indicator based on average costs in historical service provider bids, cost benchmarking software does not adequately reflect new developments such as adaptive studies or the trend toward more complex, global studies. And because it’s a broader, top-down approach, cost benchmarking can’t reflect the unique qualities of your own study, which adds to the potential for meaningful variance. Sponsor companies and CROs vary significantly in processes and resources. Global sponsors and CROs will yield a higher cost estimate than small start-up sponsors and small CROs, and it can be difficult to control for these variances in a benchmarking approach.
#5: Activity-Based Costing Software
Activity-based costing software is commercial software that builds study budgets and forecasts from the “bottom up,” based on the clinical assumptions you enter. Activity-based costing is a methodology widely used in other industries that have already experienced the higher scrutiny on costs and resources now facing the life sciences.
One of the key advantages of activity-based costing software is that it is the most accurate: within 90% of actual spend when few clinical assumptions are known, and up to 97% accurate when detailed assumptions are entered. It also gives transparency to the clinical assumptions on cost drivers and enables better forecasting of costs and FTE demand based on when the work will actually occur. Thus, activity-based costing software moves beyond being a simple costing tool to providing accurate cost projections, controls, and negotiation leverage.
The most powerful benefit with activity-based costing software is its ability to quickly generate “what-if” scenarios for planning purposes. This type of software enables rapid testing of different underlying assumptions to view their impact on study cost, timelines, and resource demand. The optimal scenario can be modeled for your business objectives in the context of a competitive landscape. Previously, scenario analyses required a team of subject matter experts and financial analysts weeks to
simulate.
Of all of the methods available, activity-based planning software will provide the most accurate budget and resource estimates for clinical studies. However, in order to be effective, the software cannot be generic — it needs to have built-in intelligence that adjusts estimates based on user input, therapeutic area and indication, number and locations of study sites, and a host of other factors. Activitybased planning software should also be updated regularly to ensure it adapts to changing practices and conditions.
Determine Your Requirements
In order to choose the clinical study costing method that’s right for your organization, carefully weigh the different approaches against your own requirements. For example, how accurate do your ballpark estimates need to be? Do you need to develop and revise budgets quickly? Do you consider scenario planning critical for creating better study plans? Is transparency to assumptions, cost drivers, and resource demand critical to planning and making decisions? Is alignment with service providers on tactical plans, timeline, resources, and costs imperative in the planning and implementation stages? You’ll want to choose the level of sophistication that best meets your needs.
About The Author
Kenneth Wu, principal consultant of Kenneth Wu & Associates, LLC, is a 20-year veteran in clinical operations at medical device and biopharmaceutical companies. He worked in several successful early stage start-ups, midsize, and large biopharmaceutical companies.