Introduction
A software vendor audit is a prerequisite to using a software application for regulatory purposes. In this article, I will explain the importance of performing software vendor audits, some of the challenges, and what areas to target during an audit. Software vendor audits are an integral part of your quality system. There are two reasons why software vendor audits are an integral part of your quality system. First, software vendor audits are a requirement of the quality system. Secondly, software applications either have a direct impact to patient safety or an indirect impact to patient safety. The second point leads us into why software vendor audits are so crucial.
Importance of software vendor audits
The main reason why software vendor audits are so important is ultimately patient safety. As mentioned earlier, software applications used for regulatory purposes can have a direct impact or indirect impact on patient safety. Therefore, it is essential that we not only validate them for their intended use, but evaluate the software development behind the finished product. Software applications used for regulatory purposes must be reliable and perform consistently in order to avoid data integrity issues. The integrity of the data created, processed, stored, and archived in these regulated software applications is paramount. Furthermore, the security of the software application is just as important. 21 CFR Part 11, Electronic Records and Electronic Signatures helps ensure both data integrity and adequate security. 21 CFR Part 11 leads me into the next section of common challenges with software vendor audits.
Common challenges with software vendor audits
The biggest challenge associated with software vendor audits is verifying 21 CFR Part 11 compliance. Often times software vendors claim their application is 21 CFR Part 11 compliant because they have an audit trail or because they know Part 11 is a buzzword that attracts potential clients. I find that in most cases, software vendors that claim their respective application is Part 11 compliant simply do not understand Part 11. Consequently, they end up with a software product that is not 21 CFR Part 11 compliant. Here is a very important point to consider. Although a software vendor may have all of the 21 CFR Part 11 functionality built into their application, it is invalid if the software vendor does not show objective evidence. In other words, the vendor must test all of the 21 CFR Part 11 functionality and provide objective evidence. An example of objective evidence is a screenshot of a test result. Time and time again, software vendors fail to test their Part 11 functionality.
Another crucial point is that just because the vendor tested for Part 11 in their environment, does not preclude you from testing 21 CFR Part 11 requirements in your environment. The same holds true for the reciprocal. If the vendor has not tested for Part 11, does not mean that your testing will suffice. At the end of the day, the FDA wants to see 21 CFR Part 11 requirements tested by both parties in their own environments.
Another common challenge with software vendor audits is the absence of a quality system on the side of the software vendor. So many software companies make great products for the regulated space, but are unaware of the FDA requirements their customers must meet before using their software products. This automatically causes a dilemma for the customer, especially if the software product meets the needs of the user community. In instances where the customer wants to use the software product without a supporting quality system, I recommend the customer document all of their audit observations and then require a corrective action plan to implement a reasonable quality system from the software vendor. The result, you have a documentation trail that shows you have fulfilled your obligation of assessing the vendor as well as requiring them to remediate and implement a quality system in a reasonable amount of time. Two important points here, ensure you have Standard Operating Procedures that govern these processes and make sure to follow-up with the software vendor periodically to ensure they are on track. Now, we will turn our focus on what areas to target when auditing a software vendor.
Target Areas
First, an important area to target is certifications. It is always a good idea to ask the vendor if they have achieved any certifications such as ISO or industry recognitions/awards for performance or technology. Certifications show a good faith effort by the software vendor to employ sound business and development practices.
The next area to target is standard operating procedures, work instructions, and policies. It is imperative the software vendor has some procedures in place. If not, there is a high likelihood their processes are out of control. Some essential procedures I like to see in place are: Software Development Lifecycle, Building and Data Center Security, Training, Bugs Tracking and Issue Tracking, Customer Support, Validation Lifecycle, Network Maintenance and Security, Code Management, Coding Standards, Organizational Chart, Change Control, and Escrow Account Management. In addition to the procedures, a site tour is also necessary.
The site tour should consist of a tour of the development space, general building security, and the data center and its security. The developer’s work space should be neat and orderly as opposed to unorganized. Although this may seem trivial, it is actually very important the space is neat and orderly, as that is a good indication of the type of software development that occurs. An unorganized work space may be indicative of sloppy developing and coding practices. The data center tour should include an evaluation of the security of the data center. Physical security as well as the network security including local terminals.
We are touching again on the importance of 21 CFR Part 11 compliance. When evaluating the vendor for 21 CFR Part 11, ensure they have the following in place: 21 CFR Part 11 requirements, 21 CFR Part 11 Test Scripts, Objective Evidence of 21 CFR Part 11 testing, and 21 CFR Part 11 Training evidence.
Training is two fold for the evaluation of the vendor. First you should ensure employees and contractors are trained on the applicable internal procedures, as outlined in their training matrix and job responsibilities. Secondly, the software vendor employees should receive some external 21 CFR Part 11 training. Please note that all training be documented so their is evidence of such training. Additional training that is recommended for all software vendor employees developing regulated software is “General FDA Awareness Training”. Throughout the process of developing regulated software, there are supporting systems such as a bug tracking system, issue tracking system, change control system, code management system.
Because the software vendor is developing, marketing, and selling regulated software to be hosted onsite or at a customer site, they must validate for their intended use the supporting systems such as the bug tracking system, issue tracking system, change control system, and code management system. These systems are critical to the development and maintenance of the regulated software and therefore should be validated per their intended use.
Conclusion
This concludes this article. However, it is important to reiterate the importance of software vendor audits and the role of software applications in the regulated arena. Remember, regulated software has either a direct impact or indirect impact on patient safety, therefore, it would be prudent to exercise solid audit practices when auditing these vendors.
|
