Has Prism been validated? Does it comply with the FDA regulation 21 CFR Part 11?

Who do these regulations pertain to?

If your company submits experimental results to the Food and Drug Administration (FDA) for approval, your company is required to comply with 21 CFR 11 (aka “Part 11”), a large set of regulations that includes extensive documentation and open records for auditors. Drug regulators in other countries tend to use principles and regulations similar to those of the FDA in the United States.

Who is responsible for complying with the regulations?

GraphPad Prism organizes data, makes graphs and performs statistical analyses. Those analyses may be used in regulated research, but mostly are not. The FDA considers GraphPad Prism to be "off the shelf software". For that reason, GraphPad Software is not directly responsible for compliance with FDA regulations. Instead, responsibility for compliance lies with the user. Read the FDA policies on software. [These policies seem to be written for software embedded in a medical device. I cannot find any statement from the FDA about software used to analyze data to be submitted to the FDA.]

Does GraphPad plan to validate our software?
Not at this time. As part of the validating processes, GraphPad would have to create a lengthy document to ensure each step of software development (planning, programming, reviewing, testing, updating and documenting...) follows very detailed written procedures. The extra steps and review would be costly. While GraphPad Software is proud of how we develop software, we simply cannot comply with these requirements and continue to sell software in our price range (few hundreds of dollars). Because only a small fraction of all GraphPad customers are governed by validation regulations, we can not justify the additional expense.

Does GraphPad Prism provide security, user tracking, or maintain an audit trail?
Prism does not maintain an audit trail. However, users can print reports before and after any changes. If you require extensive security, we suggest that you implement network security and establish corporate Standard Operating Procedures to supplement data protection. When you import a text or Excel file into a Prism data table, you can check an option to maintain a link between the Prism data table and the original import file. This essentially locks the data in Prism – it is not possible to edit the data without unlinking the table first. If you always import write-protected files, and always check the option to prevent edits within Prism, you can maintain control over values were analyzed.

Does GraphPad provide test scripts to facilitate validation?
No one at GraphPad Software has experience with performing validations. Therefore, GraphPad cannot help you validate Prism except to provide these general comments.

• Installation. The Prism installer simply copies a few files into some folders. There are no issues with connecting to databases, other software components, etc. If Prism launches, it has been installed properly. To be completely sure, check the size and dates of all installed files to make sure that every computer you install Prism onto is running the same version.
• Operation. With off-the-shelf software, pharmaceutical companies can rely on the reputation of the vendor. As mentioned above, GraphPad does not validate Prism.
• Performance. You should test Prism using data as similar as possible to the actual data you will use it for. See below.

Does Prism produce accurate results?
We test our software against textbook examples and against other programs. We have high confidence that the results are correct. Some of our regulated customers have validated certain uses of GraphPad Prism with no errors or problems. However, we cannot guarantee there are no bugs. GraphPad recommends that you create test data that are representative of your intended usage, and compare Prism’s results with those from other programs or hand calculations. Then rerun those test files with every update to Prism (or perhaps your system). Your test data should have the same range of X and Y values, the same number of data points, the same number of excluded points, the same analysis choices, etc. Very important results should be checked on several programs. If you encounter any error or bug, contact us and we will work diligently with you to resolve the issue.

From the FDA: "Software testing has limitations that must be recognized and considered when planning the testing of a particular software product. Except for the simplest of programs, software cannot be exhaustively tested. Generally it is not feasible to test a software product with all possible inputs, nor is it possible to test all possible data processing paths that can occur during program execution. There is no one type of testing or testing methodology that can ensure a particular software product has been thoroughly tested. Testing of all program functionality does not mean all of the program has been tested. Testing of all of a program's code does not mean all necessary functionality is present in the program. Testing of all program functionality and all program code does not mean the program is 100% correct! Software testing that finds no errors should not be interpreted to mean that errors do not exist in the software product."

Does GraphPad provide test data?

We provide a set of files that test Prism's nonlinear regression with challenging problems created by the NIST.

If you need more help.
No one at GraphPad Software has experience with submitting analyses to the FDA or performing validations If you need help understanding the interface between software and the FDA, consider hiring a consultant who specializes in this field.

We could use your help

Please contact me if you have ideas to make Prism better comply with FDA guidelines, or if you have suggestions to make this article better.

Harvey Motulsky
Founder, GraphPad Software
suggestions at graphpad.com