How many digits of precision does Prism store? How accurate are the results? What are the largest and smallest numbers Prism can store?
Prism 7
Prism 7 stores all values in double precision, which is about 12-14 decimal digits. You can enter (and Prism will remember) values as small as 1.0e-300 and as large as 1.0e300
Prism 1-6
Prism versions prior to 7 stores data you enter in single precision floating-point format.
Number of significant digits stored
Values are accurate to six (and sometimes seven) significant digits. The numbers are stored as binary values, so the number of digits of precision (in decimal) is not always the same. The least significant of the seven digits can sometimes change (be one off from what you entered), but the first six digits are always accurate. If you try to enter (or import or paste..) values with more digits than that, Prism will round to six or seven digits.
Don't confuse the number of digits of precision with the number of digits after the decimal point. The number of digits of precision counts all digits, before and after the decimal point.
Very large and tiny numbers cannot be entered
Single-precision positive values must be between 1.18 × 10−38 and 3.4 × 1038. Negative values must be between -1.18 × 10−38 and -3.4 × 1038. Numbers outside this range cannot be stored. If you have data with very large or tiny values, we suggest that you either change units so the values are not so large or small, or transform the values to logarithms before you enter them into Prism.
Calculations
Calculations are always carried out in double-precision, so the results should retain the original accuracy -- up to six, and usually seven, digits of precision. But the results of the calculations are stored in single-precision, with the limitations listed above.
Keywords: single precision, floating point, number of bits