How many digits can Prism store?
Prior to Prism 7, Prism stored only seven digits (actually it is binary, so it is 'about' seven decimal digits) so did not always store the exact value you entered. For example, if you enter the value 99.2492427 into a data table, Prism rounded it to 99.249250.
Prism now stores numbers with double precision, so it can store about fourteen significant figures. If you enter the value 99.2492427 into a data table, Prism stores it as 99.2492427
How to show fewer digits than you entered
By default, Prism tries to show all the digits you entered, and increases the number of digits after the decimal place to accommodate the extra digits. But you can use the Decimal Format dialog to show fewer digits. And the results of analyses can be rounded as well.
What you see on the Prism data table matters only when you look at that table. Notes:
•Choosing to show fewer digits on the data table won't affect how Prism analyzes or graphs your data. Analyses are always based on all the digits you entered (up to the number Prism can store).
•When you save the Prism file (as .PZF or .PZFX), Prism saves all the digits you entered (up to fourteen), so the values are not truncated when you save.
•When you copy data from a Prism data table to the clipboard, Prism copies the values you entered, not only the digits you see on the table.
•When you export a data table as a .TXT or .CSV file, Prism exports the digits you entered, and not only those that appear on the table.
Rounding
There are two issues that can make rounding seem erratic.
•When the next digit is a 5. Rounding values is straightforward unless the next digit is 5. Should 2.5 be rounded up to 3 or rounded down to 2? Both are equally accurate, and many strategies have been proposed to deal with this issue. The Wikipedia article on rounding explains the many possibilities. Prism rounds values that end with 5 somewhat unpredictably. For example, 0.25 gets rounded up to 0.3, but 0.45 gets rounded down to 0.4. These are both correct, since rounding a 5 up or down is an arbitrary decision. Prism Windows uses the C function sprintf() to display real numbers. The function used by the Mac Xcode compiler is similar but not quite identical.
• Decimal to binary to decimal. Computers store the number as binary digits, so need to convert to and from decimal. For many values, this requires rounding as well. For example, let's say you type in the number 0.055, and ask Prism to display the value to only two decimal places. The next, third, digit is 5. One common rule to always round up when the next digit is 5, so you'd expect Prism to display 0.06. But the computer can't store the value 0.055. Instead, all values are converted to binary, and the binary value is close to 0.05499996 when converted back to decimal. When rounding this value to two decimal places, the next (third) digit is a 4. So Prism Windows rounds down in this case and displays 0.05. Prism Mac rounds up, in this case, and displays 0.06.
Rounding with Prism 8
The Format Decimal Format dialog rounds down what you see on the table. But when you go to edit the data, all the digits you entered (or imported or pasted) are still present. Starting with Prism 8, you can use a transform round the data to a specified number of digits. This is a built-in transform (bottom of the list) for both X and Y values.
The results will appear on a results page. They will actually show one more digit than you want and that last digit will always be zero. You can use Format Decimal format to show one fewer decimal place. From the results page, you can do further analyses or copy or export the rounded values.
Rounding with earlier versions of Prism
Click Analyze and choose Transform. Then choose User-defined Y transform. Then click "Add..." to add a new function. Here is a transform that rounds numbers to only include two digits after the decimal:
Y = Floor(Y*10^2 + 0.5)/10^2
The transform multiplies the Y value by 10^2 or 100, adds 0.5, and then rounds down to the lower integer (that is what the Floor function does). Then divide by 100.
Replace "2" in two places by "3" to round to three places, etc.
The results will appear on a results page. They will actually show one more digit than you want and that last digit will always be zero. You can use Format Decimal format to show one fewer decimal place.
