Floating point numbers in the OpenVMS are stored as binary fraction times a power of 2. (In computer language, the first part is called the mantissa and the latter, the exponent.) For example:
100.0 is stored as .78125 x 2^7 = 25/32 x 128
10.0 is stored as .625 x 2^4 = 5/8 x 16
2.0 is stored as .5 x 2^2 = 1/2 x 4
1.0 is stored as .5 x 2^1 = 1/2 x 2
.5 is stored as .5 x 2^0 = 1/2 x 1
.25 is stored as .5 x 2^(-1) = 1/2 x 1/2
.1 is stored as .8 x 2^(-3) = 6.4/8 x 1/8
Note that .10 cannot be represented exactly because 6.4/8 is not an integral fractional power of 2 but an infinite series. The only fractional decimal values that can be represented exactly are those that are fractional powers of two or integral multiples thereof; for example,
1/2 = .5 1/16 = .0625
1/4 = .25 3/4 = .75
1/8 = .125 7/8 = .875
Fractional powers of ten (except when the value is also a fractional power of two) can never be exact on a binary machine; the fraction is always an infinite series. Consequently, arithmetic results cannot be guaranteed to be exact. In fact, using penny values in a dollars and cents calculation, only the values .00, .25, .50, and .75 can be represented exactly.
To provide a reasonable interface to the real world, numeric print routines (like those in ACCENT R when print pictures are used) round the value for display only. Thus, though a number like "12.35" would be printed, the actual value might be 12.349 999 998 73.
In addition, the system function @RND can be used to round off numbers to a specified digit and the field @EPS can be used to specify the exactitude of comparison between numbers.
Within the framework of this machine format, obviously, the more bits used to store the binary fraction (mantissa) of a number, the more accurate the number can be represented. The number of decimal digits that can be contained in a specified number of bits is log 10 (2^N). The number of bits per decimal digit is N/(log 10 (2^N)), where N > 1, and is approximately 3.32. So, in a single floating number where 26 bits are used to store the binary fraction of a number, you can expect an accuracy within 7 digits (e.g. 26/3.32 = 7.82). The following list details the accuracy you can expect from the floating point numbers used in OpenVMS.
Integer (I) and Real (R) fields are represented internally in 32 bits in the VAX. Numeric (N) fields are represented in 64 bits in the VAX.
Accuracy of OpenVMS system values:
Largest positive signed integer = 2^31 -1 = 2,147,483,647
Number of decimal digits = 9.331 929 87
Real, all 24 bits mantissa.
Maximum value = 1 x 2^127 = 1.701 412 x 10^28
Number of significant decimal digits = 24 / 3.32 = 7.224 719 896
Real, 23 bits with "real world" one bit round off error
Maximum value = 1 x 2^127 = 1.701 412 x 10^28
Number of significant decimal digits = 23 / 3.32 = 6.923 689 9
Hardware format double precision, all 56 bits
Maximum value = 1 x 2^127 = 1.701 412 x 10^28
Number of significant decimal digits = 56 / 3.32 = 16.857 679 76
Hardware format double precision, 55 bits with one‑bit round off error
Maximum value = 1 x 2^127 = 1.701 412 x 10^28
Number of significant decimal digits = 55 / 3.32 = 16.556 649 76
ACCENT R is constrained by hardware double precision accuracy.
The transcendental functions @SINR, @COSR, and @TANR are optimized for radian arguments in accordance with generally accepted standards. Their degree counterparts (@SIN, @COS, @TAN) are provided for convenience, but they do contain a minor inaccuracy resulting from the design of the CPU.