Next: Block Designs Up: The External Representation of Previous: Permutation groups   Contents

Numerical Data Types

Some of the numerical data in the external representation are the result of possibly inexact computations. Basically, there are three sources of this inaccuracy:

• The inaccuracy of the finite floating point representation.

• Arithmetical errors during computation.

• Cutting short an otherwise infinite approximation process.

The end result is that, in general, numbers in the external representation can be considered correct only within certain limits. We say they are ``precise'' up to some significant figures (see the details below).

The external representation version 1.1 provides the following numerical data types:

• Arbitrary precision integers (element <z>).

• Arbitrary precision rationals (element <q>) written in format where and are integers.

• Floating point decimals (element <d>) up to some given precision specified as the number of significant digits.

A conforming software implementation must provide the corresponding internal representations.

Here are the rules for representing numerical data in the external representation:

• If a number is the result of an inexact computation then it must be represented using the decimal data type.

• The decimal representation of an inexact number must always contain the decimal point regardless the number would round up to an integer.

• Exact numbers must be represented either using the integer or the rational data type.

The precision of decimal numbers is indicated by an optional attribute precision of particular elements. The elements which can have this attribute are: <list_of_designs>, <block_design>, <function_on_...>,
<statistical_properties>. The rationale for having many elements with the optional precision attribute is to provide flexible scoping rules and avoid unnecessary repetition.

The precision attribute gives the number of significant figures of all decimal numbers in the tree whose root contains the attribute. This precision can be overridden by giving different precision in one or more subtrees. In general, a precision of a decimal number is the precision given in the root of the smallest subtree containing the number and with a root having a specified precision attribute. If an external representation document contains any data which is the result result of inexact computation, precision must be specified.

Next: Block Designs Up: The External Representation of Previous: Permutation groups   Contents
Peter Dobcsanyi 2003-12-15