Why can custom fields round differently in Empower? - WKB191261

ENVIRONMENT

• Empower
• Oracle
• Microsoft Windows

ANSWER

The difference in values is related to the difference between how Microsoft and Oracle define double precision floating point numbers. When results are calculated, but before they are stored, the number format is governed by Microsoft's operating system rules. After saving, the number format of results is governed by the Oracle rules.

Example:

A custom field CM001 has the formula = SST3.1 (Area/Height).

Only the PQ Unk 1 sample has the label of SST3. This means for all other samples the CM001 calculation always references Area/Height of the PQ Unk 1 sample. Hence, it is expected that all samples will have the same value for CM001, but they do not.

In this example, CM001 for PQ Unk 1 is determined during processing of PQ Unk1 and is thus calculated based on the Area and Height values that have been determined but are not yet stored in the Oracle database. Hence, these values are governed by Microsoft rules.

After processing PQ Unk 1, the result and values are stored in the Oracle database and governed by the Oracle rules. The values are then used in the subsequent samples.

When other sample results are processed, the Area and Height from PQ Unk 1, as stored in the Oracle database, are retrieved and CM001 is determined.

Therefore, for the first processed result, CM001 values will likely differ slightly at the nth decimal place compared to the other resulting CM001 values.

The Acetone and Acetophenone values differ at the 14th decimal place for PQ Unk 1 compared with all other processed samples.

ADDITIONAL INFORMATION

See also TECN1851937.