Floating point precision in Tcl

Moderator: GiD Team

Post Reply
eduardrocas
Posts: 30
Joined: Thu Dec 11, 2014 2:34 pm

Floating point precision in Tcl

Post by eduardrocas »

Hello,

in GiD-Tcl, when summing 10+0.46 I'm getting values like 10.46000000000001. This makes impossible to compare floating-point values, because they will always be inequal. So I wanted to change the value of the built-in variable tcl_precision, but it seems that this Tcl built in variable is not "built in" GiD. Does this variable have a different name or it has to be done differently?

Thanks,
Eduard
eduardrocas
Posts: 30
Joined: Thu Dec 11, 2014 2:34 pm

Re: Floating point precision in Tcl

Post by eduardrocas »

In relation to my previous message, I would like to point to the following link:
http://wiki.tcl.tk/1650

It seems that in Tcl 8.5 setting tcl_precision to 0 avoids for example 1.4 to be 1.3999999999999999. I'm using GiD version 12.0.3. What Tcl version does it implement?

Thanks,
Eduard
User avatar
escolano
Posts: 1918
Joined: Sun Sep 05, 1982 10:51 pm

Re: Floating point precision in Tcl

Post by escolano »

The same happen in all programming languages like C/C++ or FORTRAN when converting to strings,
and a simple exact value like 0.1 (decimal base) has an infinite series of numbers expressed in binary base, then it will be always truncated!!

You must not compare two floating points directly comparing strings (and for example in floating point or exponential format will be very different). You must compare the absolute value of its difference with a given tolerance

In GiD we are internally setting a very big tcl_precision value (we use 17 instead of historical value of 12)
This variable handle the precision converting the double internal number to a string expression, not the precision of internal operations.
As you pointed, from Tcl 8.5 it seems that it changed its behaviour, we must study in GiD if we must set to the value to 0 that has a special meaning. In any case I recommend you to not change this variable and left the GiD default value.
User avatar
escolano
Posts: 1918
Joined: Sun Sep 05, 1982 10:51 pm

Re: Floating point precision in Tcl

Post by escolano »

Current GiD uses Tcl 8.6.1 (can see it in Help->About... - More)
eduardrocas
Posts: 30
Joined: Thu Dec 11, 2014 2:34 pm

Re: Floating point precision in Tcl

Post by eduardrocas »

Excellent. Very good answer.
Yes, I will use a function as you propose. It will be nice and convenient when GiD makes use of the "tcl_precision" set to 0.

Thanks,
Eduard
Post Reply