Better real number printing about json-fortran

[zbeekman]

This sounds like an odious task.

[szaghi]

Real comparisons are not trivial... it is difficult to check if the real is
actually zero or not (finite precision representation, that changes between
different architectures).

I think you should implement a zero machine. This falls into a
portability problem: into IR_Precision module I have tried to solve
this....).

Il giorno 22:51 ven 09/gen/2015 Izaak Beekman [email protected] ha
scritto:

This sounds like an odious task.

—
Reply to this email directly or view it on GitHub
#39 (comment)
.

[jacobwilliams]

I'm not sure what you mean by "zero machine".

[szaghi]

A portable zero representation. When you are facing with real-type comparison a statement like a==0.0 where a being defined as real:: a is not safe for many reasons. I guess that for a better printing you are searching for a pseudo code like the following:

real:: a
...
if (a==0.0) then
  write(out,'(A'))'0.0'
else
  write(out,'(E...)')a
endif

The real comparison a==0.0 is not portable neither well posed (the finite precision of a representation is a problem). It is more safe to introduce a tolerance representing your zero and do a safer (for reals) comparison like:

...
if (abs(a)<=zero) then
  write(out,'(A))'0.0'
else
...

Where zero should be the best portable zero representation. After some tests I am presently using a zero tolerance like:

integer, parameter:: R8P  = selected_real_kind(15,307)  !< 15  digits, range \([10^{-307} , 10^{+307}  - 1]\); 64 bits.
...
real(R8P),  parameter:: ZeroR8  = nearest(1._R8P, 1._R8P) - &
                                  nearest(1._R8P,-1._R8P)   !< Smallest representable difference of kind=R8P variable.

Using some different architectures (risc, cisc...) I found that the new portability functions of Fortran 90+ are very useful.

See you soon.

[zbeekman]

I don’t understand why it should be the responsibility of a JSON library to detect potential underflow or floating point rounding issues. It is always a burden that the programmer carries, and the json library should faithfully represent whatever data is passed to it rather than rounding it “for them.” To this end I think we should be very careful and write out all available digits. Obviously, trailing zeros may be safely trimmed.

[szaghi]

Hi Izaak, I agree, but this wan not the original question Jacob. If you decide to handle a better printing, I just like to point out the above eventual portability issue.

In general I try to completely avoid comparison involving reals.

[jacobwilliams]

My original idea was not to attempt to do any rounding, but just to print the same number in a nicer way, in order to minimize the number of characters and make it more human-readable. Currently, I always use (E30.16E3) formatting for reals. So, the number 123.0 gets printed as 0.1230000000000000E+003. It would be better to print it as 123.0. I don't believe there is a single format statement that does what I want for all possible values. So, the format statement would need to vary depending on the value, or I'd have to print it to a string and then examine the string, removing any unnecessary characters (e.g., trailing zeros).

[szaghi]

Hi Jacob,

your algorithm is not trivial just because the presence of possible rounding issues. Depending on the architecture/OS/compiler/real-definition... weather... the finite representation of 123.0 can be 123.0000000034 or 123.000000000001... All depend on how you make the library portable.

However, even if you are a paranoiac of portability like me, I do not know any built-in functions able to understand when the rounding is to be discharged. The only algorithm that I imagine is based on real comparison with a tolerance that must indicate when the rounding is to be discharged.

In other words, the algorithm to transform 123.000000000012 to 123.0 is not trivial.

When you think to 123.0000000000 (instead of 123.000000000012) you are implicitly assuming that your actual real variable is represented without rounding. In my opinion, you can be confident with this assumption when the following requirements are satisfied:

• your code is portable, with parametrized real kinds;
• after you have load your real variable you do not perform other operations than can modify its memory (e.g. if you stay with INTENT(OUT) on this variable).

In the case you are sure that 123.0 is really represented with 123.0, namely the above two requirements are satisfied, I think that your pretty printer is feasible, but it is still not simple.

I agree with Izaak that the best (simple and safe) way is to use all digit the library internally uses for representing the real variable and then eventually trim out trailing zeros. To this aim, into IR_Precision all parametrized kind-numbers have a set of casting functions (number-to-string and string-to-number, https://github.com/szaghi/IR_Precision/blob/master/IR_Precision.f90 lines 164+): it is very easy to implement a modified version for trimming out not relevant trailing zeros...

I hope I was clear...

See you soon.

[zbeekman]

With 'e' edit descriptors things are tricky because you don't know the exponent a priori. You can figure it out using something like floor(log10( x )) for positive floats (assuming 'es' scientific edit descriptor). The 'f' edit descriptor supports minimal width editing, e.g., f30.0 but is not flexible for large or small floats. The 'g' edit descriptor will switch between the 'e' and 'f' form and might be worth testifying: at least it will remove the exponent when it's 0. I think the only realistic way to accomplish what you want to do is to write to an intermediate string and analyze the exponent and significand.

[szaghi]

Yes Izaak,

I agree.

A possible scenario based on strings:

real(R8P):: x
integer:: i1,i2,i3
character(len:) , allocatable:: s
...

s = str(x) ! casting x to a string by means of scientific notation
i1 = index(s,'E') ! index of E character
i2 = find_first_trailing_zeros(s)
s = s(1:i2-1)+s(i1:)
...
! implement find_first_trailing_zeros

[zbeekman]

I’ve finished making these changes, such that the real number printing has better control.
The default real edit descriptor is calculated based on:

1. the precision of RK is used to control the d in the ew.dee edit descriptor.
   • multiplied by rp_safety_factor, an integer safety factor, right now set to 1
   • adding rp_addl_safety and integer fudge factor, right now also set to one
     • setting these to larger values will make it more likely that writing data, and then reading it back in will yield the same bitwise representation.
2. real_to_string now cleans up reals under the two following conditions
   1. If the real has an exponent then it the text joining the significant and exponent is saved and reused. Searches for common exponent notation matching the following regexp: `[eEdD]?[+-]? ensuring that if the exponent is just the sign, it must be to the right of the decimal, and to be considered an exponent, at least one of the characters matching the previous regexp must be present.
   2. All trailing zeros except for the first after the decimal are trimmed. E.g. 1.04500 becomes 1.045 and 10.00000 becomes 10.0.
      1. All leading zeros in the exponent are trimmed, but the last is saved if the exponent is 0. E.g., E0003 becomes E3, e+0010becomese+10andd-0003becomesd-3`

I chose the rp_addl_safety and rp_safety_factor so that minimal changes would be made to the outputs of the json_example test program. Setting them to larger values causes the printed values to deviate slightly due to rounding from what they are now, however it is likely that these deviations are closer to the actual machine representation. (Assuming that compiler vendors implement correct rounding on io)

[jacobwilliams]

This is terrific!

FYI: take a look at my string-tests branch, where I've got a couple of changes in the works. One is to have the option to revert back to the old way, for anybody who wanted to do that for whatever reason. The new way will still be the default. Also, I think the creation of real_fmt only needs to happen once, since it won't ever be able to change. That's a minor detail.