RFC: add `getPrecision` method. about math

[BenMorel]

Hi, what would be the difference between getScale() and getPrecision()?

[bendavies]

hi @BenMorel

The PostgreSQL manual will say it better than me:

We use the following terms below: The precision of a numeric is the total count of significant digits in the whole number, that is, the number of digits to both sides of the decimal point. The scale of a numeric is the count of decimal digits in the fractional part, to the right of the decimal point. So the number 23.5141 has a precision of 6 and a scale of 4. Integers can be considered to have a scale of zero.

[bendavies]

@BenMorel any interest? thanks!

[alexviar]

@BenMorel

Working with precision is generally better than working with scale

Check this example:

(0.03/360)*8*30 = 0.02

Woking with scale of 10:
0.03/360 = 0.0000833333(3)
0.0000833333*8=0.0006666664(0)
0.0006666664*30=0.0199999920(0)

❌ 0.0199999920 == 0.02

Working with precision of 10:
0.03/360 = 0.00008333333333(3)
0.00008333333333*8=0.0006666666666(40)
0.0006666666666*30=0.01999999999(80)=0.02

✔0.02==0.02

[bendavies]

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

[alexviar]

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

It has a scale of 14 but a precision of 10. The precision refers to how many significant digits it has, not how many decimal places it has. https://www.logicbig.com/quick-info/programming/precision-and-scale.html

For example Big.js uses scale whereas Decimal.js uses precision

[alexviar]

@alexviar i'm not sure what your example shows but it's not precision of 10

Working with precision of 10:
0.03/360 = 0.00008333333333(3)

0.00008333333333(3) has as precision of 16

I did put in parentheses digits that was rounded, and underlined digits are the Periodo of decimal

[bendavies]

ah right thanks, i forgot that leading zeros are not significant.
but also for storage purposes (postgres), 0 <= scale <= precision so, It has a scale of 14 but a precision of 10. doesn't work, in practise.

anyway, the point of this of was to simple add a getPrecision() method, similar to getScale, nothing mnore

[BenMorel]

Hi @bendavies, sorry for the late reply. I'm not against adding a getPrecision() method if it is useful to you, so please feel free to open a PR along with a few tests, in particular to clarify what happens for leading zeros.

@alexviar I'm not sure what the decimal period has to do with the precision? Anyway, you make me think that this would be a great addition to BigRational: a way to output 10/3 as 3.3(3) for example. Would you want to work on this?

[BenMorel]

I opened issue #74 to track the BigRational as decimal number with period separately.

[alexviar]

@alexviar I'm not sure what the decimal period has to do with the precision?

I think I didn't explain myself well. I did put rounded decimals (and decimal period) in parentheses for illustrative purposes only. Precision is given by working with n significant digits instead of n decimal place:

Operation	Rounded to 10 significant digits	Rounded to 10 decimal places
0.03	0.03000000000	0.0300000000
360	360.0000000	360.0000000000
8	8.000000000	8.0000000000
30	30.00000000	8.0000000000
0.03/360	0.00008333333333	0.0000833333
(0.03/360)*8	0.0006666666666	0.0006666666
(0.03/360)*8*30	0.02000000000	0.0200000000

Now, if we take the rounded values to do subsequent operations, we have:

Operation	Rounded to 10 significant digits	Rounded to 10 decimal places
0.03	0.03000000000	0.0300000000
360	360.0000000	360.0000000000
8	8.000000000	8.0000000000
30	30.00000000	8.0000000000
0.03/360	0.00008333333333	0.0000833333
(0.03/360)*8	0.0006666666666	0.0006666664
(0.03/360)*8*30	0.02000000000	0.0199999920

Currently it can be circunvent by using BigRational, but this workaround doesn't work with irrational opreations like square root for example (sqrt(2)*sqrt(2) Never mind, I had read an article with an example like this, but I can not found it)