Nov 9, 2020 ¤ Journal
TIL that 27 digits of significance are possible in Ruby...why not? Here's something that should just be
>> 19120.01.to_d / 2124445 => 0.9000002353555869886017289e-2
Say you were estimating how many significant digits this way:
>> (19120.01.to_d / 2124445).to_s.split(".").size => 27
This digit estimation works great for small, cleaner divisions. But it falls apart, since floating point math never truly escapes us. As usual,
#round to the rescue:
>> (19120.01.to_d / 2124445).round(5) => 0.9e-2 >> (19120.01.to_d / 2124445).round(5).to_s => "0.009"
Just a friendly reminder that you'll probably, always, want to
BigDecimal's when dividing. Thank me later.