Skip to content

logBase 10 floating issues #5

@Brandon-Rozek

Description

@Brandon-Rozek

I was reading through the code and noticed a TODO item in the function ordersOfMagnitude
-- TODO: logBase 10 1000000 /= 6 but 5, fix this
I noticed that it came from this line
(floor :: Double -> Int) . logBase 10)

There was a discussion on StackOverflow about the logbase 10 issues https://stackoverflow.com/questions/26844123/haskell-logbase-error.

Is there any particular reason why we use floor in this scenario? Since we're always dealing with integer results, perhaps we can implement something similar to what GHC did or even just round in a simpler case.
https://github.com/ghc/packages-integer-gmp/blob/master/GHC/Integer/Logarithms.hs

Thanks for your time!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions