Emulate Float64

[ggkountouras]

1) Make it work

Using the theory from SoftFloat (https://github.com/ucb-bar/berkeley-softfloat-3) and the partially finished libMetalFloat64 (https://github.com/philipturner/metal-float64), implement a proof-of-concept version. At this stage, it is okay to have low throughput compared to native Float32.

2) Make it right

Implement rounding modes. Add atomics. Ensure IEEE-754 compliance with tests.

3) Make it fast

Add option to drop strict IEEE-754 compliance (remove denormals, don't check for Inf/NaN). Add vectorization. Inline at a higher level. Implement Fused Multiply-Add.

[maleadt]

I think this would be useful to have, but ideally as part of a vendor-neutral package (a la DoubleFloats.jl -- maybe something exists already).
Are you proposing to work on this, or are you just checking if people are interested?

[ggkountouras]

Are you proposing to work on this, or are you just checking if people are interested?

A bit of both. I would need some guidance on what is required.

A vendor-neutral package would be best as a long-term goal. Metal.jl makes sense as a short-term target, since 1) it is a mainstream platform and 2) it has no hardware support for Float64. For maximum performance, the internals will have to be optimized for each different platform anyway.

[maleadt]

I'd suggest starting with something vendor neutral, designing it so that it is GPU compatible and can accommodate the performance optimizations from metal-float64, and implementing those platform-specific optimizations as a Metal extension. That way you can easily develop and debug the package in a full-blown Julia/CPU environment, which is always much easier.

It's not clear to me how we would plug this into the compiler, though. I guess we could compile down to LLVM IR and replace all operations on double IR values to the soft-double library just like we do with the GPUCompiler.jl runtime, but that's a bit messy. Doing it at the Julia IR level is another possibility (cc @vchuravy, maybe you have suggestions).

The alternative, re-using the existing library or writing something that compiles down to LLVM bitcode and plugging that into Metal.jl would work too, but is far less portable.

[ggkountouras]

I talked to the author of libMetalFloat64, and he said it's unlikely that software-emulated Float64 will accelerate my use case (solving Differential Equations), and that it's better to re-work the algorithm to work with Float32.

I'm still interested in this for compatibility reasons.