Precision (deterministic) floating point operations

[bitmarkcc]

For my application, I need llm.c to calculate the loss of the model (on some subset of the data) with exactly the same result for all platforms. The operations on the 'float's is nondeterministic. It gives slightly different results for different machines. I don't care about this for training, but when I evaluate the model to see how good the loss is, I need it to be consistent between all platforms. What works for me is using the boost multiprecision library (boost/multiprecision/mpfr.hpp), but memory and cpu usage is way too high. Any ideas for how to achieve the same effect without much of a cost in CPU time and memory usage?

Should I use doubles to make the calculations, then convert back to float? I just need about 7 sig figs of precision. You can see my code in my btm branch: https://github.com/bitmarkcc/llm.c/tree/btm.