You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I see that from #19 the decision was made to match the Perl implementation by truncating integers. This was perhaps in part due to an "accumulation of error" from #18
For example, if you have 23.4 as a sample count you can upscale it to 234, then set factor to 0.1
Which implies that decimal points are supported.
In firestorm the units are in nanoseconds. At least on my computer, the minimum resolution for a sample is 100ns. To "unclutter" the text, I would like to display microseconds instead with fractional values. Eg: instead of 17,100ns, show 17.1μs. This would require fractional values to do well. The current behavior sometimes ends up truncating values to 0.
The accumulation of error can be avoided by not using floating point for calculations throughout, but instead only use the factor at the last moment when formatting the SVG text.
The text was updated successfully, but these errors were encountered:
I see that from #19 the decision was made to match the Perl implementation by truncating integers. This was perhaps in part due to an "accumulation of error" from #18
But, the documentation specifies that
Which implies that decimal points are supported.
In firestorm the units are in nanoseconds. At least on my computer, the minimum resolution for a sample is 100ns. To "unclutter" the text, I would like to display microseconds instead with fractional values. Eg: instead of 17,100ns, show 17.1μs. This would require fractional values to do well. The current behavior sometimes ends up truncating values to 0.
The accumulation of error can be avoided by not using floating point for calculations throughout, but instead only use the
factor
at the last moment when formatting the SVG text.The text was updated successfully, but these errors were encountered: