-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add post on PyTables direct chunking API #12
Conversation
Suggested by @FrancescAlted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, the blog looks excellent to me! I just have some minor suggestions (that you may want to ignore if you don't agree with them). Well done!
posts/pytables-direct-chunking.rst
Outdated
:width: 75% | ||
:align: center | ||
|
||
In this case direct chunking yields 4.5x write and 1.9x read speedups, with write/read speeds of 0.8 GB/s and 1.6 GB/s. The absolute numbers are of course not as impressive, but the performance is still much better than that of the regular mechanism, especially when writing. Please note that the M1 CPU has a hybrid efficiency+performance core configuration. We also run the tests on another hybrid CPU of the high-performance range, in this case an Intel Core i9-13900K CPU with 8+16 cores and 32 MB L2 cache, clocked at 5.7 GHz: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be honest, I'd not add so many benchmarks of different machines, as this can overwhelm readers a bit ;-). I'd skip the Intel machine, as it has a similar architecture (and behavior) than the AMD.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, I agree, though the write results on the Intel are IMO too good not to mention them, so I replaced the whole passage with a short summary sentence in commit fa0dc00. Does it look ok to you like this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that's fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thanks @ivilata
To avoid overwhelming the reader, as suggested by @FrancescAlted.
Thanks @FrancescAlted and @avalentino for the review! |
The development of this feature was funded by the NumFOCUS Foundation under a Small Development Grant.