Add benchmarking script, update API_DESIGN.md to reflect the results. #161
Conversation
| """ | ||
|
|
||
| images = [] | ||
| for aug in [VectorizedRandomCutout, VMapRandomCutout, MapFnRandomCutout]: |
There was a problem hiding this comment.
Would it be useful to also have numbers with XLA compilation?
There was a problem hiding this comment.
#165 lets do in a follow up.
There was a problem hiding this comment.
(but I do think so)
There was a problem hiding this comment.
In this graph I've only temp enabled the XLA for the VMap class (TF 2.8):
There was a problem hiding this comment.
In this case for Vectorized and VMap:
There was a problem hiding this comment.
In this case for all the 3 classes:
There was a problem hiding this comment.
We also need to consider the general discussion we had at #146 (comment)
As this one is a special case that is running fine with vectorized_map/fallback_to_while_loop=False.
Also, this simple case doesn't have any specific not JITable ops like e.g. ImageProjectiveTransformV3 (E.g. for rotation) or tf.bincount (see #141 (comment))
| """ | ||
| import time | ||
|
|
||
| import matplotlib.pyplot as plt |
There was a problem hiding this comment.
Is the benchmark a test or an example?
https://github.com/keras-team/keras-cv/blob/master/setup.py#L30
| `BaseImageAugmentationLayer` requires you to implement augmentations in an | ||
| image-wise basis instead of using a vectorized approach. This design choice | ||
| was based made on the results found in the | ||
| [vectorization\_strategy\_benchmark.py](../benchmarks/vectorization_strategy_benchmark.py) |
There was a problem hiding this comment.
How we are going to run this from command-line?
There was a problem hiding this comment.
As this was not isolated as a performance test we need to tell to the devs to:
pip install -e .
And the to pip uninstall keras-cv
| plt.subplot(3, 3, i + 1) | ||
| plt.imshow(images[i].numpy().astype("uint8")) | ||
| plt.axis("off") | ||
| plt.show() |
There was a problem hiding this comment.
This is not going to work well in containers like in our official keras dev container. Isn't better to default plot on file?
…keras-team#161) * add benchmarking script, update API_DESIGN.md to reflect the results. * Format benchmarks * Fix lint * add () to call method * update api design guidelines
…keras-team#161) * add benchmarking script, update API_DESIGN.md to reflect the results. * Format benchmarks * Fix lint * add () to call method * update api design guidelines
* Adds unit normalization and tests * Adds layer normalization and initial tests * Fixes formatting in docstrings * Fix type issues for JAX * Fix nits * Initial stash for group_normalization and spectral_normalization * Adds spectral normalization and tests * Adds group normalization and tests * Formatting fixes * Fix small nit in docstring * Fix docstring and tests * Adds RandomContrast and associated tests * Remove arithmetic comment * Adds RandomBrightness and tests * Fix docstring and format * Fix nits and add backend generator * Inlines random_contrast helper * Add bincount op * Add CategoryEncoding layer and tests * Fix formatting * Fix JAX issues * Fix JAX bincount * Formatting and small fix * Fix nits and docstrings * Add args to bincount op test
No description provided.