Invertable affine augmentations #10
Conversation
...to (Kornia)AugmentationPipeline. Returned trans_matrices allow apply_inverse
even if aug.return_transform is False
failed on windows before, due to drive letter being interpreted as uri schema
There was a problem hiding this comment.
the augmentation pipleine gets an additional kwarg: return_transform (default False) just like all kornia augmentations have.
I think that part makes sense.
If True it returns a the transformed tensors and the responding affine transformation matrices.
But the transformation is in general not an affine matrix. Quite a few augmentations can be expressed as affine, but this does not hold true in general, e.g. for elastic deformations.
These can be used, e.g. to call
AugmentaionPipeline.apply_inverseto invert the (geometric part of the) transformations.
I think we need to be aware of what transformation and then choose the correct inverse function based on the transformation (or throw some error if the inverse for the transformation is not defined.)
torch_em/transform/augmentation.py
Outdated
| class KorniaAugmentationPipeline(torch.nn.Module): | ||
| interpolatable_torch_tpyes = [torch.float16, torch.float32, torch.float64] | ||
| interpolatable_numpy_types = [np.dtype('float32'), np.dtype('float64')] | ||
| class AugmentationPipeline(torch.nn.Module): |
There was a problem hiding this comment.
I think "KorniaAugmentationPipeline" makes more sense as a name, because it's not for generic Augmentations.
torch_em/transform/augmentation.py
Outdated
|
|
||
| def halo(self, shape): | ||
| return self.halo | ||
|
|
||
| def apply_inverse(self, *tensors: torch.Tensor, forward_transforms: Sequence[torch.Tensor], padding_mode="border"): |
There was a problem hiding this comment.
This only works for affine trafos / anything that can be expressed as affine trafo. But that's certainly not the case for all augmentations we have, e.g. elastic deformations.
There was a problem hiding this comment.
renamed it to apply_inverse_affine to make it explicit that it applies only to transformations that can be expressed as an affine transformation.
|
I think it would still make sense to eventually come up with a more general functionality for inverting the augmentations, but for now we can go with this option. I think we should def. add tests here though... |
idea: the augmentation pipleine gets an additional kwarg: return_transform (default False) just like all kornia augmentations have. If True it returns a the transformed tensors and the responding affine transformation matrices. These can be used, e.g. to call
AugmentaionPipeline.apply_inverseto invert the (geometric part of the) transformations.