find_transform returns inconsistent results?

[LeoBosse]

Hello,
I'm discovering astroalign. It seems great for my purpose, which is to align star fields, and would avoid having to use a proprietary software. However, I have troubles understanding the behavior of the find_transform function.
When I run my script several times on the two same fits images, it does not return the same transformation matrix! And the differences are not negligible. (I also tried to convert the fits data to "<f4" as proposed here: #73)
Here are 2 examples with images. As you can see, the second call to the function is way worst than the first. How is that possible? It does not select the same stars, even though the detection_sigma is set to 5.
Is there a solution? Running it multiple times and taking the average improves the reproducibility a bit, but it feels like a dirty fix and still leads to large differences.
I might miss the obvious, but couldn't find any solutions online.

Thanks for your help!

Transformation 1:

[[ 9.98525129e-01, -5.34912594e-02,  4.06667567e+00],
     [ 5.34912594e-02,  9.98525129e-01, -1.30381042e+02],
     [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]]

Transformation 2:

[[ 1.00472461e+00, -5.55261918e-02, -5.29514493e+00],
     [ 5.55261918e-02,  1.00472461e+00, -1.38557707e+02],
     [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]]

[martinberoiz]

Hi Leo,

Looking at the transformation outputs, it seems the rotation angle is about the same, but the translation vector is off by about 10 pixels.

I can see the disparity, can you send me (just) a couple of images and a notebook where you notice the problem? I can take a look this weekend and diagnose the problem better.

One thing you can try in the meantime is to restrict the number of sources it uses to find the transformation (max_control_points keyword). For example:

aligned_image, footprint = aa.register(source_image, target_image, max_control_points=10)

[martinberoiz]

by the way, to have a better idea of the error of the transformation, sometimes it's useful to look at the scale, rotation and translation of the transform, rather than the bare entries of the transformation matrix.

transf.rotation, transf.translation, transf.scale

[LeoBosse]

Thank you for your quick response Martin! I wouldn't want to make you work on the week-ends though.
Using max_control_points seems to help for the consistency! However, wouldn't it decrease the quality of the fit?

Here is the script and the 2 images used in the example. (quiet heavy, I couldn't load more here, but it should be enough). To reproduce the problem, just run the script a few times. If like me, the transformations should not be exactly the same than in the example.
example.zip
For context, this wouldn't be too much trouble if I only had a few images to align. Just save the best fit in a file, and reuse it. But here, I have a lot, and I can't verify each of them every time.

[martinberoiz]

No worries. I'll take a look and will try to find out what's going on. I'm also interested in understanding this behavior.

[LeoBosse]

Hello Martin, did you have time to take a look? I tried to go through the star selection algorithm because it felt like where the randomness come from. But I got lost in the sep module and couldn't understand much.

[LeoBosse]

So, I've tried a new way. This doesn't solve the issue sadly. The idea is to use find_transform() multiple times with different parameters and register all found stars in a big list. Then pass this list to find_transform().
This doesn't really work as planned. When passing the long list of stars to find_transform(), the returned list is much shorter (200 in, 50 out), and so the transformation doesn't take into account all the stars I give it! Even when setting the max_control_points=1000.
It seems to me this issue could be related to the first one, but I have no clue why or how. I attach the new script I use for this (just replace the .txt to .py, github doesn't allow python files in here). The main function where I implement this idea is the first one called FindTransform(). The rest is unchanged.
align.txt

[martinberoiz]

Hi Leo, I haven't had time to look into this issue yet. I'm caught trying to find the time to finish a new release that deals with masking. When I finish that one, I'll take a look at this issue. Thanks for keep digging on this!

I can't remember now from the top of my head, but since there's an O(N^2) part of the algorithm, astroalign will restrict the number of sources it works with. Also, more stars may just crowd the invariant space and confuse the algorithm. You'd have the best luck with a few (O(10)) bright stars that appear on both frames.

[LeoBosse]

Thanks for the update! I'll continue to mess around a bit. No rush :)
I just had a new idea. Most of the times, the guide stars are all located in the same region of the image. For example, in the bottom right corner. And it is what cause the upper left corner to be misaligned. So I could cut my pictures in smaller chunks, and take a few stars from each chunks, to force an equal repartition of the guide stars on all the image.
I'll try that, and continue to write update here if it can ever help somebody.

[LeoBosse]

So as promised, I messed around today, but couldn't find a satisfying solution. I think I'll wait for you to take a look, as I don't have time or knowledge to dig into astroalign code. Here is a brief summary of what I did:
Breaking the image in chunks to gather guiding stars in each one, gatheing every star list, then finding the transformation using this list of stars. Same idea with using a mask to erase all stars in the center of the image. 2 cases: either I pass very few stars scattered in the corners, and get this error: astroalign.MaxIterError: List of matching triangles exhausted before an acceptable transformation was found. Or I pass more stars, but then it selects only a portion grouped in a small region of the image which gives a bad fit on the other side.
I tried to take the median/average of the transformation of each chunk, but this gives very poor results.
I join the script, which starts to be a bit messy, sorry...
align.txt

So from what I understand, the algorithm tries to find the best fit, which seems logical. But by doing so, it restricts to nearby stars that match well, and forget that the image is much bigger, causing misalignment on the other side. Maybe a solution would be to add the option to give a higher priority in using spread out stars, and only then searching for the best fit with those? Or accepting a worst fit when given few stars (maybe just with a warning) ? Not sure if and how this is doable though...
For my use, a small error consistent on the whole image is much better than a good and a bad side.
I also have to say that my images are not dark and flat corrected. The field of view is around 40x30 degrees. I don't think there can be a strong lense distortion between the two images. I don't see how this could mess up the alignment, but maybe I'm missing something.
I'll be putting this aside for now, until you had a time to look into it.