Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memmap exception if BLANK header is set #475

Open
pkgw opened this issue Oct 16, 2024 · 0 comments
Open

memmap exception if BLANK header is set #475

pkgw opened this issue Oct 16, 2024 · 0 comments

Comments

@pkgw
Copy link
Contributor

pkgw commented Oct 16, 2024

Hi —

I'm trying to use reproject with an integer-valued image that has the BLANK keyword set. This turns out to lead to an exception, at least with the interpolating mode:

ValueError: mmap length is greater than file size

The problem is because of hdu_to_numpy_memmap(). It has various guards in place to only memmap if the HDU data array aligns exactly with what's on disk. The presence of BLANK causes Astropy to change my u16 data to f32, which means that the memmap should not be performed, but as things currently stand, the check doesn't get this right.

The current guard code is:

    if (
        hdu.header.get("BSCALE", 1) != 1
        or hdu.header.get("BZERO", 0) != 0
        or hdu.fileinfo() is None
        or hdu._data_replaced
        or hdu.fileinfo()["file"].compression is not None
    ):
        return hdu.data

I could easily add a test for the existence of the BLANK keyword, but I'm wondering if there's a "better" way to check for this case. Looking at the Astropy FITS code, there's a field called _data_needs_rescale alongside the existing _data_replaced; it's False for this HDU, but maybe that's an Astropy bug, and it should be true? There are also an hdu._blank field. Naively, maybe it's an Astropy bug that _data_replaced is not True in this situation?

I'm happy to submit a pull request for this, as well as provide a specific reproduction case, but I was wondering if anyone had input about the best approach to take.

pkgw added a commit to pkgw/pywwt that referenced this issue Oct 16, 2024
See astropy/reproject#475 . This should of
course be fixed upstream, but in the meantime, we can avoid the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant