Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WCS sampling & FITS to TOAST FITS #86

Merged
merged 32 commits into from
Aug 31, 2022

Commits on Aug 18, 2022

  1. "black" formatting

    imbasimba committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    69bd1ff View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    013e873 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    303ed3f View commit details
    Browse the repository at this point in the history
  4. Added utility functions to (1) guess appropriate base layer level & (…

    …2) get all ancestors of a set of tiles
    imbasimba committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    3a2f7d7 View commit details
    Browse the repository at this point in the history
  5. Added array/wcs sampler

    imbasimba committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    5cef0cb View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    13aa67b View commit details
    Browse the repository at this point in the history
  7. Added tile_toast functionality from input array/wcs. This is now the …

    …default way to tile images with a large FoV
    imbasimba committed Aug 18, 2022
    Configuration menu
    Copy the full SHA
    df2262b View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    be98692 View commit details
    Browse the repository at this point in the history

Commits on Aug 19, 2022

  1. Configuration menu
    Copy the full SHA
    ce17f64 View commit details
    Browse the repository at this point in the history
  2. Updated view docs now that TOAST is the default tiling method for ima…

    …ges with large angular size
    imbasimba committed Aug 19, 2022
    Configuration menu
    Copy the full SHA
    019be3a View commit details
    Browse the repository at this point in the history
  3. Better Java detection

    imbasimba committed Aug 19, 2022
    Configuration menu
    Copy the full SHA
    9534fbd View commit details
    Browse the repository at this point in the history
  4. Test for TOAST from FITS

    imbasimba committed Aug 19, 2022
    Configuration menu
    Copy the full SHA
    7f98528 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    973d0f2 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2022

  1. Now using the same import of the TilingMethod enum, to ensure that en…

    …um things like TilingMethod.TOAST actually equals TilingMethod.TOAST. Also added assertion for the resulting imageset's projection type after creation by tests.
    imbasimba committed Aug 23, 2022
    Configuration menu
    Copy the full SHA
    5c81023 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a924ffc View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f092aae View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2022

  1. Configuration menu
    Copy the full SHA
    543ea7a View commit details
    Browse the repository at this point in the history
  2. toasty/samplers.py: fix the latlon tile sampler for images covering >…

    …180 degrees of longitude
    pkgw committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    0fb2d5f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3373e26 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4b95075 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8c45b1c View commit details
    Browse the repository at this point in the history
  6. toasty/fits_tiler.py: check if hips_{pixel_cut,data_range} are present

    In one test image I have, hipsgen produces a "BAD PIXEL CUT" warning and
    doesn't emit this information. (I think it's because the file has an
    integer data type.) Avoid the crash if this is the case.
    pkgw committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    07832f7 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    9fe7184 View commit details
    Browse the repository at this point in the history

Commits on Aug 25, 2022

  1. Setting distinct out dir for TOAST and TAN tiled FITS - to not overri…

    …de the other if it exists
    imbasimba committed Aug 25, 2022
    Configuration menu
    Copy the full SHA
    d3f950e View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    56d828b View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2022

  1. Configuration menu
    Copy the full SHA
    d0d18f4 View commit details
    Browse the repository at this point in the history
  2. toasty/collection.py: improve DASCH hack/fix

    As noted in the comments, my DASCH sample file that crosses the pole has
    WCS headers that seem to have another problem beyond their specification
    of the TPV polynomial distortions. I need to set LONPOLE explicitly to
    get the correct orientation on the sky.
    pkgw committed Aug 29, 2022
    Configuration menu
    Copy the full SHA
    19ef824 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0e6ceac View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2022

  1. toasty/merge.py: make tiles_to_process a set, not a list

    The "initiate readiness state" algorithm repeatedly tests whether
    positions are in the tiles_to_process set. If large numbers of tiles
    (tens of millions) are being processed, the algorithm basically fails
    because such tests have O(n) scaling with lists. With a set, things
    become tractable.
    
    There may well be further room for improvement here, but this one change
    makes an enormous difference for big problems.
    pkgw committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    3a77819 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8c07b23 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f5b85ad View commit details
    Browse the repository at this point in the history
  4. Cythonize the latlon_tile_filter function

    In processing a FITS-to-TOAST dataset going down to level 14, with about
    24 million tiles to process, I discovered that the tile filter function
    was a non-trivial bottleneck -- it took about 10 minutes just to count
    the tiles that needed to be processed! Some profiling showed that most
    of the time was actually in the Numpy min/max functions. Here we
    Cythonize the function and avoid the Numpy functions. My measurements
    indicate a speedup of a factor of around 20.
    pkgw committed Aug 31, 2022
    Configuration menu
    Copy the full SHA
    e9a8774 View commit details
    Browse the repository at this point in the history