Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor LS8 Query Code #7

Open
3 of 6 tasks
espg opened this issue Apr 21, 2021 · 0 comments
Open
3 of 6 tasks

Refactor LS8 Query Code #7

espg opened this issue Apr 21, 2021 · 0 comments

Comments

@espg
Copy link
Contributor

espg commented Apr 21, 2021

Currently, LS8 query runs as follows:

  • Read in all LS8 corner points
  • Create BallTree for level 1 'coarse' query'
  • Use point in object on candidates for 'refinement'

Drawbacks on this current method:

  1. Reading LS8 excel sheet is slow, and it has to be done on every query. Better would be to create a binary data structure (.npy or .h5)
  2. The query isn't actually too accurate. We discussed this with @whyjz 's plots that compared the vector footprint to a sample data file. This is because the orbital overpass drifts-- so we can't use a static index for the refinement stage and expect high precision.
  3. The geopandas package pulls in a high amount of dependencies, including pysal, which pulls in more dependencies.

Suggested solution:

  • Use @jonathan-taylor 's contains code on the xlsx Landsat8 corner data as the first 'coarse' pass.
  • Since the above code doesn't expect a geometry, convert the LS8 index into a binary index (should be light, possibly under a meg)
  • To make the 'coarse' pass inclusive, we can pa d the query a few kilometres.
  • For the fine scale 'refinement', use boto3 / gdal to read the specific COG headers off of AWS for the granule level metadata, and update the footprints using this data. The metadata should be accurate to the actual corners and center point 'as collected', and will be different from the static catalog.
  • Filter based on the updated refinement metadata fetch

Other notes: the WGS grid is valid for multiple Landsat missions, so we may be able to query more of the catalog.

To Do:

  • Import contains and associated code
  • Unit tests for above
  • Calculate bearing from existing catalog @espg (needed for above code)
  • Build and upload new static binary index
  • Update existing methods to use new query method above
  • Add example, math, and documentation to Docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant