Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loading only a small region #3

Open
akasom89 opened this issue Oct 28, 2024 · 3 comments
Open

loading only a small region #3

akasom89 opened this issue Oct 28, 2024 · 3 comments

Comments

@akasom89
Copy link

Thanks for developing such a great tool.

I am interested to do a time-series analysis over several years for a small region on my local system.

Is it possible to download a one-degree by one-degree region ONLY using this library(instead of downloading the whole scene for each observation)?

Thanks

@ghiggi
Copy link
Owner

ghiggi commented Oct 28, 2024

Unfortunately, despite the conceptual simplicity, this is a difficult task to perform without downloading whole scenes ...
However, you could try using kerchunk or virtualizarr to create on your local system the references for all files and then just read on the fly all the data chunk of interest from the S3 bucket.

@akasom89
Copy link
Author

Thanks @ghiggi

You are exactly right. It seems much more challenging than I thought. Do you know of any tutorial or documentation with the same goal?
I would greatly appreciate it if you could share any.

@ghiggi
Copy link
Owner

ghiggi commented Dec 18, 2024

Not really sorry.
I would suggest to adapt the code in the VirtualiZarr Usage section: https://virtualizarr.readthedocs.io/en/stable/index.html#usage
Instead of glob.glob('/my/files*.nc') pass the filepaths returned by goes_api.find_files with protocol="s3" and connection_type="bucket".
See tutorial here on how to search files: https://github.com/ghiggi/goes_api/blob/main/tutorials/01_find_utility.py
Let me know how it works ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants