Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to download assets with original filename #6

Closed
matthewhanson opened this issue Oct 24, 2022 · 5 comments · Fixed by #77
Closed

Add option to download assets with original filename #6

matthewhanson opened this issue Oct 24, 2022 · 5 comments · Fixed by #77
Assignees

Comments

@matthewhanson
Copy link
Member

Currently stactask will download assets and name the file as the <asset_key>., thereby keeping the extension, but possibly (probably) renaming the file.

Sometimes a subprocess will expect the file to be named the same as it was originally. Present as option.

@ircwaves ircwaves self-assigned this Jan 9, 2024
@ircwaves
Copy link
Member

ircwaves commented Jan 9, 2024

@matthewhanson @philvarner @gadomski -- I'd like to motion this move from "add an option" to "make original filenames The Behavior™". But any valid argument for f"{asset_key}.{ext}" file names being preferable will easily convince me otherwise.

And yes, I've run into some external science code which depends on filenames .... again.

@matthewhanson
Copy link
Member Author

This is supported by stac-asset, although don't think it's the default.
What we really need to do is drop fsspec here in favor or stac-download.

ircwaves added a commit to ircwaves/stac-task that referenced this issue Jan 10, 2024
@ircwaves
Copy link
Member

Ok, that makes sense. I'll keep a fork for immediate use, and consider #76.

@gadomski
Copy link
Member

But any valid argument for f"{asset_key}.{ext}" file names being preferable will easily convince me otherwise.

I'm not sure which should be the default, but one benefit of f"{asset_key}.{ext}" is that you're guaranteed to not have name clashes when downloading all assets to a single directory (as long as you don't have duplicate dictionary keys). There's cases for both — IMO the "I just want to see what's there" use-case wants key-based, whereas the "I'm using this in a data pipeline" use-case wants orig-file-name.

@ircwaves
Copy link
Member

name clashes when downloading all assets to a single directory

OK, that's the trick: asset-key (or any) file naming can run amuck if path_template doesn't separate items out by ${id}. Seems like the PR may be useful until #76 can be addressed. Incoming.

ircwaves added a commit to ircwaves/stac-task that referenced this issue Jan 10, 2024
ircwaves added a commit to ircwaves/stac-task that referenced this issue Jan 10, 2024
Closes stac-utils#6

Update CHANGELOG.md

ignore .python-version file for pyenv-virtualenv
ircwaves added a commit to ircwaves/stac-task that referenced this issue Jan 10, 2024
Closes stac-utils#6

Update CHANGELOG.md

ignore .python-version file for pyenv-virtualenv
ircwaves added a commit to ircwaves/stac-task that referenced this issue Jan 11, 2024
Closes stac-utils#6

Update CHANGELOG.md

ignore .python-version file for pyenv-virtualenv
gadomski added a commit that referenced this issue Jan 17, 2024
* keep original filenames

Closes #6

Update CHANGELOG.md

ignore .python-version file for pyenv-virtualenv

* remove keep_original_filenames from CLI

keep_original_filenames is task specific, not invocation specific

* update CHANGELOG.md

* Update CHANGELOG.md

---------

Co-authored-by: Pete Gadomski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants