Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Partial rewrite of the MOCServer module #3139

Merged
merged 13 commits into from
Jan 23, 2025
Merged

Conversation

ManonMarchand
Copy link
Member

@ManonMarchand ManonMarchand commented Nov 27, 2024

Hello astroquery,

This is a quite big rewrite of the MOS Server module.

The two main motivations behind it were:

  • there was a bug that using query_region with a MOC would write a real file on the people's current folder. This file would be named moc.fits and could potentially overwrite a pre-existing file with the same name silently.
  • the MOC server upstream now works with time MOCs and space-time MOCs, so these features were needed here

The actual changes in this PR

  • new method query_hips allows to filter only hips in a shortcut rather than having to remember all the time to have to type hips_frame=* in the meta_data (convenience method)
  • before, you could only search by region or by meta_data. This was unnecessary, and now the two methods are exactly the same and allow to filter on region and to add a criteria. I was wondering whether find_datasets should be deprecated
  • new parameters casesensitive and coordinate_system allow to chose whether the query will respect the case, and to chose a specific system for the datasets (ex: mars, venus, sky)
  • new methods list_coordinate_systems that prints the currently available systems (can evolve as providers post new datasets)
  • new method list_fields to see the possible fields (there are 134 today) sorted by occurrence and with an example
  • the default returned table now have a sub-selection of fields rather than the whole 134 ones (this speeds up the module considerably)
  • return_moc is no longer only a boolean. It keeps it former behavior (True means Space MOC) but it can now also take the values smoc, tmoc, or stmoc to specify the kind of MOC that should be returned
  • in query_region, the region argument can now also be a mocpy.TimeMOC and a mocpy.STMOC on top of the previously accepted classes
  • removed the regions module from the mandatory dependencies

Changes while I was there

@pep8speaks
Copy link

pep8speaks commented Nov 27, 2024

Hello @ManonMarchand! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2025-01-22 23:00:36 UTC

@ManonMarchand ManonMarchand force-pushed the mocserver branch 4 times, most recently from 68d1db0 to 065b393 Compare November 27, 2024 15:43
@ManonMarchand

This comment was marked as outdated.

Copy link

codecov bot commented Nov 27, 2024

Codecov Report

Attention: Patch coverage is 93.45794% with 7 lines in your changes missing coverage. Please review.

Project coverage is 67.57%. Comparing base (b177f06) to head (0bed33f).
Report is 15 commits behind head on main.

Files with missing lines Patch % Lines
astroquery/mocserver/core.py 93.39% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3139      +/-   ##
==========================================
+ Coverage   67.42%   67.57%   +0.14%     
==========================================
  Files         229      229              
  Lines       18612    18593      -19     
==========================================
+ Hits        12550    12565      +15     
+ Misses       6062     6028      -34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bsipocz bsipocz added this to the v0.4.8 milestone Nov 27, 2024
@ManonMarchand ManonMarchand marked this pull request as draft December 2, 2024 16:47
@ManonMarchand
Copy link
Member Author

Switching to draft as I asked for a review internally in CDS

Copy link
Contributor

@tboch tboch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR.
My main comment is about space_sys which I don't like much, and about 'C' which really should be 'sky'.

I see no harm in starting to have a deprecation message for find_datasets.
Should we have a new get_coverage or get_moc method which would be cleaner than the return_moc parameter? This can be added in a later PR of course.

astroquery/mocserver/__init__.py Outdated Show resolved Hide resolved
astroquery/mocserver/__init__.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
@ManonMarchand ManonMarchand force-pushed the mocserver branch 2 times, most recently from 2a04790 to 8a5937d Compare December 19, 2024 13:06
@ManonMarchand ManonMarchand marked this pull request as ready for review December 19, 2024 13:07
@ManonMarchand
Copy link
Member Author

ManonMarchand commented Dec 19, 2024

I applied Thomas's suggestions. This is ready to be reviewed 🙂

@bsipocz
Copy link
Member

bsipocz commented Dec 19, 2024

I'll get back to this in the new year. Thanks, Manon and Happy Holidays!

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. I have a few very minor comments. The two must be done are the rebase -- I'm sorry for not getting this into 0.4.8 -- and adding the doctest require to the documentation as the doctests fail for the documentation when there is no mocpy present.

astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
astroquery/mocserver/core.py Outdated Show resolved Hide resolved
docs/mocserver/mocserver.rst Outdated Show resolved Hide resolved
docs/mocserver/mocserver.rst Outdated Show resolved Hide resolved
docs/mocserver/mocserver.rst Outdated Show resolved Hide resolved
docs/mocserver/mocserver.rst Outdated Show resolved Hide resolved
docs/mocserver/mocserver.rst Show resolved Hide resolved
@ManonMarchand ManonMarchand force-pushed the mocserver branch 3 times, most recently from 1c74e2b to 833ca5d Compare January 17, 2025 15:39
@ManonMarchand
Copy link
Member Author

ManonMarchand commented Jan 17, 2025

Hello Brigitta,

Thanks for the review!

No worries for the 0.4.8, we'll get the changes in the next release.

I think I addressed all you comments except the one about meta_data that could be renamed either criteria or dataset_filter depending on what you prefer. I'll also patch the docstring to remove Algebraic once we decide on the new name.

@bsipocz
Copy link
Member

bsipocz commented Jan 17, 2025

I think I addressed all you comments except the one about meta_data that could be renamed either criteria or dataset_filter depending on what you prefer. I'll also patch the docstring to remove Algebraic once we decide on the new name.

Yeap, I'm not sure anymore. I like both your suggestions with a very slight preference for criteria. On the other hand, meta_data is not a new kwarg as I first assumed it was, so I'm now unsure if the rename would be worth the effort of deprecation.

@ManonMarchand
Copy link
Member Author

It's new in query_hips and query_region. The only method that already had meta_data is find_datasets, which is deprecated now anyhow. So I switched to criteria in the two current methods, but kept the old name in the old method.

@bsipocz
Copy link
Member

bsipocz commented Jan 20, 2025

Do you prefer this to go out in 0.4.9 later this week, or keep in main after merge for a little longer?

@ManonMarchand
Copy link
Member Author

I would be better if this could fit in 0.4.9. The bug on overwritting files is really critical

Copy link
Member

@bsipocz bsipocz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's go ahead with this one. It needs a rebase, and a minor edit to the changelog as I don't think the warning renders out as expected.

- Fix query by MOC that would write a file ``moc.fits`` where the method was executed in
overwriting mode (potentially deleting data if there was a conflicting file) [#3139]

- [:warning: BREAKING] Returned tables now have a default list of fields instead of the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This warning won't work here, I'll edit this out during rebase.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, I leave this to the before-release changelog cleanup, and go ahead with the PR as is.

Thanks @ManonMarchand!

mostly achieved by looking at a small MOC in test_moc_order_param rather than the whole sky
for tables, we use the fact that Table accepts dictionnaries. For MOCs, the dictionnary parsing has been improved in MOCpy > 0.12 and does not require to remove empty orders anymore
this is mainly motivated by the new support for time mocs and space-time mocs upstream
allows to work in the browser with wasm-based python implementations
this makes this parameter different from the one the server understands, but it's more consistant with the other possible values for spacesys
note that this change is only user-facing, the upstream name is spacesys
everything done with list_fields
@bsipocz bsipocz merged commit 013261c into astropy:main Jan 23, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants