Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

De MRC Auto get Metadata #311

Merged
merged 18 commits into from
Nov 12, 2024
Merged

De MRC Auto get Metadata #311

merged 18 commits into from
Nov 12, 2024

Conversation

CSSFrancis
Copy link
Member

@CSSFrancis CSSFrancis commented Sep 16, 2024 •
edited
Loading

Description of the change

A few sentences and/or a bulleted list to describe and motivate the change:

  • Fix a bug with the newest version of DE Server (Some of the metadata "Keys" were changes)
  • Add auto determination of the metdata_info.txt file based on the naming scheme.

Progress of the PR

  • Change implemented (can be split into several points),
  • update docstring (if appropriate),
  • update user guide (if appropriate),
  • add a changelog entry in the upcoming_changes folder (see upcoming_changes/README.rst),
  • Check formatting of the changelog entry (and eventual user guide changes) in the docs/readthedocs.org:rosettasciio build of this PR (link in github checks)
  • add tests,
  • ready for review.

Minimal example of the bug fix or the new feature

from rsciio.mrc import file_reader
file_reader("your_mrc_file.mrc") # new automatically finds metadata file
file_reader("your_mrc_file.mrc", metadata_file = "your_mrc_file_info.txt ") # old

Note that this example can be useful to update the user guide.

github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 16, 2024
View reviewed changes
rsciio/mrc/_api.py Fixed Show fixed Hide fixed
@codecov Codecov
Copy link

codecov bot commented Sep 16, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 93.68421% with 6 lines in your changes missing coverage. Please review.

Project coverage is 87.88%. Comparing base (88923bf) to head (d7569e7).
Report is 25 commits behind head on main.

Files with missing lines Patch % Lines
rsciio/mrc/_api.py 93.68% 4 Missing and 2 partials ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #311      +/-   ##
==========================================
+ Coverage   87.84%   87.88%   +0.03%     
==========================================
  Files          85       85              
  Lines       11180    11241      +61     
  Branches     2280     2294      +14     
==========================================
+ Hits         9821     9879      +58     
- Misses        860      863       +3     
  Partials      499      499

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-advanced-security[bot]
github-advanced-security bot found potential problems Sep 20, 2024
View reviewed changes
rsciio/mrc/_api.py Fixed Show fixed Hide fixed
@CSSFrancis CSSFrancis force-pushed the de_mrc_auto branch 2 times, most recently from 3ab2923 to 6d54935 Compare October 8, 2024 19:01
@CSSFrancis CSSFrancis requested review from jlaehne and ericpre October 8, 2024 19:27
@jlaehne
Copy link
Member

jlaehne commented Oct 8, 2024

Is it possible to downscale the size of the movie file by e.g. binning?

@CSSFrancis
Copy link
Member Author

Is it possible to downscale the size of the movie file by e.g. binning?

Sure if that is helpful I can do that.

@jlaehne
Copy link
Member

jlaehne commented Oct 10, 2024

Is it possible to downscale the size of the movie file by e.g. binning?

Sure if that is helpful I can do that.

We download only for testing, but still 32 MB for one data format is quite heavy. So far we have 137 MB of data and the edax format is the only other one that exceeds 30 MB where it could not be reduced (usually up to 10). So wherever we can be more slim without compromising testing, we should do so: https://rosettasciio.readthedocs.io/en/latest/contributing.html#making-test-data-files

@ericpre ericpre added this to the v0.7 milestone Oct 20, 2024
ericpre
ericpre reviewed Oct 20, 2024
View reviewed changes
Copy link
Member

@ericpre ericpre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CSSFrancis, when reviewing this PR, I start to wonder if the DE MRC metadata should be documented in the https://hyperspy.org/rosettasciio/file_specification/index.html section? What do you think?

doc/user_guide/supported_formats/mrc.rst Outdated Show resolved Hide resolved
doc/user_guide/supported_formats/mrc.rst Show resolved Hide resolved
rsciio/mrc/_api.py Show resolved Hide resolved
rsciio/mrc/_api.py Show resolved Hide resolved
rsciio/mrc/_api.py Show resolved Hide resolved
@CSSFrancis
Copy link
Member Author

@CSSFrancis, when reviewing this PR, I start to wonder if the DE MRC metadata should be documented in the https://hyperspy.org/rosettasciio/file_specification/index.html section? What do you think?

I added references to where the .mrc file format is defined :)

ericpre
ericpre reviewed Oct 22, 2024
View reviewed changes
doc/user_guide/supported_formats/mrc.rst Outdated Show resolved Hide resolved
rsciio/mrc/_api.py
"Non-raster scans are not fully supported yet. Please raise an issue on GitHub"
" if you need this feature."
)
if in_stem_mode == -1:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding my previous comment on adding information in file specification, I was thinking of this type of metadata, which are not documented in the MRC 2014 format.
Then these can be used for the review! :)

This doesn't need to be in this PR but longer term, we should aim to document format specification for the community. For example, in the past, the TIA-SER format was very documented online (the link is most likely still in the code) but this is not available anymore and this information is possibly lost or not openly accessible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good point, although the metadata there is basically whatever information we have from the microscope/ our camera. I'm working to standardize it a bit and add some documentation, but I might do that a bit later.

We can also write hspy which in the longer term is a bit more desirable with the eventual goal to support writing into .zspy.

ericpre
ericpre reviewed Oct 22, 2024
View reviewed changes
rsciio/mrc/_api.py Outdated Show resolved Hide resolved
rsciio/mrc/_api.py Outdated Show resolved Hide resolved
@ericpre
Copy link
Member

ericpre commented Oct 28, 2024

pre-commit.ci autofix

ericpre
ericpre approved these changes Oct 28, 2024
View reviewed changes
Copy link
Member

@ericpre ericpre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tweak the documentation - please have a look and merge if you are happy with these changes.

@ericpre
Copy link
Member

ericpre commented Oct 28, 2024

One workflow fails, because some test fails which are present on main are missing from this branch - a rebase should fix it or it is merge to main.

@ericpre
Copy link
Member

ericpre commented Nov 7, 2024

There is a genuine test failure due to the # character in the filename that doesn't play well with downloading the test files using pooch.

It is trying to download from the following url:
https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc

while the actual path is https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20%230_sum.mrc

In the pooch registry, the file are saved as:

'mrc/20241021_00405_0_Virt #0_sum.mrc' 3f798cf43ff1f3c39750b379a22962a4c6d2d546f1298461fabc6bced0cc876f
'mrc/20241021_00405_1_Virt #1_sum.mrc' 3f798cf43ff1f3c39750b379a22962a4c6d2d546f1298461fabc6bced0cc876f
'mrc/20241021_00405_ext1_Ext #1.mrc' 4962c3ead6547d819e22447765e93e357911bacd2905191d97d339b5cc0dd1dc

I added a third to use the custom url functionality but I think that the correct fix is that pooch parse the url. The update of the registry will fails because of the third columns and pre-commit CI will fail.

pytest log 
=========================== short test summary info ============================
ERROR tests/test_import.py::test_import_version - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_rsciio_dir - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_rsciio_utils - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_import_all - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_format_name_aliases - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin0] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin1] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin2] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin3] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin4] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin5] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin17] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin18] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin19] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin20] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin21] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin22] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin23] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin24] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin25] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin26] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin27] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin28] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin29] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin30] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin31] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin32] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[dens] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[digitalsurf] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[impulse] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[jobinyvon] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[msa] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[netcdf] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[pantarhei] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[phenom] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[protochips] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[renishaw] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_lazy_not_implemented.py::test_lazy_not_implemented[trivista] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_set_log_level.py::test_set_log_level[CRITICAL] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
Error: ests/test_set_log_level.py::test_set_log_level[ERROR] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
Warning: ts/test_set_log_level.py::test_set_log_level[WARNING] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_set_log_level.py::test_set_log_level[INFO] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_set_log_level.py::test_set_log_level[DEBUG] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_set_log_level.py::test_set_log_level[NOTSET] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c_dask - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_f - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_corder_from_c_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_cordermask_from_cmasked - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_rgbx2regular_array_cordermask_from_cmasked_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_c - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_f - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_corder_from_c_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_cordermask_from_cmasked - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_rgbtools.py::TestRGBTools::test_regular_array2rgbx_cordermask_from_cmasked_slices - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_msxml_sanitization - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_default_x2d - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_skip_interchild_text_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_concat_interchild_text_val_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_list_interchild_text_val_flatten - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_wrong_type_x2d_initiation - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_fail - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_dtype - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_extra_dict_ok - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_sarray - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_partial_sarray - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_type_cast_ok - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_type_cast_invalid - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_string_cut - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_array1 - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_array2 - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_d2s_arrayX - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_serial_date_to_ISO_format - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_ISO_format_to_serial_date - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_datetime_to_serial_date - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_get_date_time_from_metadata - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape0] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape1] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape2] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/utils/test_utils.py::test_get_chunk_slice[shape3] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin6] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin7] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin8] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin9] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin10] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin11] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin12] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin13] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin14] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin15] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
ERROR tests/test_import.py::test_dir_plugins[plugin16] - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/CSSFrancis/rosettasciio/raw/de_mrc_auto/rsciio/tests/data/mrc/20241021_00405_0_Virt%20#0_sum.mrc
================== 37 skipped, 91 errors in 81.02s (0:01:21) ===================

@CSSFrancis
Copy link
Member Author

@ericpre What is the action item here? I can remove the # from the file names. That is actually just a default for the virtual detector name so it shouldn't effect any of the auto laoding.

@CSSFrancis
Copy link
Member Author

pre-commit.ci autofix

@CSSFrancis
Copy link
Member Author

I'll avoid self merging but @ericpre feel free to merge if you think this is good.

@ericpre ericpre merged commit 40b8570 into hyperspy:main Nov 12, 2024
32 checks passed
@ericpre
Copy link
Member

ericpre commented Nov 12, 2024

Renaming the files is the most simplest here and the issue with # in the filename should be handle in pooch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ericpre ericpre ericpre approved these changes

@jlaehne jlaehne Awaiting requested review from jlaehne

Assignees
No one assigned
Labels
type: enhancement
Projects
None yet
Milestone
v0.7
Development

Successfully merging this pull request may close these issues.
3 participants
@CSSFrancis @jlaehne @ericpre