Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to load either processed or unprocessed from Oxford Instrument's H5OINA file #692

Closed
hakonanes opened this issue Oct 11, 2024 · 5 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@hakonanes
Copy link
Member

hakonanes commented Oct 11, 2024

So about importing .h5oina data in Kikuchipy, it seems that the Dataname 'Processed Patterns' is still valid. But once there are also Unprocessed Patterns stored in the h5, it seems to me that the Kikuchipy reader starts to load these into memory, even when Lazy=True. Does the reader by default import all other data into memory, accept for the 'Processed Patterns', when Lazy=True?

Sorry for the spam, but I solved this issue, at least temporarily. I found the line that specifies that the pattern dataset should not be read into memory (I think), and added "Unprocessed Patterns" to it. I changed line 99 in oxford_h5ebsd.py to the following:
dd = _hdf5group2dict(group["EBSD/Data"], data_dset_names=[self.patterns_name, "Unprocessed Patterns"])
This solves the issue for me.
Perhaps this is not a suitable permanent solution though. I guess the user would need to have the option to import unprocessed patterns, if they like, instead of processed patterns...

Originally posted by @Tijmenvermeij in #690 (comment)


I did not think about H5OINA files storing both processed and unprocessed patterns when writing the reader. Thank you for spotting the bug, @Tijmenvermeij!

The solution is, as you say, to give the user the opportunity to load either of them (or both, if they load consecutively). The other, if present, should not be read into memory.

We should look at releasing a v0.11.0 0.12.0 within the next couple weeks with these changes.

@hakonanes hakonanes added bug Something isn't working enhancement New feature or request labels Oct 11, 2024
@hakonanes hakonanes added this to the v0.11.0 milestone Oct 11, 2024
@Tijmenvermeij
Copy link

For loading "Unprocessed Patterns" from .h5oina files, it probably also makes sense that the static BG is also loaded from the file. I'm not sure if this is currently done already...

@hakonanes
Copy link
Member Author

The static background is loaded

# --- Static background
scan_dict["static_background"] = hd.get("Processed Static Background")

and available in the property EBSD.static_background. This is also mentioned in the IO user guide.

@hakonanes
Copy link
Member Author

hakonanes commented Nov 20, 2024

@Tijmenvermeij, I made a fix for the processed/unprocessed patterns from H5OINA in https://github.com/hakonanes/kikuchipy/tree/692-h5oina-un-processed-patterns and #702.

Could you try it out? python -m pip install kikuchipy@git+https://github.com/hakonanes/kikuchipy.git@692-h5oina-un-processed-patterns

@hakonanes
Copy link
Member Author

Should be fixed in #702.

@hakonanes
Copy link
Member Author

@Tijmenvermeij, the latest 0.11.1 release now has this option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants