Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File_Writer for DigitalSurf files #280

Merged
merged 24 commits into from
Jul 4, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1bb6728
support for .sur .pro export, bugfixes
Attolight-NTappy Jun 20, 2024
72aa59c
support for .sur .pro export, bugfixes
Attolight-NTappy Jun 20, 2024
3bc3901
Merge branch 'hyperspy:main' into save_sur
Attolight-NTappy Jun 20, 2024
2d8fdc8
merged remote
Attolight-NTappy Jun 20, 2024
5c9b4ff
Fixing bugs, making doc
Attolight-NTappy Jun 21, 2024
cc52a66
Fix RGBImageSeries, enhance tests, improve doc
Attolight-NTappy Jun 24, 2024
1e6e4b9
increase codecov, fix bugs
Attolight-NTappy Jun 25, 2024
f968fe5
Fix bug causing error from untitled metadata
Attolight-NTappy Jun 25, 2024
c6587c9
Linting using black
Attolight-NTappy Jun 25, 2024
d914fda
Increase codecov
Attolight-NTappy Jun 25, 2024
94f0ae1
Suppress implicit return in _is_data_scaleint, also codestyle
Attolight-NTappy Jun 25, 2024
acfb7f8
refactor n_objects_to_read, remove useless binary get / set defaults …
Attolight-NTappy Jun 27, 2024
e3cb18a
Fixed typos in doc
Attolight-NTappy Jun 27, 2024
41f4305
codecov, fix binary images, add RGBSurface tests
Attolight-NTappy Jun 28, 2024
3e4fdb9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 28, 2024
7cfd261
Update doc/user_guide/supported_formats/digitalsurf.rst
Attolight-NTappy Jun 30, 2024
61b932b
Increase codecov, code cleanup
Attolight-NTappy Jul 1, 2024
d541780
Merge branch 'save_sur' of https://github.com/Attolight-NTappy/rosett…
Attolight-NTappy Jul 1, 2024
e674433
Cleanup obsolete testfile
Attolight-NTappy Jul 1, 2024
8440db1
Exposed parse_metadata to public api
Attolight-NTappy Jul 4, 2024
3d84269
docstring updates
Attolight-NTappy Jul 4, 2024
3319edd
More linting
Attolight-NTappy Jul 4, 2024
53341dd
ruff fixes
ericpre Jul 4, 2024
1f5e4c5
Improve changelog entry
ericpre Jul 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 50 additions & 10 deletions doc/user_guide/supported_formats/digitalsurf.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,56 @@
DigitalSurf format (SUR & PRO)
------------------------------

The ``.sur`` and ``.pro`` files are a format developed by the digitalsurf company to handle various types of
scientific measurements data such as profilometer, SEM, AFM, RGB(A) images, multilayer
surfaces and profiles. Even though it is essentially a surfaces format, 1D signals
are supported for spectra and spectral maps. Specifically, this file format is used
by Attolight SA for its scanning electron microscope cathodoluminescence
(SEM-CL) hyperspectral maps. Metadata parsing is supported, including user-specific
metadata, as well as the loading of files containing multiple objects packed together.

The plugin was developed based on the MountainsMap software documentation, which
contains a description of the binary format.
``.sur`` and ``.pro`` is format developed by digitalsurf to import/export data in their MountainsMap scientific
analysis software. Target datasets originally result from (micro)-topography and imaging instruments: SEM, AFM,
profilometer. RGB(A) images, multilayer surfaces and profiles are also supported. Even though it is essentially
a surfaces format, 1D signals are supported for spectra and spectral maps. Specifically, this is the fileformat
used by Attolight SA for its scanning electron microscope cathodoluminescence (SEM-CL) hyperspectral maps. This
plugin was developed based on the MountainsMap software documentation.

Support for loading ``.sur`` and ``.pro`` datasets is complete, including parsing of user/customer-specific
metadata, and opening of files containing multiple objects. Some rare specific objects (e.g. force curves)
are not supported, due to no example data being available. Those can be added upon request and providing of
example datasets. Heterogeneous data can be represented in ``.sur`` and ``.pro`` objects, for instance
floating-point/topography and rgb data can coexist along the same navigation dimension. Those are casted to
a homogeneous floating-point representation upon loading.

Support for data saving is partial as ``.sur`` and ``.pro`` do not support all features of hyperspy signals.
First, they have limited dimensionality. Up to 3d data arrays with either 1d (series of images) or 2d
(hyperspectral studiable) navigation space can be saved. Also, ``.sur`` and ``.pro`` do not support non-uniform
axes and saving of models. Finally, ``.sur`` / ``.pro`` linearize intensities along a uniform axis to enforce
an integer-representation of the data (with scaling and offset). This means that export from float-type hyperspy
signals is inherently lossy.

Within these limitations, all features from ``.sur`` and ``.pro`` fileformats are supported, notably data
compression and setting of custom metadata. The file writer splits a signal into the suitable digitalsurf
dataobject primarily by inspecting its dimensions and its datatype, ultimately how various axes and signal
quantity are named. The criteria are listed here below:

+-----------------+---------------+------------------------------------------------------------------------------+
| Nav. dimension | Sig dimension | Extension and MountainsMap subclass |
+=================+===============+==============================================================================+
| 0 | 1 | ``.pro``: Spectrum (based on axes name), Profile (default) |
+-----------------+---------------+------------------------------------------------------------------------------+
| 0 | 2 | ``.sur``: BinaryImage (based on dtype), RGBImage (based on dtype), |
| | | Surface (default), |
+-----------------+---------------+------------------------------------------------------------------------------+
| 1 | 0 | ``.pro``: same as (1,0) |
Attolight-NTappy marked this conversation as resolved.
Show resolved Hide resolved
+-----------------+---------------+------------------------------------------------------------------------------+
| 1 | 1 | ``.pro``: Spectrum Serie (based on axes name), Profile Serie (default) |
+-----------------+---------------+------------------------------------------------------------------------------+
| 1 | 2 | ``.sur``: RGBImage Serie (based on dtype), Surface Series (default) |
+-----------------+---------------+------------------------------------------------------------------------------+
| 2 | 0 | ``.sur``: same as (0,2) |
+-----------------+---------------+------------------------------------------------------------------------------+
| 2 | 1 | ``.sur``: hyperspectralMap (default) |
+-----------------+---------------+------------------------------------------------------------------------------+

Axes named one of ``Wavelength``, ``Energy``, ``Energy Loss``, ``E``, are considered spectral, and quantities
named one of ``Height``, ``Altitude``, ``Elevation``, ``Depth``, ``Z`` are considered surface. The difference
between Surface and IntensitySurface stems from the AFM / profilometry origin of MountainsMap. "Surface" has
the proper meaning of an open boundary of 3d space, whereas "IntensitySurface" is a mere 2D mapping of an arbitrary
quantity.

API functions
^^^^^^^^^^^^^
Expand Down
Loading
Loading