Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add capability to calculate FCs in MTH5 #271

Open
4 of 7 tasks
kkappler opened this issue Dec 20, 2024 · 1 comment
Open
4 of 7 tasks

Add capability to calculate FCs in MTH5 #271

kkappler opened this issue Dec 20, 2024 · 1 comment

Comments

@kkappler
Copy link
Collaborator

kkappler commented Dec 20, 2024

Currently FCs are generated only in aurora. There are two pathways to get FCs in aurora, one that follows Gary's code and another that uses a simpler, scipy built-in. The simpler scipy built-in method could be moved into mth5.

This would allow for

  • most applications of FC generation
  • end-to-end testing of the synthetic data from time series to Spectrograms, and eventually to features.

This requires:

  • adding prewhitening methods for spectral leakage
  • adding a core stft module
  • migrating the stft tests of synthetic data from aurora to mth5
    • To run the stft in mth5 we also need to make standalone FCDecimation objects from mt_metadata alone (not using aurora)
    • These FCDecimations currently only support a single decimation level by default. They chould be modified in one of two ways
      • allow for a standard, baseline, 4x decimation scheme like aurora uses
      • create time series decimation levels within the mth5 test datasets
kkappler added a commit that referenced this issue Jan 3, 2025
- still needs more tests
- updated task list in aurora issue 364
@kkappler
Copy link
Collaborator Author

kkappler commented Jan 14, 2025

This is more or less working now. However, there is an issue that has come to light after updating the metadata structures in the FC layers. The issue is that FCs in an mth5 built before these changes will have an incompatible metadata structure with the new metadata.

This could potentially be worked around with some getter/setter methods mapping the legacy metadata values to the new ones in FCDecimation metadata, but a more general solution would be nice. While breaking changes in mt_metadata will be avoided, it is plausible that they will happen from time to time. What would be nice would be to have a tool that can

  • access the old metadata, translate it to the new, and then update the metadata to work with the new data-structure.
  • barring that, maybe copy the legacy metadata to somewhere in mt_metadata and use that to read the old ...
  • it seems that tagging the metadata with the version of code that created it is going to become relevant here.

In any case, metadata access is not a problem, metadata access via mt_metadata is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant