Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance Point2Grid to filter quality control strings with config file options #2880

Closed
18 of 22 tasks
JohnHalleyGotway opened this issue May 8, 2024 · 2 comments · Fixed by #2984
Closed
18 of 22 tasks

Enhance Point2Grid to filter quality control strings with config file options #2880

JohnHalleyGotway opened this issue May 8, 2024 · 2 comments · Fixed by #2984
Assignees
@JohnHalleyGotway
Labels
MET: PreProcessing Tools (Point) priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Milestone
MET-12.0.0

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented May 8, 2024 •
edited
Loading

Describe the Enhancement

Note, the changes for this issue rename the existing point2grid “-qc” command line option as “-goes_qc”, although the old “-qc” option is still quietly supported.

While developing a wrapper for the Point2Grid tool via dtcenter/METplus#2540, a short-coming in the Point2Grid configuration file was discovered. Please see this issue comment for details. Put simply, the existing quality_mark_thresh config file option should be replaced by the obs_quality_inc and obs_quality_exc config file options supported for the Point-Stat and Ensemble-Stat tools.

Be sure to update the unit tests to demonstrate these options and update the documentation accordingly.

Time Estimate

1 day

Sub-Issues

Consider breaking the enhancement down into sub-issues.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2782544

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as the next official version or Backlog of Development Ideas
  • For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y.Z Development project for development toward the next official release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing requestor: NOAA/EMC NOAA Environmental Modeling Center alert: NEED ACCOUNT KEY Need to assign an account key to this issue MET: PreProcessing Tools (Point) priority: high High Priority labels May 8, 2024
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone May 8, 2024
@JohnHalleyGotway JohnHalleyGotway self-assigned this May 8, 2024
@JohnHalleyGotway JohnHalleyGotway changed the title Enhance Point2Grid to filter quality control strings specified in the config file Enhance Point2Grid to filter quality control strings with config file options May 8, 2024
@georgemccabe georgemccabe mentioned this issue May 13, 2024
Merged
15 tasks
@JohnHalleyGotway JohnHalleyGotway added priority: blocker Blocker and removed priority: high High Priority labels May 15, 2024
@JohnHalleyGotway
Copy link
Collaborator Author

This issue is blocking dtcenter/METplus#2540.

@georgemccabe georgemccabe added the required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone label Jun 4, 2024
@georgemccabe georgemccabe mentioned this issue Jun 4, 2024
Closed
22 tasks
JohnHalleyGotway added a commit that referenced this issue Jun 24, 2024
@JohnHalleyGotway
Per #2880, remove Point2Grid quality_mark_thresh config option and ad…
fe24056 
…d obs_quality_inc and obs_quality_exc from the config file.
JohnHalleyGotway added a commit that referenced this issue Jun 24, 2024
@JohnHalleyGotway
Per #2880, update point2grid docs and reformat whitespace throughout.
2ac2448
JohnHalleyGotway added a commit that referenced this issue Jun 24, 2024
@JohnHalleyGotway
Per #2880, remove quality_mark_thresh and add obs_quality_inc and obs…
c789a22 
…_quality_exc.
@JohnHalleyGotway JohnHalleyGotway moved this from 🟢 Ready to 🎯 Up Next in MET-12.0.0 Development Sep 5, 2024
@JohnHalleyGotway JohnHalleyGotway moved this from 🎯 Up Next to 🏗 In progress in MET-12.0.0 Development Sep 13, 2024
JohnHalleyGotway added a commit that referenced this issue Sep 18, 2024
@JohnHalleyGotway
Unrelated to #2880, fix formatting of this R-string which cannot incl…
2d49f68 
…ude any special formatting, such as \n.
JohnHalleyGotway added a commit that referenced this issue Sep 20, 2024
@JohnHalleyGotway
Per #2880, not working quite right yet but this is progress
deddf97
@JohnHalleyGotway
Copy link
Collaborator Author

Added logic in point2grid to filter by the obs_qty_inc/exc settings but I get a segfault whenever that code is exercised. Recommend debugging on seneca to see why the MetPointObsData::get_obs_qty(int index) segfaults. Presumably, the obs_qids array it is references is not populated... or something like that.

JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, add a write_css(IntArray) utility function.
baa5af1
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, update NcPointObsData class to read the obs quality values…
cbf60bf 
… from the input file.
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, update the log message about the quality control filter op…
6dea58e 
…tions applied.
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, tweak log messages.
6ad7d73
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, tweak log messages.
73ae273
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, add a Point2Grid unit test to demonstrate using the obs_qu…
481bd99 
…ality_inc and obs_quality_exc options.
@JohnHalleyGotway JohnHalleyGotway linked a pull request Oct 1, 2024 that will close this issue
Feature #2880 point2grid qc #2984
Merged
17 tasks
@JohnHalleyGotway JohnHalleyGotway removed the alert: NEED ACCOUNT KEY Need to assign an account key to this issue label Oct 1, 2024
@JohnHalleyGotway JohnHalleyGotway moved this from 🏗 In progress to 🔎 In review in MET-12.0.0 Development Oct 1, 2024
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, declare these get accessor functions as const to avoid Son…
b082e7d 
…arQube code smell.
JohnHalleyGotway added a commit that referenced this issue Oct 1, 2024
@JohnHalleyGotway
Per #2880, many changes to the vx_nc_obs library and point2grid appli…
72d9653 
…cation to replace dynamically allocated memory with STL vectors to satisfy SonarQube code smells.
JohnHalleyGotway added a commit that referenced this issue Oct 2, 2024
@JohnHalleyGotway
Per #2880, fix compilation error.
9e9a171
JohnHalleyGotway added a commit that referenced this issue Oct 2, 2024
@JohnHalleyGotway
Per #2880, fix compilation error
b1e06dc
JohnHalleyGotway added a commit that referenced this issue Oct 2, 2024
@JohnHalleyGotway
Per #2880, revert skip_times back to vector<bool> since I wasn't post…
0c5d72b 
…ive the casting (int *) to (bool *) would actually work in the way I expect. This is safer.
JohnHalleyGotway added a commit that referenced this issue Oct 2, 2024
@JohnHalleyGotway
Per #2880, rename the -qc command line option as -goes_qc, but still …
ba5f4a4 
…quietly support -qc
JohnHalleyGotway added a commit that referenced this issue Oct 3, 2024
@JohnHalleyGotway
Per #2880, based on guidance from @hsoh, reset the var_cell_mapping v…
315d409 
…ector for each loop iteration.
JohnHalleyGotway added a commit that referenced this issue Oct 3, 2024
@JohnHalleyGotway
Feature #2880 point2grid qc (#2984)
b8a355c 
* Per #2880, remove Point2Grid quality_mark_thresh config option and add obs_quality_inc and obs_quality_exc from the config file.

* Per #2880, update point2grid docs and reformat whitespace throughout.

* Per #2880, remove quality_mark_thresh and add obs_quality_inc and obs_quality_exc.

* Unrelated to #2880, fix formatting of this R-string which cannot include any special formatting, such as \n.

* Per #2880, not working quite right yet but this is progress

* Per #2880, add a write_css(IntArray) utility function.

* Per #2880, update NcPointObsData class to read the obs quality values from the input file.

* Per #2880, update the log message about the quality  control filter options applied.

* Per #2880, tweak log messages.

* Per #2880, tweak log messages.

* Per #2880, add a Point2Grid unit test to demonstrate using the obs_quality_inc and obs_quality_exc options.

* Per #2880, declare these get accessor functions as const to avoid SonarQube code smell.

* Per #2880, many changes to the vx_nc_obs library and point2grid application to replace dynamically allocated memory with STL vectors to satisfy SonarQube code smells.

* Per #2880, fix compilation error.

* Per #2880, fix compilation error

* Per #2880, revert skip_times back to vector<bool> since I wasn't postive the casting (int *) to (bool *) would actually work in the way I expect. This is safer.

* Per #2880, rename the -qc command line option as -goes_qc, but still quietly support -qc

* Per #2880, based on guidance from @hsoh, reset the var_cell_mapping vector for each loop iteration.
@github-project-automation github-project-automation bot moved this from 🔎 In review to 🏁 Done in MET-12.0.0 Development Oct 3, 2024
JohnHalleyGotway added a commit that referenced this issue Oct 14, 2024
@JohnHalleyGotway
Per #2880, remove unneeded comment
a569129
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JohnHalleyGotway JohnHalleyGotway

Labels
MET: PreProcessing Tools (Point) priority: blocker Blocker requestor: NOAA/EMC NOAA Environmental Modeling Center required: FOR OFFICIAL RELEASE Required to be completed in the official release for the assigned milestone type: enhancement Improve something that it is currently doing
Projects
MET-12.0.0 Development
Status: 🏁 Done
Milestone
MET-12.0.0
Development

Successfully merging a pull request may close this issue.

Feature #2880 point2grid qc dtcenter/MET
2 participants
@JohnHalleyGotway @georgemccabe