Improvements to HQ residual solves in CG #1387
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…solves requesting a HQ tolerance
…s and only trigger when the HQ residual has dropped by an appropriate amount
…ix/heavy-quark-restart
|
Update: @detar is doing a set of thorough correctness checks, thank you!! |
- Rename the workflow to include rocm in the name - Remove unused build options - Add more precisions to build workflow
…ce/quda into hotfix/heavy-quark-restart
|
@detar confirmed offline that this PR addresses the issues he reported in #1376 , so I've removed the "draft" status of this PR and done some last bits of tidy-up. @maddyscientist and/or @mathiaswagner can you give this PR a review/test run and let me know what you think? Barring any showstoppers, I'd like to get this in as-is sooner as opposed to later. |
|
Follow-up work is being tracked here: #1389 |
hummingtree
reviewed
Jul 3, 2023
lib/inv_cg_quda.cpp
Outdated
|
|
||
| // whether to select alternative reliable updates | ||
| // if we're computing the heavy quark residual, force "traditional" reliable updates |
There was a problem hiding this comment.
This comment should be in the heavy quark path?
There was a problem hiding this comment.
Ah, no, I just forgot to clean that up. I'll take care of that...
hummingtree
approved these changes
Jul 3, 2023
|
This passed a |
This was referenced Aug 30, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR provides an algorithmic solution to convergence issues with HQ residual solves noted in #1376 . In brief, instead of having one unified code path for "regular" L2-norm solves and HQ residual solves, this PR creates separate code path for solves requesting a HQ residual.
This new code path contained in a function
hqsolveincludes improved logic for tracking reliable updates based on the L2 residual as well as the HQ residual. In the interest of simplicity, this separate code path does not feature all of the optimizations in the traditional CG code. Since HQ solves typically take O(100) iterations, as opposed to thousands to tens of thousands of iterations for light quark solves, this seemed like an acceptable decision. Of note, this new code path still supports mixed precision.The two key improvements in
hqsolveare as follows:hqsolvelogic now only tracks HQ reliable update conditions if there is no L2 convergence requirement (i.e.,tol == 0).In the case of my own reproducers, the first improvement addresses the issues @detar noted in #1376 . The second improvement leads to a great reduction in the number of iterations required for a HQ solve (much more in line with what is seen in MILC's CPU implementation) with the benefit of retaining mixed precision. This is due to removing the requirement of L2 breakdown before switching to HQ reliable updates.
Outstanding tasks:
clang-format