Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Marked region removal and general code removal transformation #276

Merged
merged 11 commits into from
Apr 12, 2024

Conversation

mlange05
Copy link
Collaborator

@mlange05 mlange05 commented Apr 8, 2024

This PR combines two closely related features:

  • The addition of a new feature that allows pragma-marked code regions to be removed
  • A clean-up and reshuffling to provide marked region removal, call removal and dead code removal all under a one umbrella Transformation for inclusion in more complex pipelines.

The new pragma-based region removal will pretty much do as it says: Upon encountering a region marked with !$loki remove / !$loki end remove it will simply remove the entire code region.

The new loki.transfom.transform_remove_code basically includes the previous dead code elimination (renamed), and combines it with the new region removal and a re-write of the RemoveCallsTransformation. The latter is now a as a single Transformer with a wrapper routine that allows it to be applied to Subroutine objects. All three utilities are then provided as a single RemoveCodeTransformation for use with the Scheduler and batch-processing engine.

Since the wrapper routines for these utilities would otherwise create name clashes with the respective flags of the overarching Transformation, we adopt the do_remove_xxx naming convention, as discussed offline.

In some more detail:

  • Renamed loki.transform.transform_dead_code => loki.transform.transform_remove_code
  • Add RemoveMarkedRegionsTransformer and do_remove_marked_regions wrapper
  • Add RemvoeCallsTransformer and do_remove_calls wrapper
  • Rename eleminate_dead_code to RemoveDeadCodeTransformer and do_remove_dead_code
  • Use new RemoveCodeTransformation in loki-transform and remove the old RemoveCallsTransformation from transformations
  • Rename transformations.utility_rouines => transformations.drhook, as it's the last custom utility left in there.

@mlange05 mlange05 requested a review from reuterbal April 8, 2024 10:48
Copy link

github-actions bot commented Apr 8, 2024

Documentation for this branch can be viewed at https://sites.ecmwf.int/docs/loki/276/index.html

@mlange05 mlange05 force-pushed the naml-code-removal-skip branch 2 times, most recently from d1aa9be to cc8b63a Compare April 8, 2024 11:03
@mlange05
Copy link
Collaborator Author

mlange05 commented Apr 8, 2024

Clean with EC-physics regression.

@mlange05 mlange05 marked this pull request as ready for review April 9, 2024 03:35
@reuterbal reuterbal added the rebase required Rebase required for the PR branch to resolve conflicts label Apr 10, 2024
@mlange05 mlange05 force-pushed the naml-code-removal-skip branch 2 times, most recently from ab4f5f9 to fd8b8d5 Compare April 10, 2024 11:18
@mlange05 mlange05 removed the rebase required Rebase required for the PR branch to resolve conflicts label Apr 10, 2024
Copy link

codecov bot commented Apr 10, 2024

Codecov Report

Attention: Patch coverage is 97.81022% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 92.89%. Comparing base (a594b44) to head (da9cf28).

Files Patch % Lines
loki/transform/transform_remove_code.py 96.80% 3 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #276   +/-   ##
=======================================
  Coverage   92.88%   92.89%           
=======================================
  Files         102      102           
  Lines       18253    18291   +38     
=======================================
+ Hits        16954    16991   +37     
- Misses       1299     1300    +1     
Flag Coverage Δ
lint_rules 96.39% <ø> (ø)
loki 92.87% <96.96%> (+0.02%) ⬆️
transformations 92.10% <100.00%> (-0.13%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@reuterbal reuterbal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks very good and is a valuable utility. Also, appreciate the reshuffling code in more sensible places.

I've left a few nitpick comments about docstrings, which would be useful to clean up, but implementation looks good.

Comment on lines 35 to 37
* :method:`do_remove_calls`
* :method:`do_remove_marked_regions`
* :method:`do_remove_dead_code`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The markup is not producing the right appearance: https://sites.ecmwf.int/docs/loki/276/loki.transform.transform_remove_code.html#module-loki.transform.transform_remove_code

I think the keyword to create the link is :meth:, or simply :any: should work as well.
Not sure why the list isn't rendered as such, maybe requires an empty line before the first item?

"""
:any:`Transformer` class that removes provably unreachable code paths.

The pirmary modification performed is to prune individual code branches
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: pirmary -> primary

Comment on lines +153 to +157
if condition == 'True':
return body

if condition == 'False':
return else_body
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realise that this has just been moved from the old module, but this comparison against a string put me off for a second... This is using the implicit String comparison to evaluate the condition? Would we want to account for something like IF(1)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is actually intentional, as IF(1) is basically illegal in a Fortran-context (strong typing; 1 is not a logical) The visual oddness comes from "True" actually being our internal representation of .true. (the fortran intrinsic logical value), and that is the only thing we should match in a condition, since everything else should evaluate to that via simplify (eg. result of a logical operator will be a logical, never a numerical value).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks - I think I did not actually know that :)

to specific named subroutines.

This :any:`Transformer` will by default also remove the enclosing
inline-conditional when encountering calls of the form ```if
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A stray '`' here and after the stop.

@mlange05
Copy link
Collaborator Author

Hi @reuterbal just a quick heads-up: there was some cross-fire between PR #271 and this one, causing a test failure for scheduler / pipeline configs. I've rebased over latest main and adjusted accordingly. Only the top two commits should be new. Hope this works now.

@reuterbal
Copy link
Collaborator

Many thanks for catching the rebase fall-out! As discussed offline, I've done a little touch-up on the docstrings, otherwise this should be good to go!

@reuterbal reuterbal added the ready for merge This PR has been approved and is ready to be merged label Apr 12, 2024
@reuterbal reuterbal merged commit 10a98fe into main Apr 12, 2024
12 checks passed
@reuterbal reuterbal deleted the naml-code-removal-skip branch April 12, 2024 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for merge This PR has been approved and is ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants