Transformations: SCCRawStackPipeline and SCC config-from-file #314
Conversation
|
Documentation for this branch can be viewed at https://sites.ecmwf.int/docs/loki/314/index.html |
reuterbal
force-pushed
the
naml-sbrm-rawstack-transformation
branch
from
7009ea5
to
6fa205d
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #314 +/- ##
==========================================
+ Coverage 95.11% 95.25% +0.14%
==========================================
Files 165 170 +5
Lines 35065 36258 +1193
==========================================
+ Hits 33351 34539 +1188
- Misses 1714 1719 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
The horizontal size does not get remapped via the call argument map. When it has a different name in the driver, we get a wrong allocation, so we add a manual ovefrride.
We're leaving the defaults in for the moment until the full hit comes.
mlange05
force-pushed
the
naml-sbrm-rawstack-transformation
branch
from
6fa205d
to
fd09b09
Compare
There was a problem hiding this comment.
Many thanks for taking this over and apologies for the delay in reviewing. I've not gone in full detail through the transformation logic but the test seems reasonably comprehensive to bring this in. I've left a few comments where I noticed things that could be improved.
|
|
||
| assignments = FindNodes(Assignment).visit(kernel3.body) | ||
|
|
||
| assert fgen(assignments[0].lhs).lower() == 'jd_zde1' |
There was a problem hiding this comment.
Should this really all be fgen(...).lower() == ...? Couldn't we use simple string-comparison for expressions here?
There was a problem hiding this comment.
The .rhs test with explicit kind-specifiers need the fgen, but otherwise this seems to work. Will do.
| zde3 = pzz | ||
| zde3(1:nlon,1:klev) = pzz | ||
|
|
||
| !$acc iend data |
There was a problem hiding this comment.
| !$acc iend data | |
| !$acc end data |
| basedir = gettempdir()/'test_pool_allocator_temporaries' | ||
| basedir.mkdir(exist_ok=True) | ||
| (basedir/'driver.F90').write_text(fcode_driver) | ||
| (basedir/'kernel1_mod.F90').write_text(fcode_kernel1) | ||
| (basedir/'kernel2_mod.F90').write_text(fcode_kernel2) | ||
| (basedir/'kernel3_mod.F90').write_text(fcode_kernel3) |
There was a problem hiding this comment.
instead of gettempdir() it's better to use the tmp_dir fixture, which ensures clean-up at the end even in case of test failures, and saves the explicit rmtree.
| else: | ||
| definitions = () | ||
|
|
||
| scheduler = Scheduler(paths=[basedir], config=SchedulerConfig.from_dict(config), frontend=frontend, |
There was a problem hiding this comment.
We should use xmods=[tmp_path] (if you're switching to that fixture) here. That ensures OMNI writes its xmod files to that tempdir path instead of the current working directory. That avoids littering the source tree and prevents name clashes with same program unit names in other tests.
There was a problem hiding this comment.
As discussed offline: I tried, I failed, I'm giving up now. I vote we keep this for a separate "unfoo-OMNI" PR.
| def __init__( | ||
| self, block_dim, horizontal, stack_name='STACK', | ||
| local_int_var_name_pattern='JD_{name}', directive=None, | ||
| key=None, driver_horizontal=None, **kwargs |
There was a problem hiding this comment.
We should likely drop the key argument here, as this can cause issues in pipelines, no?
| The items corresponding to successor routines called from :data:`routine` | ||
| """ | ||
| successor_map = { | ||
| successor.ir.name.lower(): successor |
There was a problem hiding this comment.
Shouldn't that simply be
| successor.ir.name.lower(): successor | |
| successor.local_name: successor |
|
|
||
| Parameters | ||
| ---------- | ||
| routine : :any:'Subroutine |
There was a problem hiding this comment.
| routine : :any:'Subroutine | |
| routine : :any:`Subroutine` |
|
|
||
| Parameters | ||
| ---------- | ||
| routine : :any:'Subroutine |
There was a problem hiding this comment.
| routine : :any:'Subroutine | |
| routine : :any:`Subroutine` |
|
|
||
| Parameters | ||
| ---------- | ||
| routine : :any:'Subroutine |
There was a problem hiding this comment.
| routine : :any:'Subroutine | |
| routine : :any:`Subroutine` |
mlange05
force-pushed
the
naml-sbrm-rawstack-transformation
branch
from
e8da699
to
9ffaab9
Compare
mlange05
force-pushed
the
naml-sbrm-rawstack-transformation
branch
from
b7314dd
to
b8dfc88
Compare
mlange05
force-pushed
the
naml-sbrm-rawstack-transformation
branch
from
b8dfc88
to
26bd8cd
Compare
reuterbal
added
the
ready for merge
The main point of this PR is a re-base-and-replay of PR #201 by @rolfhm . It primarily rebases the main implementation of the
TemporariesSCCRawStackTransformationover recentmainand squashes the slightly convoluted commit history. I then added the necessary steps to integrate this with the recent addition of SCC "pipeline" classes and added a specific override that we need to integrate this with CLOUDSC regression testing. The latter is then implemented in the CLOUDSC companion PR (ecmwf-ifs/dwarf-p-cloudsc#86).The small override option needed for CLOUDSC regression testing (
driver_horizontal) comes from the horizontal variable name being different in the driver routine (NPROMA) to the kernel routines (KLON). As this new allocator does not remap the horizontal over the calls, I added a hard-override. The idea is to merge this now and then mop up the rest when we consolidate theloki.transformations.temporariessub-package.Due to the same reasoning this PR also does not change much in the implementation of the RawStackAllocator itself. The implementation is cleanly isolated and close the stack pool allocator, but there is some overlap where we could reduce redundancy by refactoring common sub-steps. However, I think this is beyond the scope of this PR.
And finally, @rolfhm could you please take a look and indicate here if you are happy with this and if this still works for you?