Merge branch 'martin_fileinfo_cache' into martin_fileinfo_cache_2

.github/workflows/linkcheck.yml

@@ -4,7 +4,8 @@ jobs:
   check-links:
     if: ${{ github.repository != 'stfc/PSyclone-mirror' }}
     name: Run linkspector
-    runs-on: ubuntu-latest
+    # TODO 2838: linkspector doesn't support ubuntu-latest at the moment.
+    runs-on: ubuntu-22.04
     steps:
       - uses: actions/checkout@v4
       - name: Run linkspector

.github/workflows/python-package.yml

@@ -1,7 +1,7 @@
 # -----------------------------------------------------------------------------
 # BSD 3-Clause License
 #
-# Copyright (c) 2020-2024, Science and Technology Facilities Council.
+# Copyright (c) 2020-2025, Science and Technology Facilities Council.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -36,6 +36,7 @@
 # Modified by A. J. Voysey, Met Office
 # Modified by J. Henrichs, Bureau of Meteorology
 # Modified by N. Nobre, STFC Daresbury Lab
+# Modified by A. B. G. Chalk, STFC Daresbury Lab
 
 # This workflow will install Python dependencies, run tests and lint with a
 # variety of Python versions.
@@ -103,7 +104,8 @@ jobs:
     - run: cd doc/developer_guide; make html; make doctest
   build:
     if: ${{ github.repository != 'stfc/PSyclone-mirror' }}
-    runs-on: ubuntu-latest
+    # TODO 2837: Switch to ubuntu-latest once Python 3.7 support is removed.
+    runs-on: ubuntu-22.04
     strategy:
       matrix:
         python-version: [3.7, 3.8, 3.13]

README.md

@@ -2,7 +2,7 @@
   <img width="460" src="doc/logo/psyclone_v1.0.png">
 </p>
 
-[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.11190458.svg)](https://doi.org/10.5281/zenodo.11190458)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.11190457.svg)](https://doi.org/10.5281/zenodo.11190457)
 ![Build Status](https://github.com/stfc/PSyclone/workflows/PSyclone%20tests%20and%20examples/badge.svg)
 [![codecov](https://codecov.io/gh/stfc/PSyclone/branch/master/graph/badge.svg)](https://codecov.io/gh/stfc/PSyclone)
 

changelog

@@ -1,3 +1,25 @@
+	1) PR #2827. Update Zenodo with release 3.0.0 and update link in
+	README.md.
+
+	2) PR #2759 for #2758. Fixes failure seen in compilation tests if
+	a previous build of infrastructure existed.
+
+	3) PR #2807 for #2806. Adds a new 'RESOLVE_IMPORTS' global variable
+	to transformation scripts which allows the user to instruct the
+	frontend to chase down some (or all) module imports when constructing
+	the PSyIR for existing code.
+
+	4) PR #2821 for #1247. Adds support for parsing comments from the fparser2
+	AST tree. There are two new FortranReader arguments to select the behaviour
+	for comments and directives (by default they are not parsed).
+
+	5) PR #2836 for #2835. Reverts GHA to use Ubuntu 22.04 as python-3.7
+	not available in ubuntu-latest and linkspector is also broken.
+
+	6) PR #2820 for #2773. Adds support for the Vernier timing library.
+
+release 3.0.0 6th of December 2024
+
 	1) PR #2477 for #2463. Add support for Fortran Namelist statements.
 
 	2) PR #2367 for #2296. Extends ModuleInfo so that it can be queried
@@ -225,8 +247,8 @@
 
 	77) PR #2708 for #2663. Add LFRic OpenACC in the integration tests.
 
-	78) PR #2678 for #2636. Alter LFRic PSy-layer to lookup nlevels for
-	each kernel
+	78) PR #2678 for #2636. Alter LFRic PSy-layer to lookup nlayers for
+	each kernel.
 
 	79) PR #2687 for #2684. Add support for WHERE constructs that contain
 	references without range-notation (and prevent some incorrect cases).
@@ -329,6 +351,12 @@
 	123) PR #2809 for #2737. Updates documentation on OpenMP to point to
 	examples of tranformations for CPU and GPU.
 
+	124) PR #2815 for #2636. Get nlayers from first field or operator
+	argument to each kernel in LFRic.
+
+	125) PR #2817 for #2816. Fix index type in initialisation of arrays
+	of fields in PSyAD test harness construction.
+
 release 2.5.0 14th of February 2024
 
 	1) PR #2199 for #2189. Fix bugs with missing maps in enter data

doc/developer_guide/module_manager.rst

@@ -41,6 +41,8 @@
 
 .. _module_manager:
 
+
+
 Module Manager
 ##############
 
@@ -135,3 +137,65 @@ which prints the filenames of all modules used in ``tl_testkern_mod``:
     Module: constants_mod constants_mod.f90
     Module: fs_continuity_mod fs_continuity_mod.f90
     Module: kernel_mod kernel_mod.f90
+
+
+
+FileInfo
+========
+
+FileInfo is a class that is used to store information about Fortran files.
+
+This information can include:
+
+- The source code itself
+- The fparser tree information
+- The PSyIR tree information
+
+All this information is gathered in this single class since this also
+allows for caching of it, see next section
+
+
+
+Caching
+=======
+
+The `ModuleManager` and `FileInfo` support a caching of the
+fparser tree representation of a source code.
+(Support for PSyIR is planned)
+
+This caching has to be **explicitly enabled** in the constructor
+of `ModuleManager`.
+
+
+.. testcode ::
+
+    mod_manager = ModuleManager.get(use_caching=True)
+
+
+Most of the time in the PSyIR generation is currently spent in the
+fparser tree generation. Consequently, this leads to significant
+speed-ups in the process of reading and parsing the source code
+of modules.
+
+The default cache file is named the same way as the source file,
+but replaces the file extension with `.psycache`. E.g., a cache file
+for the source file `foo.f90` will be called `foo.psycache`.
+
+The caching algorithm to obtain the fparser tree OR PSyIR is briefly described as follows:
+
+- If fparser tree / PSyIR was read before: RETURN fparser tree or PSyIR
+- If source code is not yet read:
+
+    - Read the content of the file
+    - Create the source's checksum.
+- Read cache file if it exists:
+
+    - If the checksum of the cache is the same as the one of the source:
+
+        - load the fparser tree / PSyIR from the cache file and RETURN fparser tree or PSyIR
+- Create the fparser tree / PSyIR from the source code
+- Save cache file IF it was not loaded before:
+
+    - Update cache information
+    - Store to cache file
+- RETURN fparser tree or PSyIR

doc/developer_guide/psy_data.rst

@@ -72,8 +72,8 @@ the same time as checking that read-only values are indeed not modified,
 different module names and data types must be used.
 
 PSyData divides its application into different classes. For example,
-the class "profile" is used for all profiling tools (e.g. TAU, DrHook or the
-NVIDIA profiling tools). This class name is used as a prefix for
+the class "profile" is used for all profiling tools (e.g. TAU, Vernier,
+DrHook, or the NVIDIA profiling tools). This class name is used as a prefix for
 the module name, the ``PSyDataType`` and functions. So if a profiling application
 is linked the above code will actually look like this::
 
@@ -101,7 +101,7 @@ The class prefixes supported at the moment are:
 Class Prefix            Description
 ======================= =======================================================
 profile                 All libraries related to profiling tools like TAU,
-                        DrHook, NVIDIA's profiling tools etc. See
+                        DrHook, NVIDIA's profiling tools, Vernier etc. See
                         :ref:`user_guide:profiling` for details.
 extract                 For libraries used for kernel data extraction. See
                         :ref:`user_guide:psyke` for details.
@@ -970,7 +970,7 @@ by storing pointers to the profiling data to be able to
 print all results in a ProfileFinalise() subroutine.
 Some of the wrapper libraries use the PSyData base class (e.g. dl_timer,
 simple_timing, template), others do not (e.g. NVIDIA profiling, TAU,
-DrHook wrapper).
+Vernier, DrHook wrapper).
 
 .. _psyke:
 

doc/developer_guide/psyir.rst

@@ -898,7 +898,7 @@ class, for example:
 
 .. code-block:: python
 
-    from psyclone.psyir.nodes.commentable_mixin import CommentableMixin
+    from psyclone.psyir.commentable_mixin import CommentableMixin
 
     class MyNode(Node, CommentableMixin):
         ''' Example node '''

doc/reference_guide/doxygen.config

@@ -38,7 +38,7 @@ PROJECT_NAME           = "Reference Guide"
 # could be handy for archiving the generated documentation or if some version
 # control system is used.
 
-PROJECT_NUMBER         = 2.5.0
+PROJECT_NUMBER         = 3.0.0
 
 # Using the PROJECT_BRIEF tag one can provide an optional one line description
 # for a project that appears at the top of each page and should give viewer a

doc/user_guide/conf.py

@@ -246,9 +246,10 @@
 # (source start file, target name, title, author,
 #  documentclass [howto/manual]).
 latex_documents = [
-    ('index', 'psyclone.tex', 'PSyclone Documentation',
-     'Andrew Coughtrie, Rupert Ford, Joerg Henrichs, Iva Kavcic,\\\\ '
-     'Andrew Porter, Sergi Siso and Joseph Wallwork', 'manual'),
+    ('index', 'psyclone.tex', 'PSyclone User Guide',
+     'Oakley Brunt, Andrew Coughtrie, Rupert Ford, \\\\ '
+     'Joerg Henrichs, Iva Kavcic, Andrew Porter, Sergi Siso \\\ '
+     'and Joseph Wallwork', 'manual'),
 ]
 
 # Set maximum depth for the nested lists to prevent LaTeX

doc/user_guide/dynamo0p3.rst

@@ -1938,7 +1938,7 @@ conventions, are:
 2) Include ``nlayers``, the number of layers in a column. ``nlayers``
    is an ``integer`` of kind ``i_def`` and has intent ``in``. PSyclone
    will obtain the value of ``nlayers`` to use for a particular kernel
-   from the first field (in the argument list) that is written to.
+   from the first field or operator in the argument list.
 3) For each scalar/field/vector_field/operator in the order specified by
    the meta_args metadata:
 

doc/user_guide/profiling.rst

@@ -51,9 +51,9 @@ transformation within a transformation script.
 
 
 PSyclone can be used with a variety of existing profiling tools.
-It currently supports dl_timer, TAU, Dr Hook, the NVIDIA GPU profiling
-tools and it comes with a simple stand-alone timer library. The
-:ref:`PSyData API <psy_data>` (see also the
+It currently supports dl_timer, TAU, Vernier, Dr Hook, the NVIDIA GPU
+profiling tools and it comes with a simple stand-alone timer library.
+The :ref:`PSyData API <psy_data>` (see also the
 :ref:`Developer Guide <dev_guide:psy_data>`)
 is utilised to implement wrapper libraries that connect the PSyclone
 application to the profiling libraries. Certain adjustments to
@@ -77,8 +77,8 @@ Interface to Third Party Profiling Tools
 ----------------------------------------
 
 PSyclone comes with :ref:`wrapper libraries <libraries>` to support
-usage of TAU, Dr Hook, dl_timer, NVTX (NVIDIA Tools Extension library),
-and a simple non-thread-safe timing library. Support for further
+usage of TAU, Vernier, Dr Hook, dl_timer, NVTX (NVIDIA Tools Extension
+library), and a simple non-thread-safe timing library. Support for further
 profiling libraries will be added in the future. To compile the
 wrapper libraries, change into the directory ``lib/profiling``
 of PSyclone and type ``make`` to compile all wrappers. If only
@@ -107,23 +107,29 @@ libraries that come with PSyclone:
 ``lib/profiling/dl_timer``
     This wrapper uses the apeg-dl_timer library. In order to use
     this wrapper, you must download and install the dl_timer library
-    from ``https://bitbucket.org/apeg/dl_timer``. This library has
+    from https://bitbucket.org/apeg/dl_timer. This library has
     various compile-time options and may be built with MPI or OpenMP
     support. Additional link options might therefore be required
     (e.g. enabling OpenMP, or linking with MPI).
 
 ``lib/profiling/tau``
     This wrapper uses TAU profiling and tracing toolkit. It can be
-    downloaded from ``https://www.cs.uoregon.edu/research/tau``.
+    downloaded from https://www.cs.uoregon.edu/research/tau.
 
 ``lib/profiling/drhook``
     This wrapper uses the Dr Hook library. You need to contact
     ECMWF to obtain a copy of Dr Hook.
 
+``lib/profiling/vernier``
+    This wrapper uses the UK Met Office's Vernier library. It can be
+    downloaded from https://github.com/MetOffice/Vernier. This
+    library writes its output to files ``vernier-out-<RANK>``, and
+    will overwrite existing output files.
+
 ``lib/profiling/nvidia``
     This is a wrapper library that maps the PSyclone profiling API
     to the NVIDIA Tools Extension library (NVTX). This library is
-    available from ``https://developer.nvidia.com/cuda-toolkit``.
+    available from https://developer.nvidia.com/cuda-toolkit.
 
 ``lib/profiling/lfric_timer``
     This profile wrapper uses the timer functionality provided by