Merge pull request #445 from NCAR/docs-new_model_porting

Documentation update: new model porting
fixes #439 #431

CHANGELOG.rst

@@ -22,6 +22,11 @@ individual files.
 
 The changes are now listed with the most recent at the top.
 
+**January 27 2023 :: Documentation update for porting new models. Tag v10.6.2**
+
+- Improved 'porting new models to DART' documentation.
+- Removed outdated references to previous build system.
+
 **December 21 2022 :: Documentation update for CLM and the DART Tutorial. Tag v10.6.1**
 
 - Improved instructions for the CLM-DART tutorial.  

README.rst

@@ -246,6 +246,7 @@ References
    :caption: Run DART with your model
 
    guide/advice-for-new-collaborators
+   guide/instructions-for-porting-a-new-model-to-dart
    DART build system <guide/quickbuild.rst>
    guide/assimilation-complex-model
    guide/mpi_intro

assimilation_code/location/README

@@ -13,7 +13,7 @@ needed by the DART algorithms.
 
 Each of the different location_mod.f90 files provides the same
 set of interfaces and defines a 'module location_mod', so by
-selecting the proper version in your path_names_xxx file you
+selecting the proper version in quickbuild.sh you
 can compile your model code with the main DART routines.
 
 threed_sphere:

assimilation_code/location/channel/location_mod.rst

@@ -28,8 +28,7 @@ Location-independent code
 
 All types of location modules define the same module name ``location_mod``. Therefore, the DART framework and any user
 code should include a Fortran 90 ``use`` statement of ``location_mod``. The selection of which location module will be
-compiled into the program is controlled by which source file name is specified in the ``path_names_xxx`` file, which is
-used by the ``mkmf_xxx`` scripts.
+compiled into the program is controlled by the LOCATION variable in ``quickbuild.sh``.
 
 All types of location modules define the same Fortran 90 derived type ``location_type``. Programs that need to pass
 location information to subroutines but do not need to interpret the contents can declare, receive, and pass this

assimilation_code/location/location_mod.rst

@@ -16,7 +16,7 @@ provides code for creating, setting/getting, copying location information (coord
 specific coordinate information. It also contains distance routines needed by the DART algorithms.
 
 Each of the different location_mod.f90 files provides the same set of interfaces and defines a 'module location_mod', so
-by selecting the proper version in your path_names_xxx file you can compile your model code with the main DART routines.
+by selecting the proper version in quickbuild.sh you can compile your model code with the main DART routines.
 
 -  :doc:`./threed_sphere/location_mod`:
    The most frequently used version for real-world 3d models. It uses latitude and longitude for horizontal coordinates,

assimilation_code/location/oned/location_mod.rst

@@ -15,9 +15,9 @@ between 0 and 1. A type that abstracts the location is provided along with opera
 compute distances between locations. This is a member of a class of similar location modules that provide the same
 abstraction for different represenations of physical space.
 
-All possible location modules define the same module name ``location_mod``. Therefore, the DART framework and any user
-code should include a Fortran 90 'use' statement of 'location_mod'. The selection of exactly which location module is
-compiled is specified by the source file name in the ``path_names_xxx`` file, which is read by the ``mkmf_xxx`` scripts.
+All types of location modules define the same module name ``location_mod``. Therefore, the DART framework and any user
+code should include a Fortran 90 ``use`` statement of ``location_mod``. The selection of which location module will be
+compiled into the program is controlled by the LOCATION variable in ``quickbuild.sh``.
 
 The model-specific ``model_mod.f90`` files need to define four ``get_close`` routines, but in most cases they can simply
 put a ``use`` statement at the top which uses the routines in the locations module, and they do not have to provide any

assimilation_code/location/threed_cartesian/location_mod.rst

@@ -26,8 +26,7 @@ Location-independent code
 
 All types of location modules define the same module name ``location_mod``. Therefore, the DART framework and any user
 code should include a Fortran 90 ``use`` statement of ``location_mod``. The selection of which location module will be
-compiled into the program is controlled by which source file name is specified in the ``path_names_xxx`` file, which is
-used by the ``mkmf_xxx`` scripts.
+compiled into the program is controlled by the LOCATION variable in ``quickbuild.sh``.
 
 All types of location modules define the same Fortran 90 derived type ``location_type``. Programs that need to pass
 location information to subroutines but do not need to interpret the contents can declare, receive, and pass this

assimilation_code/location/threed_sphere/location_mod.rst

@@ -340,8 +340,7 @@ Location-independent code
 
 All types of location modules define the same module name ``location_mod``. Therefore, the DART framework and any user
 code should include a Fortran 90 ``use`` statement of ``location_mod``. The selection of which location module will be
-compiled into the program is controlled by which source file name is specified in the ``path_names_xxx`` file, which is
-used by the ``mkmf_xxx`` scripts.
+compiled into the program is controlled by the LOCATION variable in ``quickbuild.sh``.
 
 All types of location modules define the same Fortran 90 derived type ``location_type``. Programs that need to pass
 location information to subroutines but do not need to interpret the contents can declare, receive, and pass this

assimilation_code/programs/model_mod_check/model_mod_check.rst

@@ -279,54 +279,6 @@ below.
       interp_test_vertcoord = 'VERTISHEIGHT'
      /
 
-Other modules used
-------------------
-
-::
-
-   assimilation_code/location/threed_sphere/location_mod.f90
-   assimilation_code/location/utilities/default_location_mod.f90
-   assimilation_code/location/utilities/location_io_mod.f90
-   assimilation_code/modules/assimilation/adaptive_inflate_mod.f90
-   assimilation_code/modules/assimilation/assim_model_mod.f90
-   assimilation_code/modules/assimilation/assim_tools_mod.f90
-   assimilation_code/modules/assimilation/cov_cutoff_mod.f90
-   assimilation_code/modules/assimilation/filter_mod.f90
-   assimilation_code/modules/assimilation/obs_model_mod.f90
-   assimilation_code/modules/assimilation/quality_control_mod.f90
-   assimilation_code/modules/assimilation/reg_factor_mod.f90
-   assimilation_code/modules/assimilation/sampling_error_correction_mod.f90
-   assimilation_code/modules/assimilation/smoother_mod.f90
-   assimilation_code/modules/io/dart_time_io_mod.f90
-   assimilation_code/modules/io/direct_netcdf_mod.f90
-   assimilation_code/modules/io/io_filenames_mod.f90
-   assimilation_code/modules/io/state_structure_mod.f90
-   assimilation_code/modules/io/state_vector_io_mod.f90
-   assimilation_code/modules/observations/forward_operator_mod.f90
-   assimilation_code/modules/observations/obs_kind_mod.f90
-   assimilation_code/modules/observations/obs_sequence_mod.f90
-   assimilation_code/modules/utilities/distributed_state_mod.f90
-   assimilation_code/modules/utilities/ensemble_manager_mod.f90
-   assimilation_code/modules/utilities/netcdf_utilities_mod.f90
-   assimilation_code/modules/utilities/null_mpi_utilities_mod.f90
-   assimilation_code/modules/utilities/null_win_mod.f90
-   assimilation_code/modules/utilities/obs_impact_mod.f90
-   assimilation_code/modules/utilities/options_mod.f90
-   assimilation_code/modules/utilities/parse_args_mod.f90
-   assimilation_code/modules/utilities/random_seq_mod.f90
-   assimilation_code/modules/utilities/sort_mod.f90
-   assimilation_code/modules/utilities/time_manager_mod.f90
-   assimilation_code/modules/utilities/types_mod.f90
-   assimilation_code/modules/utilities/utilities_mod.f90
-   assimilation_code/programs/model_mod_check/model_mod_check.f90
-   models/your_model_here/model_mod.f90
-   models/model_mod_tools/test_interpolate_threed_sphere.f90
-   models/model_mod_tools/model_check_utilities_mod.f90
-   models/utilities/default_model_mod.f90
-   observations/forward_operators/obs_def_mod.f90
-   observations/forward_operators/obs_def_utilities_mod.f90
-
-Items highlighted may change based on which model is being tested.
 
 Files
 -----
@@ -341,63 +293,6 @@ Files
 Usage
 -----
 
-Normal circumstances indicate that you are trying to put a new model into DART, so to be able to build and run
-``model_mod_check``, you will need to create a ``path_names_model_mod_check`` file with the following contents:
-
-::
-
-   assimilation_code/location/threed_sphere/location_mod.f90
-   assimilation_code/location/utilities/default_location_mod.f90
-   assimilation_code/location/utilities/location_io_mod.f90
-   assimilation_code/modules/assimilation/adaptive_inflate_mod.f90
-   assimilation_code/modules/assimilation/assim_model_mod.f90
-   assimilation_code/modules/assimilation/assim_tools_mod.f90
-   assimilation_code/modules/assimilation/cov_cutoff_mod.f90
-   assimilation_code/modules/assimilation/filter_mod.f90
-   assimilation_code/modules/assimilation/obs_model_mod.f90
-   assimilation_code/modules/assimilation/quality_control_mod.f90
-   assimilation_code/modules/assimilation/reg_factor_mod.f90
-   assimilation_code/modules/assimilation/sampling_error_correction_mod.f90
-   assimilation_code/modules/assimilation/smoother_mod.f90
-   assimilation_code/modules/io/dart_time_io_mod.f90
-   assimilation_code/modules/io/direct_netcdf_mod.f90
-   assimilation_code/modules/io/io_filenames_mod.f90
-   assimilation_code/modules/io/state_structure_mod.f90
-   assimilation_code/modules/io/state_vector_io_mod.f90
-   assimilation_code/modules/observations/forward_operator_mod.f90
-   assimilation_code/modules/observations/obs_kind_mod.f90
-   assimilation_code/modules/observations/obs_sequence_mod.f90
-   assimilation_code/modules/utilities/distributed_state_mod.f90
-   assimilation_code/modules/utilities/ensemble_manager_mod.f90
-   assimilation_code/modules/utilities/netcdf_utilities_mod.f90
-   assimilation_code/modules/utilities/null_mpi_utilities_mod.f90
-   assimilation_code/modules/utilities/null_win_mod.f90
-   assimilation_code/modules/utilities/obs_impact_mod.f90
-   assimilation_code/modules/utilities/options_mod.f90
-   assimilation_code/modules/utilities/parse_args_mod.f90
-   assimilation_code/modules/utilities/random_seq_mod.f90
-   assimilation_code/modules/utilities/sort_mod.f90
-   assimilation_code/modules/utilities/time_manager_mod.f90
-   assimilation_code/modules/utilities/types_mod.f90
-   assimilation_code/modules/utilities/utilities_mod.f90
-   assimilation_code/programs/model_mod_check/model_mod_check.f90
-   models/your_model_here/model_mod.f90
-   models/model_mod_tools/test_interpolate_threed_sphere.f90
-   models/utilities/default_model_mod.f90
-   observations/forward_operators/obs_def_mod.f90
-   observations/forward_operators/obs_def_utilities_mod.f90
-
-| as well as a ``mkmf_model_mod_check`` script. You should be able to look at any other ``mkmf_xxxx`` script and figure
-  out what to change. Once they exist:
-
-.. container:: unix
-
-   ::
-
-      [~/DART/models/yourmodel/work] % csh mkmf_model_mod_check
-      [~/DART/models/yourmodel/work] % make
-      [~/DART/models/yourmodel/work] % ./model_mod_check
-
 Unlike other DART components, you are expected to modify ``model_mod_check.f90`` to suit your needs as you develop your
 ``model_mod``. The code is roughly divided into the following categories:
 

assimilation_code/programs/obs_seq_to_netcdf/obs_seq_to_netcdf.rst

@@ -431,49 +431,35 @@ make sensible plots of the observations. Some important aspects are highlighted.
 Usage
 -----
 
-Obs_seq_to_netcdf
-~~~~~~~~~~~~~~~~~
-
-| ``obs_seq_to_netcdf`` is built and run in ``/DART/observations/utilities/threed_sphere`` or
-  ``/DART/observations/utilities/oned`` or in the same way as the other DART components. That directory is intentionally
-  designed to hold components that are model-insensitive. Essentially, we avoid having to populate every ``model``
-  directory with identical ``mkmf_obs_seq_to_netcdf`` and ``path_names_obs_seq_to_netcdf`` files. After the program has
-  been run, ``/DART/observations/utilities/threed_sphere/``\ ``plot_obs_netcdf.m`` can be run to plot the observations.
-  Be aware that the ``ObsTypesMetaData`` list is all known observation types and not only the observation types in the
-  netCDF file.
-
 .. _example-1:
 
-Example
-^^^^^^^
-
-.. container:: routine
+Obs_seq_to_netcdf example
 
-   ::
+.. code:: text
 
-      &schedule_nml
-         calendar        = 'Gregorian',
-         first_bin_start =  2006, 8, 1, 3, 0, 0 ,
-         first_bin_end   =  2006, 8, 1, 9, 0, 0 ,
-         last_bin_end    =  2006, 8, 3, 3, 0, 0 ,
-         bin_interval_days    = 0,
-         bin_interval_seconds = 21600,
-         max_num_bins         = 1000,
-         print_table          = .true.
-         /
+   &schedule_nml
+      calendar        = 'Gregorian',
+      first_bin_start =  2006, 8, 1, 3, 0, 0 ,
+      first_bin_end   =  2006, 8, 1, 9, 0, 0 ,
+      last_bin_end    =  2006, 8, 3, 3, 0, 0 ,
+      bin_interval_days    = 0,
+      bin_interval_seconds = 21600,
+      max_num_bins         = 1000,
+      print_table          = .true.
+      /
 
-      &obs_seq_to_netcdf_nml
-         obs_sequence_name = '',
-         obs_sequence_list = 'olist',
-         append_to_netcdf  = .false.,
-         lonlim1    =    0.0,
-         lonlim2    =  360.0,
-         latlim1    =  -80.0,
-         latlim2    =   80.0,
-         verbose    = .false.
-         /
+   &obs_seq_to_netcdf_nml
+      obs_sequence_name = '',
+      obs_sequence_list = 'olist',
+      append_to_netcdf  = .false.,
+      lonlim1    =    0.0,
+      lonlim2    =  360.0,
+      latlim1    =  -80.0,
+      latlim2    =   80.0,
+      verbose    = .false.
+      /
 
-   > *cat olist*
+   > cat olist
    /users/thoar/temp/obs_0001/obs_seq.final
    /users/thoar/temp/obs_0002/obs_seq.final
    /users/thoar/temp/obs_0003/obs_seq.final

assimilation_code/programs/obs_sequence_tool/obs_sequence_tool.rst

@@ -1,3 +1,5 @@
+.. _obs sequence tool:
+
 program ``obs_sequence_tool``
 =============================
 

conf.py

@@ -21,7 +21,7 @@
 author = 'Data Assimilation Research Section'
 
 # The full version, including alpha/beta/rc tags
-release = '10.6.1'
+release = '10.6.2'
 master_doc = 'README'
 
 # -- General configuration ---------------------------------------------------

guide/advice-for-new-collaborators.rst

@@ -1,5 +1,7 @@
-Working with collaborators on porting new models
-================================================
+.. _Using new models:
+
+Can I run my model with DART?
+=============================
 
 The DART team often collaborates with other groups to help write the interface
 code to a new model. The most efficient way to get started is to meet with
@@ -14,25 +16,28 @@ Goals of using DART
 -------------------
 
 DART is the Data Assimilation Research Testbed.  It is a collection of 
-tools and routines and scripts that allow users to built custom solutions
+tools, routines, and scripts that allow users to build custom solutions
 and explore a variety of DA related efforts.  It is not a turnkey system;
 it must be built before use and is often customized based on needs and goals.
 
 DART is often used for the following types of projects:
 
 - Learning about Data Assimilation (DA)
 - Using DART with an existing model and supported observations
-- `Adding a DART interface to a new model`_
+- Using DART with a new model: :ref:`Porting new models`
 - Using new observations with DART in an existing model
 - Using both a new model and new observations with DART
 - Using DART to teach DA
 
+You can view a list of models that are already supported at :ref:`Supported models`
+and a list of supported observations at :ref:`programs`.
+
 Everything on this "possible goals" list except adding support for a new model
 can generally be done by a single user with minimal help from the DART team.
 Therefore this discussion focuses only on adding a new model to DART.
 
-Should I consider using DART?
------------------------------
+Should I consider using DART with my model?
+-------------------------------------------
 
 DART is an ensemble-based DA system. It makes multiple runs of a model with
 slightly different inputs and uses the statistical distribution of the results
@@ -312,43 +317,4 @@ of the state variables then doing it on demand is more efficient.
 The options here are namelist selectable at runtime and the impact
 on total runtime can be easily measured and compared.
 
-Adding a DART interface to a new model
---------------------------------------
-
-DART provides a script ``new_model.sh`` which will create the necessary files
-for a new model interface.
-Enter ``./new_model.sh``, then the desired model name and location module separated
-by spaces. This will create the necessary files to get started.
-
-For example to create a model interface for a model called BOUMME which uses
-the 3D sphere location module:
-
-.. code-block::  text
-
-   cd models
-   ./new_model.sh BOUMME threed_sphere
-
-This will create an BOUMME model directory with the following files:
-
-.. code-block:: text
-
-     BOUMME/
-     ├── model_mod.f90
-     ├── readme.rst
-     └── work
-         ├── input.nml
-         └── quickbuild.sh
-
-- ``model_mod.f90`` is where to add the :doc:`required model_mod routines<required-model-mod-routines>`.
-- ``readme.rst``  is a stub to add documenation for your model interface.  
-- ``quickbuild.sh`` is used to compile DART for your model.
-
-
-Templates are chosen based on location module input. The currently supported
-location templates are for 3D and 1D modules, with the possibility for more
-in the future. At the moment, ``threed_sphere``, ``threed_cartesian``, and
-``oned``  will produce model_mod.f90 code that compile will sucessfully with ``./quickbuild.sh``.
 
-We recommend looking at the existing supported models and reusing code from them if
-possible. Models with similar grid types or vertical coordinates are good
-candidates.

guide/distributed_state.rst

@@ -87,7 +87,7 @@ of mpi_window mods:
 -  cray_win_mod.f90
 -  no_cray_win_mod.f90
 
-| We have these two modules that you can swap in your path_names files because the MPI 2 standard states:
+| We have these two modules that you can swap in because the MPI 2 standard states:
 | Implementors may restrict the use of RMA communication that is synchronized by lock calls to windows in memory
   allocated by MPI_ALLOC_MEM.
 | MPI_ALLOC_MEM uses cray pointers, thus we have supplied a window module that uses cray pointers. However,