Error in user config documentation?

[benjamin-cash]

In the documentation here: https://metplus.readthedocs.io/en/main_v4.0/Users_Guide/systemconfiguration.html#user-configuration-file it indicated that a user configuration file can consist of as little as OUTPUT_BASE. However, I am finding that there are a number of entries that must be included, even if they are present in default.conf. For example, removing LOG_DIR from my user_system file results in:

(base) login4.stampede2(1057)$ ./run_metplus.py -c /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/use_cases/met_tool_wrapper/Example/Example.conf -c /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config/stampede2.conf
Running METplus 4.0.0
Starting METplus v4.0.0
08/10 11:02:19.743 metplus (config_metplus.py:117) INFO: Starting METplus configuration setup.
08/10 11:02:19.745 metplus (config_metplus.py:247) INFO: /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/use_cases/met_tool_wrapper/Example/Example.conf: Parsed this file
08/10 11:02:19.746 metplus (config_metplus.py:247) INFO: /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config/stampede2.conf: Parsed this file
ERROR: run_metplus failed: No option 'LOG_DIR' in section: 'config'

And there are other entries that will also cause an error if they are not duplicated in my stampede2.conf. Is this an error in the documentation or have I misunderstood it?

[georgemccabe]

Hi @benjamin-cash, based on the screen output it appears that you may have removed the defaults.conf file from your METplus installation directory. This file should be present in the /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config directory. A log message will mention that this file was read. Did you rename defaults.conf to stampede2.conf? You will need to obtain the defaults.conf file again without any values removed.

You can change the values for MET_INSTALL_DIR (where MET is installed) and INPUT_BASE (location of sample input data) to the appropriate locations in defaults.conf directly if you'd like. Your stampede2.conf file should exist outside of the METplus installation directory and will need to include at least OUTPUT_BASE.

[benjamin-cash]

It appears to be there, although I did make edits to it when I was first getting started. Maybe I need a clean copy?

(base) login4.stampede2(1059)$ ls -l /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config
total 12
-rw-r--r-- 1 bcash G-821476 3780 Aug 10 10:56 defaults.conf
-rw-r--r-- 1 bcash G-821476 441 May 19 08:35 README.rst
-rw-r--r-- 1 bcash G-821476 3816 Aug 10 11:02 stampede2.conf

(base) login4.stampede2(1060)$ grep -i LOG_DIR /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config/defaults.conf
LOG_METPLUS = {LOG_DIR}/metplus.log.{LOG_TIMESTAMP_TEMPLATE}
LOG_DIR = {OUTPUT_BASE}/logs

[georgemccabe]

Hmm, that is odd. I would expect there to be a log line before reading Example.conf that looks something like this:

08/10 11:02:19.746 metplus (config_metplus.py:247) INFO: /work/02441/bcash/stampede2/metplus/METplus-4.0.0/parm/metplus_config/defaults.conf: Parsed this file

I would also expect an error to occur if the file was found but couldn't be parsed properly. Could you share your defaults.conf file so I can see what has been changed?

[benjamin-cash]

Here you go:

[config]

###############################################################################

METplus Configuration

See the METplus User's Guide System Configuration section

for more information about each of these variables.

###############################################################################

###############################################################################

MET Installation Location

* Set MET_INSTALL_DIR to the path where MET is installed

* Change MET_BIN_DIR to {MET_INSTALL_DIR}/exec if necessary

###############################################################################

MET_INSTALL_DIR = /path/to

MET_BIN_DIR = {MET_INSTALL_DIR}/bin

###############################################################################

Input/Output Information

* Set INPUT_BASE to the directory that contains sample input data

* Set OUTPUT_BASE to the directory to write output files

* Change other variables to write certain files outside of OUTPUT_BASE

* Rename metplus_final.conf config file if desired

###############################################################################

INPUT_BASE = /path/to

OUTPUT_BASE = /path/to

METPLUS_CONF = {OUTPUT_BASE}/metplus_final.conf

TMP_DIR = {OUTPUT_BASE}/tmp
STAGING_DIR = {OUTPUT_BASE}/stage

###############################################################################

Non-MET Executables

* Set CONVERT to the full path of the ImageMagick utility if it is not

in the user's path

* Set GEMPAKTOCF_JAR to the full path of the jar file if processing

GEMPAK data

###############################################################################

CONVERT = convert

GEMPAKTOCF_JAR = GempakToCF.jar

###############################################################################

Runtime Configuration

###############################################################################

LOOP_ORDER = processes

PROCESS_LIST = Usage

###############################################################################

Log File Information (Where to write logs files)

###############################################################################

LOG_METPLUS = {LOG_DIR}/metplus.log.{LOG_TIMESTAMP_TEMPLATE}

LOG_DIR = {OUTPUT_BASE}/logs

LOG_TIMESTAMP_TEMPLATE = %Y%m%d%H%M%S

LOG_TIMESTAMP_USE_DATATIME = no

LOG_MET_OUTPUT_TO_METPLUS = yes

###############################################################################

Log Level Information (How much information to log)

###############################################################################

LOG_LEVEL = INFO

LOG_MET_VERBOSITY = 2

###############################################################################

Log Formatting Information (How to format lines in log files)

###############################################################################

LOG_LINE_FORMAT = %(asctime)s.%(msecs)03d %(name)s (%(filename)s:%(lineno)d) %(levelname)s: %(message)s
LOG_ERR_LINE_FORMAT = {LOG_LINE_FORMAT}
LOG_DEBUG_LINE_FORMAT = {LOG_LINE_FORMAT}
LOG_INFO_LINE_FORMAT = %(asctime)s.%(msecs)03d %(name)s %(levelname)s: %(message)s

LOG_LINE_DATE_FORMAT = %m/%d %H:%M:%S

[georgemccabe]

The defaults.conf file looks fine to me. I am a bit baffled as to why the file is not being read first. Do you happen to have the environment variable METPLUS_PARM_BASE set in your environment? If that variable is set, then it will look relative to that directory for the metplus_config directory.

[benjamin-cash]

That was it - there was a rogue setting in my bash_profile that was causing it to look in the wrong place for defaults.conf. Thanks again!