Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add task to process reforecast variables to save on WCOSS2 #2680

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
157 commits
Select commit Hold shift + click to select a range
ec4b800
Add task to extract reforecast variables
EricSinsky-NOAA May 20, 2024
2923780
Add top level script for extractvars task
EricSinsky-NOAA May 21, 2024
7ff9a28
Merge remote-tracking branch 'origin/develop' into feature/extractvars
EricSinsky-NOAA May 22, 2024
25cf481
Merge remote-tracking branch 'origin/develop' into feature/extractvars
EricSinsky-NOAA May 28, 2024
8845dc7
Update extractvars job dependency
EricSinsky-NOAA May 28, 2024
f46544e
Add JGLOBAL script for extractvars task
EricSinsky-NOAA May 29, 2024
e9702c8
Add scripts to process atmos reforecast variables
EricSinsky-NOAA May 29, 2024
e378727
Add parm files for extractvars task
EricSinsky-NOAA May 29, 2024
4fbc7ac
Merge remote-tracking branch 'origin/develop' into feature/extractvars
EricSinsky-NOAA May 29, 2024
ff36ef2
Refine extractvars scripts
EricSinsky-NOAA May 29, 2024
c173979
Add corrections
EricSinsky-NOAA May 29, 2024
43b0ac1
Merge remote-tracking branch 'origin/develop' into feature/extractvars
EricSinsky-NOAA May 31, 2024
0deabb0
Add script to process ocean and ice
EricSinsky-NOAA May 31, 2024
177783b
Add script to process waves
EricSinsky-NOAA Jun 3, 2024
7235fc1
Merge remote-tracking branch 'origin/develop' into feature/extractvars
EricSinsky-NOAA Jun 3, 2024
d3688ed
Simplify wave script
EricSinsky-NOAA Jun 3, 2024
f807374
Fix brackets
EricSinsky-NOAA Jun 3, 2024
215a726
Add wave forecast frequency variable
EricSinsky-NOAA Jun 3, 2024
b79237c
Run shellnorms on push
EricSinsky-NOAA Jun 4, 2024
e3f61ba
Fix wave script
EricSinsky-NOAA Jun 4, 2024
c9e49d7
Clean up JGLOBAL_EXTRACTVARS script
EricSinsky-NOAA Jun 4, 2024
b1076f3
Clean up more scripts
EricSinsky-NOAA Jun 4, 2024
e7adce9
Revert "Run shellnorms on push"
EricSinsky-NOAA Jun 4, 2024
e1f2e71
Clean up extractvars script
EricSinsky-NOAA Jun 4, 2024
a310f0e
Clean up extractvars atmos script
EricSinsky-NOAA Jun 4, 2024
1917d26
Clean up atmos extractvars script
EricSinsky-NOAA Jun 4, 2024
b88bae4
Clean up atmos extractvars script further
EricSinsky-NOAA Jun 4, 2024
f2a9e33
Refine atmos extractvars script
EricSinsky-NOAA Jun 4, 2024
5b964ef
disable specific spellcheck in atmos script
EricSinsky-NOAA Jun 4, 2024
1049801
Add more corrections
EricSinsky-NOAA Jun 5, 2024
1168566
Simplify ocnice extractvars script
EricSinsky-NOAA Jun 5, 2024
c2a23b7
Remove unused variable
EricSinsky-NOAA Jun 5, 2024
1a91e4a
Add more corrections
EricSinsky-NOAA Jun 5, 2024
9417dd6
Clean up wave script in extractvars task
EricSinsky-NOAA Jun 5, 2024
99d3adf
Add more fixes to wave script in extractvars
EricSinsky-NOAA Jun 5, 2024
f3df65e
Define COM directories for extractvars task
EricSinsky-NOAA Jun 5, 2024
e0f8d32
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jun 7, 2024
64beacc
Create an option to remove extractvars task
EricSinsky-NOAA Jun 7, 2024
2a5aab9
Add more fixes to extractvars scripts
EricSinsky-NOAA Jun 10, 2024
01b601a
Modify wave parm file for extractvars task
EricSinsky-NOAA Jun 10, 2024
3a8e90f
Move extractvars task to end in gefs_tasks.py
EricSinsky-NOAA Jun 11, 2024
1ca415e
Improve task dependency for extractvars
EricSinsky-NOAA Jun 12, 2024
7a8a40f
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jun 12, 2024
a84e0d9
Remove unused variable
EricSinsky-NOAA Jun 12, 2024
ecb069c
Add DO_EXTRACTVARS to default yaml
EricSinsky-NOAA Jun 12, 2024
e280a36
Modify config.base for GEFS
EricSinsky-NOAA Jun 12, 2024
c79a8b6
Fix dependencies for extractvars
EricSinsky-NOAA Jun 12, 2024
b5893db
Update FHOUT variable
EricSinsky-NOAA Jun 12, 2024
03ba201
Remove unused variables from config.extractvars
EricSinsky-NOAA Jun 12, 2024
2a6071a
Remove more unused variables
EricSinsky-NOAA Jun 12, 2024
cdc4d89
Remove nonexistant variable
EricSinsky-NOAA Jun 12, 2024
456dbdf
Modify scripts to process netcdf input
EricSinsky-NOAA Jun 12, 2024
e496529
Fix shellcheck errors
EricSinsky-NOAA Jun 12, 2024
c7b7c4d
Remove line that has been commented out
EricSinsky-NOAA Jun 12, 2024
8ab7ec3
Add checks for ocean/ice variable extraction
EricSinsky-NOAA Jun 14, 2024
7ddb470
Fix shellcheck errors
EricSinsky-NOAA Jun 14, 2024
c44a5bc
Fix warning statement
EricSinsky-NOAA Jun 17, 2024
b61d98f
Fix typo in parmfile
EricSinsky-NOAA Jun 17, 2024
4e4715f
Add checks to the extractvars atmos scripts
EricSinsky-NOAA Jun 18, 2024
fca3c47
Merge remote-tracking branch 'remotes/EMC/develop' into feature/extra…
EricSinsky-NOAA Jun 18, 2024
94ebfbb
Address shellcheck errors
EricSinsky-NOAA Jun 18, 2024
5774c2a
Apply initial fix for ocean filename mismatch
EricSinsky-NOAA Jun 21, 2024
9c73505
Revert "Apply initial fix for ocean filename mismatch"
EricSinsky-NOAA Jun 23, 2024
3947d18
Apply an improved fix for ocean filename mismatch
EricSinsky-NOAA Jun 24, 2024
dd6b1b1
Fix shellcheck issue
EricSinsky-NOAA Jun 24, 2024
f828121
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jun 24, 2024
ecf788f
Increase flexibility of extractvars dependencies
EricSinsky-NOAA Jun 25, 2024
1beb044
Allow only specific components to be processed
EricSinsky-NOAA Jun 25, 2024
987f27f
Refine procedure to get fv3 fhours with replay ICs
EricSinsky-NOAA Jul 2, 2024
156306f
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 2, 2024
674c40f
Fix shellcheck errors
EricSinsky-NOAA Jul 2, 2024
2be9a15
Move extractvars variable defs to config
EricSinsky-NOAA Jul 2, 2024
235886b
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 8, 2024
c2db0a5
Replace COM with COMIN and COMOUT
EricSinsky-NOAA Jul 8, 2024
a140ef6
Revert changes related to offsets
EricSinsky-NOAA Jul 8, 2024
c7f8364
Rename JGLOBAL_EXTRACTVARS
EricSinsky-NOAA Jul 9, 2024
32c25a6
Add comment to explain DO_EXTRACTVARS
EricSinsky-NOAA Jul 9, 2024
d349d66
Polish some if-statements in extractvars
EricSinsky-NOAA Jul 9, 2024
8ef0bd9
Revert "Rename JGLOBAL_EXTRACTVARS"
EricSinsky-NOAA Jul 9, 2024
5310eaa
Simplify com var defs in gefs_atmos_extractvars
EricSinsky-NOAA Jul 9, 2024
1d570ea
Rename oufile in gefs_atmos_extractvars
EricSinsky-NOAA Jul 9, 2024
491be9c
Add varlist as argument for check_atmos
EricSinsky-NOAA Jul 9, 2024
d9a1681
Define a compression utility
EricSinsky-NOAA Jul 9, 2024
a0343a3
Rename oufile1 in gefs_ocnice_extractvars
EricSinsky-NOAA Jul 9, 2024
ac855af
Rename oufile1 in gefs_wav_extractvars
EricSinsky-NOAA Jul 9, 2024
f768627
Remove mode arugment from mkdir
EricSinsky-NOAA Jul 9, 2024
f35452f
Remove FHMAX and use FHMAX_GFS
EricSinsky-NOAA Jul 9, 2024
de3e5e6
Remove duplicate variable in config.extractvars
EricSinsky-NOAA Jul 9, 2024
ef7799b
Remove FHMAXHF, FHMAXLF and FHMAXHF
EricSinsky-NOAA Jul 9, 2024
2f89276
Check that FHMAX_GFS is greater than FHMAX_HF_GFS
EricSinsky-NOAA Jul 9, 2024
beb9e5e
Moved parmfile variables to config.extractvars
EricSinsky-NOAA Jul 9, 2024
55f65d5
Add extractvars function to copy output to COMROOT
EricSinsky-NOAA Jul 9, 2024
98c3c48
Copy ocn, ice and wave files individually
EricSinsky-NOAA Jul 9, 2024
1985224
Split COMOUT RFCST ATMOS into two directories
EricSinsky-NOAA Jul 9, 2024
8610220
Fix shell check errors
EricSinsky-NOAA Jul 9, 2024
4f5db1e
Replace gefs string with RUN variable
EricSinsky-NOAA Jul 9, 2024
701a85c
Replace strings with arithmetic where possible
EricSinsky-NOAA Jul 10, 2024
16bff05
Replace while loops with for loops where possible
EricSinsky-NOAA Jul 10, 2024
3a920be
Simplify some if-statements
EricSinsky-NOAA Jul 10, 2024
d3e8c34
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 10, 2024
1022872
Modify extractvars dependencies
EricSinsky-NOAA Jul 10, 2024
22de8a8
Remove gefs string from ush script filenames
EricSinsky-NOAA Jul 10, 2024
88e4ccf
Update resources used in extractvars
EricSinsky-NOAA Jul 10, 2024
44eb052
Use more efficient method to generate parmlist
EricSinsky-NOAA Jul 11, 2024
bffb227
Add job-specific variable for FHMAX_HF_GFS
EricSinsky-NOAA Jul 11, 2024
39e5582
Remove unnecessary mkdir commands
EricSinsky-NOAA Jul 11, 2024
9a03c58
Move temporary atmos files into subdata directory
EricSinsky-NOAA Jul 11, 2024
ec7a307
Rename varlist in check_atmos function
EricSinsky-NOAA Jul 11, 2024
bee6fa4
Address shell check error and fix error
EricSinsky-NOAA Jul 11, 2024
e565ff0
Add more detail to warning message
EricSinsky-NOAA Jul 11, 2024
9373721
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 11, 2024
90c692f
Fix shell check error
EricSinsky-NOAA Jul 11, 2024
220fce1
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 16, 2024
c968e1f
Declare functions and source in exscript
EricSinsky-NOAA Jul 16, 2024
d86b1e9
Fix indent
EricSinsky-NOAA Jul 16, 2024
1f77718
Replace cp with NCP
EricSinsky-NOAA Jul 16, 2024
89f5621
Replace NCP with cpfs
EricSinsky-NOAA Jul 16, 2024
2e4ca68
Remove source extractvars_tools from ush scripts
EricSinsky-NOAA Jul 16, 2024
f8d24f2
Rename wav_extractvars to wave_extractvars
EricSinsky-NOAA Jul 16, 2024
0bfc12f
Pass dcnt into daily_avg_atmos and add description
EricSinsky-NOAA Jul 16, 2024
519922c
Replace ${cycle} with t${cyc}z
EricSinsky-NOAA Jul 16, 2024
9d3ba90
Pass outres into daily_avg_atmos function
EricSinsky-NOAA Jul 16, 2024
e64a640
Rename davgtmp
EricSinsky-NOAA Jul 16, 2024
f8bd19d
Simplify the declaring of COM variables
EricSinsky-NOAA Jul 16, 2024
db4b432
Rename FHOUT_WAV_NOSCRUB to FHOUT_WAV_EXTRACT
EricSinsky-NOAA Jul 16, 2024
cd1c5d6
Add check for FHOUT_WAV_EXTRACT
EricSinsky-NOAA Jul 16, 2024
16b546d
Remove WGRIB2 definition
EricSinsky-NOAA Jul 16, 2024
0384906
Remove variable pass-throughs
EricSinsky-NOAA Jul 16, 2024
7b523eb
Use variable subdata in ocnice/wave ush scripts
EricSinsky-NOAA Jul 16, 2024
d14c291
Remove unneccesary lines in exglobal_extractvars
EricSinsky-NOAA Jul 16, 2024
a7433a3
Remove gen_parmlist and loop through infiles
EricSinsky-NOAA Jul 17, 2024
68f2714
Loop through infile when creating davg files
EricSinsky-NOAA Jul 17, 2024
655175f
Merge remote-tracking branch 'EMC/develop' into feature/extractvars
EricSinsky-NOAA Jul 17, 2024
a2abff7
Address shellcheck error and add comment
EricSinsky-NOAA Jul 17, 2024
148d8db
Add file availability check in atmos script
EricSinsky-NOAA Jul 17, 2024
ea0bd8c
Add option to extract levels for ocean 3d vars
EricSinsky-NOAA Jul 17, 2024
142450a
Fix shell check error
EricSinsky-NOAA Jul 17, 2024
380dfee
Add quotes
EricSinsky-NOAA Jul 17, 2024
b293a65
Move commas inside double quotes
EricSinsky-NOAA Jul 17, 2024
43a0b10
Remove grib2 support
EricSinsky-NOAA Jul 17, 2024
7bd4b67
Add fatal error if data file does not exist
EricSinsky-NOAA Jul 17, 2024
c7e5d08
Rename FHMAX_HF_EV to FHMAX_HF_GFS
EricSinsky-NOAA Jul 18, 2024
16425cb
Change output directory to ARCDIR
EricSinsky-NOAA Jul 18, 2024
bdb429c
Move mkdirs for arcdir to J-JOB script
EricSinsky-NOAA Jul 18, 2024
1add8e1
Add brief documentation to scripts and functions
EricSinsky-NOAA Jul 18, 2024
bf19e74
Modify description for daily_atmos
EricSinsky-NOAA Jul 18, 2024
c869271
Revert "Modify description for daily_atmos"
EricSinsky-NOAA Jul 18, 2024
f7357ea
Modify and correct description for daily_atmos
EricSinsky-NOAA Jul 18, 2024
f572f6b
Apply suggestions from code review
aerorahul Jul 19, 2024
85e18ed
Remove /dev/null to address reviewer comment
EricSinsky-NOAA Jul 19, 2024
1f40284
Simplify if-statement to address reviewer comment
EricSinsky-NOAA Jul 19, 2024
6d9ccb2
Fix syntax
EricSinsky-NOAA Jul 19, 2024
97c2f8f
Convert if-statement to arithmetic
EricSinsky-NOAA Jul 20, 2024
5aa68a7
Add proper spacing in comments and loops
EricSinsky-NOAA Jul 20, 2024
9a724f4
Change operator in if-statement from < to <=
EricSinsky-NOAA Jul 20, 2024
bffeb25
Add proper spacing to comment in config.base
EricSinsky-NOAA Jul 20, 2024
68b3e24
Merge branch 'develop' into feature/extractvars
aerorahul Jul 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions jobs/JGLOBAL_EXTRACTVARS
DavidHuber-NOAA marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "extractvars" -c "base extractvars"

# Set COM Paths
for grid in '0p25' '0p50' '1p00'; do
prod_dir="COMIN_ATMOS_GRIB_${grid}"
GRID=${grid} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx "${prod_dir}:COM_ATMOS_GRIB_GRID_TMPL"
if [[ ! -d "${!prod_dir}" ]]; then mkdir -p "${!prod_dir}"; fi
done

YMD="${PDY}" HH="${cyc}" declare_from_tmpl -rx \
"COMIN_OCEAN_HISTORY:COM_OCEAN_HISTORY_TMPL" \
"COMIN_OCEAN_GRIB:COM_OCEAN_GRIB_TMPL" \
"COMIN_OCEAN_NETCDF:COM_OCEAN_NETCDF_TMPL" \
"COMIN_ICE_HISTORY:COM_ICE_HISTORY_TMPL" \
"COMIN_ICE_GRIB:COM_ICE_GRIB_TMPL" \
"COMIN_ICE_NETCDF:COM_ICE_NETCDF_TMPL" \
"COMIN_WAVE_GRID:COM_WAVE_GRID_TMPL"

if [[ "${DO_ATM}" == "YES" ]]; then
if [[ ! -d "${ARC_RFCST_PROD_ATMOS_F2D}" ]]; then mkdir -p "${ARC_RFCST_PROD_ATMOS_F2D}"; fi
if [[ ! -d "${ARC_RFCST_PROD_ATMOS_F3D}" ]]; then mkdir -p "${ARC_RFCST_PROD_ATMOS_F3D}"; fi
fi
if [[ "${DO_OCN}" == "YES" ]]; then
if [[ ! -d "${ARC_RFCST_PROD_OCN}" ]]; then mkdir -p "${ARC_RFCST_PROD_OCN}"; fi
fi
if [[ "${DO_ICE}" == "YES" ]]; then
if [[ ! -d "${ARC_RFCST_PROD_ICE}" ]]; then mkdir -p "${ARC_RFCST_PROD_ICE}"; fi
fi
if [[ "${DO_WAVE}" == "YES" ]]; then
if [[ ! -d "${ARC_RFCST_PROD_WAV}" ]]; then mkdir -p "${ARC_RFCST_PROD_WAV}"; fi
fi

# Execute the Script
"${SCRgfs}/exglobal_extractvars.sh"
status=$?
(( status != 0 )) && exit "${status}"

##########################################
# Remove the Temporary working directory
##########################################
cd "${DATAROOT}" || true
[[ "${KEEPDATA}" = "NO" ]] && rm -rf "${DATA}"

exit 0
23 changes: 23 additions & 0 deletions jobs/rocoto/extractvars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#! /usr/bin/env bash

source "${HOMEgfs}/ush/preamble.sh"

###############################################################
echo
echo "=============== START TO SOURCE FV3GFS WORKFLOW MODULES ==============="
. "${HOMEgfs}/ush/load_fv3gfs_modules.sh"
status=$?
[[ "${status}" -ne 0 ]] && exit "${status}"

export job="extractvars"
export jobid="${job}.$$"

###############################################################
echo
echo "=============== START TO RUN EXTRACTVARS ==============="
# Execute the JJOB
"${HOMEgfs}/jobs/JGLOBAL_EXTRACTVARS"
status=$?
[[ "${status}" -ne 0 ]] && exit "${status}"

exit 0
1 change: 1 addition & 0 deletions parm/config/gefs/config.base
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ export DO_WAVE="NO"
export DO_OCN="NO"
export DO_ICE="NO"
export DO_AERO="NO"
export DO_EXTRACTVARS="@DO_EXTRACTVARS@" # Option to process and extract a subset of products to save on disk
export AERO_FCST_CDUMP="" # When to run aerosol forecast: gdas, gfs, or both
export AERO_ANL_CDUMP="" # When to run aerosol analysis: gdas, gfs, or both
export WAVE_CDUMP="" # When to include wave suite: gdas, gfs, or both
Expand Down
41 changes: 41 additions & 0 deletions parm/config/gefs/config.extractvars
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
#! /usr/bin/env bash

########## config.extractvars ##########
# Extractvars specific

echo "BEGIN: config.extractvars"

. "${EXPDIR}/config.resources" extractvars

export COMPRSCMD=${COMPRSCMD:-bzip2}

export compress_ocn=0 #1: Compress extracted ocean product, 0: Do not compress extracted ocean product
export compress_ice=0 #1: Compress extracted ice product, 0: Do not compress extracted ice product

export ocnres="5p00" # Resolution of ocean products
export iceres="5p00" # Resolution of ice products
export wavres="5p00" # Resolution of wave products

export depthvar_name="z_l" # Name of depth variable in NetCDF ocean products
export zmin="0." # Minimum depth to extract from NetCDF ocean products
export zmax="300." # Maximum depth to extract from NetCDF ocean products

export FHOUT_WAV_EXTRACT=6 # Frequency of wave output to be saved on disk

#Paramater Tables used
export varlist_2d="${PARMgfs}/product/gefs_shortparmlist_2d.parm" # Parameter table for surface variables
export varlist_3d="${PARMgfs}/product/gefs_shortparmlist_3d_h.parm" # Parameter table for upper air instantaneous variables
export varlist_3d_d="${PARMgfs}/product/gefs_shortparmlist_3d_d.parm" # Parameter table for upper air daily-averaged variables
export varlist_wav="${PARMgfs}/product/gefs_wav_shortparmlist.parm" # Parameter table for wave variables
export varlist_ocn_netcdf="${PARMgfs}/product/gefs_ocn_shortparmlist.parm" # Parameter table for ocean netcdf variables
export varlist_ice_netcdf="${PARMgfs}/product/gefs_ice_shortparmlist.parm" # Parameter table for ice netcdf variables

#Directory to save extracted variables
export ARC_RFCST_PROD="${ARCDIR}/rfcst/${PDY:0:4}/${PDY:0:6}/${PDY:0:8}/mem${ENSMEM}"
export ARC_RFCST_PROD_ATMOS_F2D="${ARC_RFCST_PROD}/atmos/f2d"
export ARC_RFCST_PROD_ATMOS_F3D="${ARC_RFCST_PROD}/atmos/f3d"
export ARC_RFCST_PROD_OCN="${ARC_RFCST_PROD}/ocn"
export ARC_RFCST_PROD_ICE="${ARC_RFCST_PROD}/ice"
export ARC_RFCST_PROD_WAV="${ARC_RFCST_PROD}/wav"

echo "END: config.extractvars"
12 changes: 12 additions & 0 deletions parm/config/gefs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -293,6 +293,18 @@ case ${step} in
export NTASKS=${npe_wavepostpnt}
;;

"extractvars")
export wtime_extractvars="00:30:00"
export npe_extractvars=1
export nth_extractvars=1
export npe_node_extractvars="${npe_extractvars}"
export wtime_extractvars_gfs="${wtime_extractvars}"
export npe_extractvars_gfs="${npe_extractvars}"
export nth_extractvars_gfs="${nth_extractvars}"
export npe_node_extractvars_gfs="${npe_node_extractvars}"
export is_exclusive=False
;;

*)
echo "FATAL ERROR: Invalid job ${step} passed to ${BASH_SOURCE[0]}"
exit 1
Expand Down
3 changes: 2 additions & 1 deletion parm/config/gefs/yaml/defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ base:
DO_BUFRSND: "NO"
DO_GEMPAK: "NO"
DO_AWIPS: "NO"
KEEPDATA: "NO"
KEEPDATA: "NO"
DO_EXTRACTVARS: "NO"
FHMAX_GFS: 120
FHMAX_HF_GFS: 0
REPLAY_ICS: "NO"
Expand Down
10 changes: 10 additions & 0 deletions parm/product/gefs_ice_shortparmlist.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
aice_h
hi_h
Tsfc_h
uvel_h
vvel_h
hs_h
albsni_h
melts_h
meltb_h
frzmlt_h
9 changes: 9 additions & 0 deletions parm/product/gefs_ocn_shortparmlist.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
temp
SST
SSH
SSS
MLD_003
taux
tauy
SSU
SSV
38 changes: 38 additions & 0 deletions parm/product/gefs_shortparmlist_2d.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
:PRES:surface:
:WEASD:surface:
:TMP:2 m above ground:
:TMP:surface:
:RH:2 m above ground:
:TMAX:2 m above ground:
:TMIN:2 m above ground:
:UGRD:10 m above ground:
:VGRD:10 m above ground:
:APCP:surface:
:CSNOW:surface:
:CICEP:surface:
:CFRZR:surface:
:CRAIN:surface:
:PWAT:entire atmosphere (considered as a single layer):
:TCDC:entire atmosphere (considered as a single layer):
:DSWRF:surface:
:DLWRF:surface:
:ULWRF:top of atmosphere:
:HLCY:3000-0 m above ground:
:CAPE:180-0 mb above ground:
:CIN:180-0 mb above ground:
:PRMSL:mean sea level:
:USWRF:surface:
:ULWRF:surface:
:TSOIL:0-0.1 m below ground:
:TSOIL:0.1-0.4 m below ground:
:SOILW:0-0.1 m below ground:
:SOILW:0.1-0.4 m below ground:
:SOILW:0.4-1 m below ground:
:SOILW:1-2 m below ground:
:PEVPR:surface:
:LHTFL:surface:
:SHTFL:surface:
:WATR:surface:
:TSNOWP:surface:
:FDNSSTMP:surface:
:HGT:highest tropospheric freezing level:
34 changes: 34 additions & 0 deletions parm/product/gefs_shortparmlist_3d_d.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
:UGRD:1 mb:
:UGRD:2 mb:
:UGRD:3 mb:
:UGRD:5 mb:
:UGRD:7 mb:
:UGRD:10 mb:
:UGRD:20 mb:
:UGRD:30 mb:
:UGRD:50 mb:
:UGRD:70 mb:
:VGRD:1 mb:
:VGRD:2 mb:
:VGRD:3 mb:
:VGRD:5 mb:
:VGRD:7 mb:
:VGRD:10 mb:
:VGRD:20 mb:
:VGRD:30 mb:
:VGRD:50 mb:
:VGRD:70 mb:
:TMP:1 mb:
:TMP:2 mb:
:TMP:3 mb:
:TMP:5 mb:
:TMP:7 mb:
:TMP:10 mb:
:TMP:20 mb:
:TMP:30 mb:
:TMP:50 mb:
:TMP:70 mb:
:HGT:10 mb:
:HGT:50 mb:
:O3MR:10 mb:
:O3MR:50 mb:
45 changes: 45 additions & 0 deletions parm/product/gefs_shortparmlist_3d_h.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
:HGT:100 mb:
:TMP:100 mb:
:UGRD:100 mb:
:VGRD:100 mb:
:O3MR:100 mb:
:HGT:200 mb:
:TMP:200 mb:
:RH:200 mb:
:UGRD:200 mb:
:VGRD:200 mb:
:HGT:250 mb:
:TMP:250 mb:
:RH:250 mb:
:UGRD:250 mb:
:VGRD:250 mb:
:HGT:500 mb:
:TMP:500 mb:
:RH:500 mb:
:UGRD:500 mb:
:VGRD:500 mb:
:HGT:700 mb:
:TMP:700 mb:
:RH:700 mb:
:UGRD:700 mb:
:VGRD:700 mb:
:HGT:850 mb:
:TMP:850 mb:
:RH:850 mb:
:VVEL:850 mb:
:UGRD:850 mb:
:VGRD:850 mb:
:HGT:925 mb:
:TMP:925 mb:
:RH:925 mb:
:UGRD:925 mb:
:VGRD:925 mb:
:TMP:1000 mb:
:RH:1000 mb:
:UGRD:1000 mb:
:VGRD:1000 mb:
:HGT:1000 mb:
:TMP:0.995 sigma level:
:RH:0.995 sigma level:
:UGRD:0.995 sigma level:
:VGRD:0.995 sigma level:
3 changes: 3 additions & 0 deletions parm/product/gefs_wav_shortparmlist.parm
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
:UGRD:surface:
:VGRD:surface:
:HTSGW:surface:
53 changes: 53 additions & 0 deletions scripts/exglobal_extractvars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#! /usr/bin/env bash

################################################################################
## UNIX Script Documentation Block
## Script name: exglobal_extractvars.sh
## Script description: Extracts variables from atmosphere, ocean, ice and wave
## products and saves these variables in arcdir
#######################
# Main body starts here
#######################

source "${USHgfs}/preamble.sh"
source "${USHgfs}/extractvars_tools.sh"

# Scripts used
EXTRCTVARA="${USHgfs}/atmos_extractvars.sh"
EXTRCTVARO="${USHgfs}/ocnice_extractvars.sh"
EXTRCTVARW="${USHgfs}/wave_extractvars.sh"

# Set FHMAX_HF_GFS equal to FHMAX_GFS if FHMAX_HF_GFS is greater than FHMAX_GFS
if (( FHMAX_GFS < FHMAX_HF_GFS )); then
export FHMAX_HF_GFS=${FHMAX_GFS}
fi

# Set FHOUT_WAV_EXTRACT equal to FHOUT_WAV if FHOUT_WAV is not a factor of FHOUT_WAV_EXTRACT
if (( FHOUT_WAV_EXTRACT % FHOUT_WAV != 0 )); then
FHOUT_WAV_EXTRACT=${FHOUT_WAV}
fi

# Extract variables for atmosphere
if [[ "${DO_ATM}" == "YES" ]]; then
${EXTRCTVARA} "${DATA}/atmos"
fi

# Extract variables for ocean
if [[ "${DO_OCN}" == "YES" ]]; then
export component_name="ocn"
${EXTRCTVARO} "${DATA}/ocn" "${varlist_ocn_netcdf}" "${ocnres}" "${compress_ocn}" "${FHOUT_OCN_GFS}" "${ARC_RFCST_PROD_OCN}"
fi

# Extract variables for ice
if [[ "${DO_ICE}" == "YES" ]]; then
export component_name="ice"
${EXTRCTVARO} "${DATA}/ice" "${varlist_ice_netcdf}" "${iceres}" "${compress_ice}" "${FHOUT_ICE_GFS}" "${ARC_RFCST_PROD_ICE}"
fi

# Extract variables for wave
if [[ "${DO_WAVE}" == "YES" ]]; then
export component_name="wav"
${EXTRCTVARW} "${DATA}/wav"
fi

exit 0
Loading
Loading