Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make --closedcontourcmd resolution specific #824

Merged
merged 6 commits into from
Oct 15, 2024
Merged

make --closedcontourcmd resolution specific #824

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
18cb9c6
make --closedcontourcmd resolution specific
chengzhuzhang Jul 19, 2024
a56320b
use pm-cpu
chengzhuzhang Jul 23, 2024
9fc48e7
update syntax, add data path for v3
chengzhuzhang Jul 26, 2024
e3a1f76
update example
chengzhuzhang Aug 23, 2024
a6331c8
update tc_analysis pre-processing and run scripts
chengzhuzhang Oct 15, 2024
ff783fa
Merge branch 'main' into res_specific_TC_detection
chengzhuzhang Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions auxiliary_tools/run_tc_analysis.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import os
from e3sm_diags.parameter.core_parameter import CoreParameter
from e3sm_diags.parameter.tc_analysis_parameter import TCAnalysisParameter
from e3sm_diags.run import runner

simulations = ['tc-v2.LR.amip.2000_2014']
sim_names = ['v2.LR.amip_0101']

data_path = '/global/homes/c/chengzhu/tests/tc_analysis/'

for idx, sim in enumerate(simulations):
print(sim)
param = CoreParameter()
param.test_data_path = data_path+sim
param.test_name = sim_names[idx]
param.test_start_yr = "2000"
param.test_end_yr = "2014"

param.reference_data_path = '/global/cfs/cdirs/e3sm/diagnostics/observations/Atm/tc-analysis'
param.ref_start_yr = '1979'
param.ref_end_yr = '2018'

prefix = f'/global/cfs/cdirs/e3sm/www/chengzhu/tc_analysis_test/'
param.results_dir = os.path.join(prefix, sim)
runner.sets_to_run = ['tc_analysis']
runner.run_diags([param])
4 changes: 3 additions & 1 deletion examples/use_zppy_to_run_e3sm_diags/zstash_extract_E3SM_data_from_NERSC_HPSS.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
# $ screen

# Active e3sm_unified to use zstash
source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_cori-haswell.sh
source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh

# Make a new folder to recieve extracted datasets.
mkdir -p /global/cfs/cdirs/e3smpub/E3SM_simulations/v2.LR.historical_0101
Expand All @@ -18,6 +18,8 @@ cd /global/cfs/cdirs/e3smpub/E3SM_simulations/v2.LR.historical_0101

# retrive h0(monthly) atmospheric data, size = 821 G
zstash extract --hpss=/home/projects/e3sm/www/WaterCycle/E3SMv2/LR/v2.LR.historical_0101 "archive/atm/hist/*h0*"
# retrive h1(daily) atmospheric data
zstash extract --hpss=/home/projects/e3sm/www/WaterCycle/E3SMv2/LR/v2.LR.historical_0101 "archive/atm/hist/*h1*"
# retrive h2(6 hourly) atmospheric data, size = 195 G [Optional] only needed for Tropical Cycle Analysis
zstash extract --hpss=/home/projects/e3sm/www/WaterCycle/E3SMv2/LR/v2.LR.historical_0101 "archive/atm/hist/*h2*"
# retrive h4(3 hourly) atmospheric data, size = 195 G [Optional] only needed for Diurnal Cycle of Precipitation
Expand Down
72 changes: 28 additions & 44 deletions model_data_preprocess/postprocessing_E3SM_data_for_TC_analysis.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,100 +1,84 @@
#!/bin/bash

# A Bash script to post-process E3SM 6 hourly (h2) instantaneous output to generate a text file storing Tropical Cyclone tracks
# tempestremap and tempestextremes are built in e3sm-unified from version 1.5.0. This script has been tested using Cori login node.
# To request an interactive session as follows: salloc --nodes=1 --partition=debug --time=30:00 -C haswell
# tempestremap and tempestextremes are built in e3sm-unified from version 1.5.0. This script has been tested using Perlmutter login node.
# To request an interactive session as follows: salloc --nodes=1 --partition=debug --time=30:00 -C cpu

source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_cori-haswell.sh
source /global/common/software/e3sm/anaconda_envs/load_latest_e3sm_unified_pm-cpu.sh

#For typical EAM v2 ne gp2 grids.
start="0051"
end="0060"
caseid="20210528.v2rc3e.piControl.ne30pg2_EC30to60E2r2.chrysalis"
drc_in=/global/cfs/cdirs/e3smdata/zppy_complete_run_nersc_output/${caseid}/archive/atm/hist
# Warning: because somehow tempest-remap only can writes grid file on SCRATCH space. The resulted files will be moved to another path at the end.
result_dir_fin=/global/cfs/cdirs/e3sm/e3sm_diags/postprocessed_e3sm_v2_data_for_e3sm_diags/${caseid}/
mkdir -p $result_dir_fin
result_dir=$SCRATCH/tests/tc-analysis/
Comment on lines -14 to -17
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With Perlmutter, we don't need to use $SCRATCH anymore? For zppy, we were using it for all machines:

# Warning: tempest-remap can only write grid file on SCRATCH space.
# The result files will be moved to another path at the end.
result_dir_fin={{ output }}/post/atm # Directory will be {{ output }}/post/atm/tc-analysis_${Y1}_${Y2}
mkdir -p $result_dir_fin
y1={{ year1 }}
y2={{ year2 }}
Y1="{{ '%04d' % (year1) }}"
Y2="{{ '%04d' % (year2) }}"
result_dir={{ scratch }}/tc-analysis_${Y1}_${Y2}/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I believe we don't need to use scratch any more, with fixing over-writing mesh file. At least, my tests worked okay.


start="2000"
end="2014"
caseid="v2.LR.amip_0101"
drc_in=/global/cfs/cdirs/e3smpub/E3SM_simulations/${caseid}/archive/atm/hist
result_dir=/global/homes/c/chengzhu/tests/tc_analysis/tc-v2.LR.amip.${start}_${end}/

#res = 30/120 for ne30/ne120 grids
res=30 # res = 30/120 for ne30/ne120 grids
pg2=true # Set false for v1 production simulations
atm_name="eam" # Use "cam" for v1 production simulations


##For typical v1 ne SE grids.
#start="1880"
#end="1881"
#caseid="20180215.DECKv1b_H1.ne30_oEC.edison"
#drc_in=/global/cfs/cdirs/e3smpub/E3SM_simulations/${caseid}/archive/atm/hist
#result_dir=/global/cfs/cdirs/e3sm/e3sm_diags/postprocessed_e3sm_v1_data_for_e3sm_diags/${caseid}/tc-analysis/
#
##res = 30/120 for ne30/ne120 grids
#res=30 # res = 30/120 for ne30/ne120 grids
#pg2=false # Set false for v1 production simulations
#atm_name="cam" # Use "cam" for v1 production simulations



mkdir -p $result_dir
file_name=${caseid}_${start}_${end}

# Generate mesh files (.g).
#/global/homes/p/paullric/tempestremap/bin/GenerateCSMesh --res $res --alt --file ${result_dir}outCSne$res.g
GenerateCSMesh --res $res --alt --file ${result_dir}outCSne$res.g
out_type="CGLL"
# For v2 production simulation with pg2 grids:
if $pg2; then
GenerateVolumetricMesh --in ${result_dir}outCSne$res.g --out ${result_dir}outCSne$res.g --np 2 --uniform
# For v2 and v3 production simulation with pg2 grids:
GenerateCSMesh --res $res --alt --file ${result_dir}outCSMeshne$res.g
GenerateVolumetricMesh --in ${result_dir}outCSMeshne$res.g --out ${result_dir}outCSne$res.g --np 2 --uniform
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--in and --out file names were the same before, which I suspect caused the intermittent issue when mesh file can't be written on cfs.

out_type="FV"
else
# For v1 production simulation with np4 grids:
GenerateCSMesh --res $res --alt --file ${result_dir}outCSne$res.g
out_type="CGLL"
fi
echo $out_type
# Generate connectivity files (.dat)
#/global/homes/p/paullric/tempestextremes/bin/GenerateConnectivityFile --in_mesh ${result_dir}outCSne$res.g --out_type CGLL --out_connect ${result_dir}connect_CSne${res}_v2.dat
GenerateConnectivityFile --in_mesh ${result_dir}outCSne$res.g --out_type $out_type --out_connect ${result_dir}connect_CSne${res}_v2.dat

# Get the list of files
cd ${drc_in};eval ls ${drc_in}/${caseid}.$atm_name.h2.*{${start}..${end}}*.nc >${result_dir}inputfile_${file_name}.txt
cd ${drc_in};eval ls ${caseid}.$atm_name.h2.*{${start}..${end}}*.nc >${result_dir}outputfile_${file_name}.txt
#cd ${drc_in};eval ls ${drc_in}/${caseid}.cam.h2.*{${start}..${end}}*.nc >${result_dir}inputfile_${file_name}.txt
#cd ${drc_in};eval ls ${caseid}.cam.h2.*{${start}..${end}}*.nc >${result_dir}outputfile_${file_name}.txt

cd ${result_dir}
# Detection threshold including:
# The sea-level pressure (SLP) must be a local minimum; SLP must have a sufficient decrease (300 Pa) compared to surrounding nodes within 4 degree radius; The average of the 200 hPa and 500 hPa level temperature decreases by 0.6 K in all directions within a 4 degree radius from the location to fSLP minima
#/global/homes/p/paullric/tempestextremes/bin/DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "PSL,300.0,4.0,0;_AVG(T200,T500),-0.6,4,0.30" --mergedist 6.0 --searchbymin PSL --outputcmd "PSL,min,0;_VECMAG(UBOT,VBOT),max,2" --timestride 1 --in_data_list ${result_dir}inputfile_${file_name}.txt --out ${result_dir}out.dat
DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "PSL,300.0,4.0,0;_AVG(T200,T500),-0.6,4,0.30" --mergedist 6.0 --searchbymin PSL --outputcmd "PSL,min,0;_VECMAG(UBOT,VBOT),max,2" --timestride 1 --in_data_list ${result_dir}inputfile_${file_name}.txt --out ${result_dir}out.dat

# --closedcontourcmd threshold is resolution depended.
if [ $res == 120 ]; then
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lines onward set DetectNodes command line resolution depended with the last parameter in --closedcontourcmd

echo $res
DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "PSL,300.0,4.0,0;_AVG(T200,T500),-0.6,4,0.30" --mergedist 6.0 --searchbymin PSL --outputcmd "PSL,min,0;_VECMAG(UBOT,VBOT),max,2" --timestride 1 --in_data_list ${result_dir}inputfile_${file_name}.txt --out ${result_dir}out.dat
elif [ $res == 30 ]; then
echo $res
DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "PSL,300.0,4.0,0;_AVG(T200,T500),-0.6,4,1.0" --mergedist 6.0 --searchbymin PSL --outputcmd "PSL,min,0;_VECMAG(UBOT,VBOT),max,2" --timestride 1 --in_data_list ${result_dir}inputfile_${file_name}.txt --out ${result_dir}out.dat
else
echo “$res value not supported”
fi

cat ${result_dir}out.dat0* > ${result_dir}cyclones_${file_name}.txt

# Stitch all candidate nodes in time to form tracks, with a maximum distance between candidates of 6.0, minimum time steps of 6, and with a maximum gap size of one (most consecutive time steps with no associated candidate). And there is threshold for wind speed, lat and lon.
#/global/homes/p/paullric/tempestextremes/bin/StitchNodes --in_fmt "lon,lat,slp,wind" --in_connect ${result_dir}connect_CSne${res}_v2.dat --range 6.0 --mintime 6 --maxgap 1 --in ${result_dir}cyclones_${file_name}.txt --out ${result_dir}cyclones_stitch_${file_name}.dat --threshold "wind,>=,17.5,6;lat,<=,40.0,6;lat,>=,-40.0,6"
StitchNodes --in_fmt "lon,lat,slp,wind" --in_connect ${result_dir}connect_CSne${res}_v2.dat --range 6.0 --mintime 6 --maxgap 1 --in ${result_dir}cyclones_${file_name}.txt --out ${result_dir}cyclones_stitch_${file_name}.dat --threshold "wind,>=,17.5,6;lat,<=,40.0,6;lat,>=,-40.0,6"
rm ${result_dir}cyclones_${file_name}.txt

# Generate histogram of detections
#/global/homes/p/paullric/tempestextremes/bin/HistogramNodes --in ${result_dir}cyclones_stitch_${file_name}.txt --iloncol 2 --ilatcol 3 --out ${result_dir}cyclones_hist_${file_name}.nc
HistogramNodes --in ${result_dir}cyclones_stitch_${file_name}.dat --iloncol 2 --ilatcol 3 --out ${result_dir}cyclones_hist_${file_name}.nc

# Calculate relative vorticity
sed -i 's/.nc/_vorticity.nc/' ${result_dir}outputfile_${file_name}.txt
#/global/homes/p/paullric/tempestextremes/bin/VariableProcessor --in_data_list ${result_dir}inputfile_${file_name}.txt --out_data_list ${result_dir}outputfile_${file_name}.txt --var "_CURL{4,0.5}(U850,V850)" --varout "VORT" --in_connect ${result_dir}connect_CSne${res}_v2.dat
sed -i 's/\.nc/_vorticity.nc/' ${result_dir}outputfile_${file_name}.txt
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update sed command line to recognize special character ., this bug was discovered when nc is included in the case name in addition as the format name at the end.

VariableProcessor --in_data_list ${result_dir}inputfile_${file_name}.txt --out_data_list ${result_dir}outputfile_${file_name}.txt --var "_CURL{4,0.5}(U850,V850)" --varout "VORT" --in_connect ${result_dir}connect_CSne${res}_v2.dat

#/global/homes/p/paullric/tempestextremes/bin/DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "VORT,-5.e-6,4,0" --mergedist 2.0 --searchbymax VORT --outputcmd "VORT,max,0" --in_data_list ${result_dir}outputfile_${file_name}.txt --out ${result_dir}aew_out.dat --minlat -35.0 --maxlat 35.0
DetectNodes --verbosity 0 --in_connect ${result_dir}connect_CSne${res}_v2.dat --closedcontourcmd "VORT,-5.e-6,4,0" --mergedist 2.0 --searchbymax VORT --outputcmd "VORT,max,0" --in_data_list ${result_dir}outputfile_${file_name}.txt --out ${result_dir}aew_out.dat --minlat -35.0 --maxlat 35.0
cat ${result_dir}aew_out.dat0* > ${result_dir}aew_${file_name}.txt

#/global/homes/p/paullric/tempestextremes/bin/StitchNodes --in_fmt "lon,lat,VORT" --in_connect ${result_dir}connect_CSne${res}_v2.dat --range 3.0 --minlength 8 --maxgap 0 --min_endpoint_dist 10.0 --in ${result_dir}aew_${file_name}.txt --out ${result_dir}aew_stitch_5e-6_${file_name}.dat --threshold "lat,<=,25.0,8;lat,>=,0.0,8"
StitchNodes --in_fmt "lon,lat,VORT" --in_connect ${result_dir}connect_CSne${res}_v2.dat --range 3.0 --minlength 8 --maxgap 0 --min_endpoint_dist 10.0 --in ${result_dir}aew_${file_name}.txt --out ${result_dir}aew_stitch_5e-6_${file_name}.dat --threshold "lat,<=,25.0,8;lat,>=,0.0,8"
rm ${result_dir}aew_${file_name}.txt

#/global/homes/p/paullric/tempestextremes/bin/HistogramNodes --in ${result_dir}aew_stitch_5e-6_${file_name}.dat --iloncol 2 --ilatcol 3 --nlat 256 --nlon 512 --out ${result_dir}aew_hist_${file_name}.nc
HistogramNodes --in ${result_dir}aew_stitch_5e-6_${file_name}.dat --iloncol 2 --ilatcol 3 --nlat 256 --nlon 512 --out ${result_dir}aew_hist_${file_name}.nc

rm ${result_dir}*out.dat00*.dat
rm ${result_dir}${caseid}*.nc
rm ${result_dir}*.txt
mv $result_dir $result_dir_fin

exit
Loading