Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revamp job_id, take 2 #2994

Merged
merged 1 commit into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 109 additions & 16 deletions .buildkite/pipeline.yml

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ ClimaAtmos.jl Release Notes

Main
-------
- ![][badge-💥breaking]. CPU/GPU runs can now share the same yaml files. The driver now calls `AtmosConfig` via `(; config_file, job_id) = ClimaAtmos.commandline_kwargs(); config = ClimaAtmos.AtmosConfig(config_file; job_id)`, which recovers the original behavior. PR [#2994](https://github.com/CliMA/ClimaAtmos.jl/pull/2994), issue [#2651](https://github.com/CliMA/ClimaAtmos.jl/issues/2651).
- Move config files for gpu jobs on ci to config/model_configs/.
PR [#2948](https://github.com/CliMA/ClimaAtmos.jl/pull/2948).

Expand Down
46 changes: 43 additions & 3 deletions config/default_configs/default_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -196,9 +196,6 @@ perf_summary:
viscous_sponge:
help: "Viscous sponge [`true`, `false` (default)]"
value: false
job_id:
help: "Uniquely identifying string for a particular job"
value: ~
tracer_upwinding:
help: "Tracer upwinding mode [`none` (default), `first_order` , `third_order`, `boris_book`, `zalesak`]"
value: none
Expand Down Expand Up @@ -290,3 +287,46 @@ prescribed_aerosols:
call_cloud_diagnostics_per_stage:
help: "A temporary Bool for calling cloud diagnostics every stage"
value: false
# EDMF configuration
turbconv:
help: "Turbulence convection scheme [`nothing` (default), 'prognostic_edmfx', 'diagnostic_edmfx']"
value: ~
advection_test:
help: "Switches off all grid-scale and subgrid-scale momentum tendencies [`false` (default), `true`]"
value: false
zero_tendency:
help: "Turns off tendencies [`nothing` (default), `grid_scale`, `subgrid_scale`]"
value: ~
implicit_sgs_advection:
help: "Whether to treat the subgrid-scale vertical advection tendency implicitly [`false` (default), `true`]"
value: false
edmf_coriolis:
help: "EDMF coriolis [`nothing` (default), `Bomex`,`LifeCycleTan2018`,`Rico`,`ARM_SGP`,`DYCOMS_RF01`,`DYCOMS_RF02`,`GABLS`]"
value: ~
edmfx_filter:
help: "If set to true, it switches on the relaxation of negative velocity in EDMFX. [`true`, `false` (default)]"
value: false
edmfx_nh_pressure:
help: "If set to true, it switches on EDMFX pressure drag closure. [`true`, `false` (default)]"
value: false
edmfx_entr_model:
help: "EDMFX entrainment closure. [`nothing` (default), `PiGroups`, `Generalized`, `GeneralizedHarmonics`]"
value: ~
edmfx_detr_model:
help: "EDMFX detrainment closure. [`nothing` (default), `PiGroups`, `Generalized`, `GeneralizedHarmonics`]"
value: ~
edmfx_upwinding:
help: "EDMFX upwinding mode [`none` (default), `first_order`, `third_order`]"
value: none
edmfx_sgsflux_upwinding:
help: "EDMFX SGS mass flux upwinding mode [`none` (default), `first_order`, `third_order`]"
value: none
edmfx_sgs_mass_flux:
help: "If set to true, it switches on EDMFX SGS mass flux. [`true`, `false` (default)]"
value: false
edmfx_sgs_diffusive_flux:
help: "If set to true, it switches on EDMFX SGS diffusive flux. [`true`, `false` (default)]"
value: false
updraft_number:
help: "Sets the number of updrafts for the EDMF scheme"
value: 1
43 changes: 0 additions & 43 deletions config/default_configs/default_edmf_config.yml

This file was deleted.

2 changes: 1 addition & 1 deletion config/default_configs/default_perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ precip_model: "0M"
dt_save_to_sol: "Inf"
rad: "allskywithclear"
warn_allocations_diagnostics: true
log_progress: false
log_progress: false
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_diagedmf.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: gpu_aquaplanet_diagedmf
dt_save_state_to_disk: "Inf"
dt_save_to_sol: "Inf"
output_default_diagnostics: false
Expand Down
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_1process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: DefaultMoninObukhov
rayleigh_sponge: true
dt: 100secs
t_end: 12hours
job_id: gpu_aquaplanet_dyamond_ss_1process
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_2process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ss_2process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_4process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ss_4process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_1process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_1process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_2process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_2process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_4process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_4process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_progedmf.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: gpu_aquaplanet_progedmf
dt_save_state_to_disk: "Inf"
dt_save_to_sol: "Inf"
output_default_diagnostics: false
Expand Down
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_hs_rhoe_equil_55km_nz63_0M.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "gpu_hs_rhoe_equil_55km_nz63_0M_4process"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/target_gpu_implicit_baroclinic_wave.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ z_elem: 63
dz_bottom: 30.0
dz_top: 3000.0
dt: "150secs"
job_id: "target_gpu_implicit_baroclinic_wave"
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ z_elem: 63
dz_bottom: 30.0
dz_top: 3000.0
dt: "150secs"
job_id: "target_gpu_implicit_baroclinic_wave_4process"
1 change: 0 additions & 1 deletion config/longrun_configs/amip_target_diagedmf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,5 @@ edmfx_sgs_diffusive_flux: true
rayleigh_sponge: true
dt: "100secs"
t_end: "40days"
job_id: "amip_target_diagedmf"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
netcdf_output_at_levels: true
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_aquaplanet_clearsky_1M.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ implicit_diffusion: true
approximate_linear_solve_iters: 2
rad: "clearsky"
dt_rad: "6hours"
job_id: "longrun_aquaplanet_clearsky_1M"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_aquaplanet_dyamond.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "50secs"
t_end: "10days"
job_id: "longrun_aquaplanet_dyamond"
toml: [toml/longrun_aquaplanet_dyamond.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_allsky_diagedmf_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ implicit_diffusion: true
approximate_linear_solve_iters: 2
rad: "clearsky"
dt_rad: "6hours"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_top: 3000.0
h_elem: 16
z_elem: 63
z_max: 55000.0
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M_earth"
moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_top: 3000.0
h_elem: 16
z_elem: 63
z_max: 55000.0
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M_earth_sleve"
moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "30days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_diffonly_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "30days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_progedmf_diffonly_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_progedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ albedo_model: "RegressionFunctionAlbedo"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_tvinsol_0M_slabocean"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ surface_setup: "DefaultExchangeCoefficients"
moist: "equil"
rad: "gray"
precip_model: "0M"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,3 @@ dt_rad: "1hours"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "longrun_aquaplanet_rhoe_equil_clearsky_tvinsol_0M_slabocean"
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_bw_rhoe_equil_highres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ t_end: "100days"
dz_bottom: 30.0
h_elem: 16
precip_model: "0M"
job_id: "longrun_bw_rhoe_equil_highres"
moist: "equil"
diagnostics:
- short_name: [pfull, wa, va, rv, hus, ke]
Expand Down
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_bw_rhoe_highres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ initial_condition: "DryBaroclinicWave"
t_end: "100days"
z_elem: 45
dt: "400secs"
job_id: "longrun_bw_rhoe_highres"
diagnostics:
- short_name: [pfull, wa, va, rv, ke]
period: 1days
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_hs_rhoe_dry_55km_nz63.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,4 @@ z_max: 55000.0
forcing: "held_suarez"
dt: "300secs"
rayleigh_sponge: true
job_id: "longrun_hs_rhoe_dry_55km_nz63"
toml: [toml/longrun_hs_rhoe_dry_55km_nz63.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "longrun_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,4 @@ precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
deep_atmosphere: true
job_id: "longrun_hs_rhoe_equil_55km_nz63_0M_deepatmos"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,3 @@ perturb_initstate: false
discrete_hydrostatic_balance: true
t_end: "800days"
dt_save_state_to_disk: "10days"
job_id: "longrun_sphere_hydrostatic_balance_rhoe"
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_bottom: 30.0
h_elem: 16
ode_algo: "SSP333"
precip_model: "0M"
job_id: "longrun_ssp_bw_rhoe_equil_highres"
moist: "equil"
apply_limiter: true
diagnostics:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_bottom: 30.0
h_elem: 16
energy_upwinding: zalesak
precip_model: "0M"
job_id: "longrun_zalesak_tracer_energy_bw_rhoe_equil_highres"
moist: "equil"
apply_limiter: true
diagnostics:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,3 @@ dt_rad: "1hours"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "aquaplanet_rhoe_equil_clearsky_tvinsol_0M_slabocean"
1 change: 0 additions & 1 deletion config/model_configs/box_density_current_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ config: "box"
vorticity_hyperdiffusion_coefficient: 0.6
scalar_hyperdiffusion_coefficient: 0.6
z_max: 6400.0
job_id: "box_density_current_test"
diagnostics:
- short_name: thetaa
period: 10secs
1 change: 0 additions & 1 deletion config/model_configs/box_hydrostatic_balance_rhoe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ config: "box"
hyperdiff: "false"
dt: "20secs"
perturb_initstate: false
job_id: "box_hydrostatic_balance_rhoe"
diagnostics:
- short_name: [wa, ua]
period: 1days
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
call_cloud_diagnostics_per_stage: true
job_id: "central_cloud_diag_gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "central_gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ dt_save_state_to_disk: "2days"
regression_test: true
initial_condition: "MoistBaroclinicWave"
dt: "450secs"
job_id: "deep_sphere_baroclinic_wave_rhoe_equilmoist"
deep_atmosphere: true
moist: "equil"
diagnostics:
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_aquaplanet.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_aquaplanet
surface_setup: DefaultMoninObukhov
rad: clearsky
turbconv: diagnostic_edmfx
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_bomex_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: "diagnostic_edmfx_bomex_box"
initial_condition: "Bomex"
subsidence: "Bomex"
edmf_coriolis: "Bomex"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: "diagnostic_edmfx_bomex_stretched_box"
initial_condition: "Bomex"
subsidence: "Bomex"
edmf_coriolis: "Bomex"
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_dycoms_rf01_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_dycoms_rf01_box
initial_condition: DYCOMS_RF01
subsidence: DYCOMS
edmf_coriolis: DYCOMS_RF01
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_dycoms_rf01_explicit_box
initial_condition: DYCOMS_RF01
subsidence: DYCOMS
edmf_coriolis: DYCOMS_RF01
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_gabls_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_gabls_box
initial_condition: GABLS
edmf_coriolis: GABLS
surface_setup: GABLS
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_rico_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_rico_box
initial_condition: Rico
subsidence: Rico
ls_adv: Rico
Expand Down
Loading
Loading