Skip to content

Commit

Permalink
Merge pull request #2994 from CliMA/ck/revamp_job_ids_take2
Browse files Browse the repository at this point in the history
Revamp `job_id`, take 2
  • Loading branch information
charleskawczynski authored May 9, 2024
2 parents e90f40e + 1e3e6d0 commit e0fe1cd
Show file tree
Hide file tree
Showing 152 changed files with 384 additions and 391 deletions.
125 changes: 109 additions & 16 deletions .buildkite/pipeline.yml

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ ClimaAtmos.jl Release Notes

Main
-------
- ![][badge-💥breaking]. CPU/GPU runs can now share the same yaml files. The driver now calls `AtmosConfig` via `(; config_file, job_id) = ClimaAtmos.commandline_kwargs(); config = ClimaAtmos.AtmosConfig(config_file; job_id)`, which recovers the original behavior. PR [#2994](https://github.com/CliMA/ClimaAtmos.jl/pull/2994), issue [#2651](https://github.com/CliMA/ClimaAtmos.jl/issues/2651).
- Move config files for gpu jobs on ci to config/model_configs/.
PR [#2948](https://github.com/CliMA/ClimaAtmos.jl/pull/2948).

Expand Down
46 changes: 43 additions & 3 deletions config/default_configs/default_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -196,9 +196,6 @@ perf_summary:
viscous_sponge:
help: "Viscous sponge [`true`, `false` (default)]"
value: false
job_id:
help: "Uniquely identifying string for a particular job"
value: ~
tracer_upwinding:
help: "Tracer upwinding mode [`none` (default), `first_order` , `third_order`, `boris_book`, `zalesak`]"
value: none
Expand Down Expand Up @@ -290,3 +287,46 @@ prescribed_aerosols:
call_cloud_diagnostics_per_stage:
help: "A temporary Bool for calling cloud diagnostics every stage"
value: false
# EDMF configuration
turbconv:
help: "Turbulence convection scheme [`nothing` (default), 'prognostic_edmfx', 'diagnostic_edmfx']"
value: ~
advection_test:
help: "Switches off all grid-scale and subgrid-scale momentum tendencies [`false` (default), `true`]"
value: false
zero_tendency:
help: "Turns off tendencies [`nothing` (default), `grid_scale`, `subgrid_scale`]"
value: ~
implicit_sgs_advection:
help: "Whether to treat the subgrid-scale vertical advection tendency implicitly [`false` (default), `true`]"
value: false
edmf_coriolis:
help: "EDMF coriolis [`nothing` (default), `Bomex`,`LifeCycleTan2018`,`Rico`,`ARM_SGP`,`DYCOMS_RF01`,`DYCOMS_RF02`,`GABLS`]"
value: ~
edmfx_filter:
help: "If set to true, it switches on the relaxation of negative velocity in EDMFX. [`true`, `false` (default)]"
value: false
edmfx_nh_pressure:
help: "If set to true, it switches on EDMFX pressure drag closure. [`true`, `false` (default)]"
value: false
edmfx_entr_model:
help: "EDMFX entrainment closure. [`nothing` (default), `PiGroups`, `Generalized`, `GeneralizedHarmonics`]"
value: ~
edmfx_detr_model:
help: "EDMFX detrainment closure. [`nothing` (default), `PiGroups`, `Generalized`, `GeneralizedHarmonics`]"
value: ~
edmfx_upwinding:
help: "EDMFX upwinding mode [`none` (default), `first_order`, `third_order`]"
value: none
edmfx_sgsflux_upwinding:
help: "EDMFX SGS mass flux upwinding mode [`none` (default), `first_order`, `third_order`]"
value: none
edmfx_sgs_mass_flux:
help: "If set to true, it switches on EDMFX SGS mass flux. [`true`, `false` (default)]"
value: false
edmfx_sgs_diffusive_flux:
help: "If set to true, it switches on EDMFX SGS diffusive flux. [`true`, `false` (default)]"
value: false
updraft_number:
help: "Sets the number of updrafts for the EDMF scheme"
value: 1
43 changes: 0 additions & 43 deletions config/default_configs/default_edmf_config.yml

This file was deleted.

2 changes: 1 addition & 1 deletion config/default_configs/default_perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ precip_model: "0M"
dt_save_to_sol: "Inf"
rad: "allskywithclear"
warn_allocations_diagnostics: true
log_progress: false
log_progress: false
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_diagedmf.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: gpu_aquaplanet_diagedmf
dt_save_state_to_disk: "Inf"
dt_save_to_sol: "Inf"
output_default_diagnostics: false
Expand Down
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_1process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: DefaultMoninObukhov
rayleigh_sponge: true
dt: 100secs
t_end: 12hours
job_id: gpu_aquaplanet_dyamond_ss_1process
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_2process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ss_2process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ss_4process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ss_4process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_1process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_1process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_2process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_2process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_dyamond_ws_4process.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,4 @@ surface_setup: "DefaultMoninObukhov"
rayleigh_sponge: true
dt: "100secs"
t_end: "12hours"
job_id: "gpu_aquaplanet_dyamond_ws_4process"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_aquaplanet_progedmf.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: gpu_aquaplanet_progedmf
dt_save_state_to_disk: "Inf"
dt_save_to_sol: "Inf"
output_default_diagnostics: false
Expand Down
1 change: 0 additions & 1 deletion config/gpu_configs/gpu_hs_rhoe_equil_55km_nz63_0M.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "gpu_hs_rhoe_equil_55km_nz63_0M_4process"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
1 change: 0 additions & 1 deletion config/gpu_configs/target_gpu_implicit_baroclinic_wave.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ z_elem: 63
dz_bottom: 30.0
dz_top: 3000.0
dt: "150secs"
job_id: "target_gpu_implicit_baroclinic_wave"
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,3 @@ z_elem: 63
dz_bottom: 30.0
dz_top: 3000.0
dt: "150secs"
job_id: "target_gpu_implicit_baroclinic_wave_4process"
1 change: 0 additions & 1 deletion config/longrun_configs/amip_target_diagedmf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,5 @@ edmfx_sgs_diffusive_flux: true
rayleigh_sponge: true
dt: "100secs"
t_end: "40days"
job_id: "amip_target_diagedmf"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
netcdf_output_at_levels: true
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_aquaplanet_clearsky_1M.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ implicit_diffusion: true
approximate_linear_solve_iters: 2
rad: "clearsky"
dt_rad: "6hours"
job_id: "longrun_aquaplanet_clearsky_1M"
toml: [toml/longrun_aquaplanet_dyamond.toml]
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_aquaplanet_dyamond.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "50secs"
t_end: "10days"
job_id: "longrun_aquaplanet_dyamond"
toml: [toml/longrun_aquaplanet_dyamond.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_allsky_diagedmf_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,4 @@ implicit_diffusion: true
approximate_linear_solve_iters: 2
rad: "clearsky"
dt_rad: "6hours"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_top: 3000.0
h_elem: 16
z_elem: 63
z_max: 55000.0
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M_earth"
moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_top: 3000.0
h_elem: 16
z_elem: 63
z_max: 55000.0
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_0M_earth_sleve"
moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "30days"
dt: "100secs"
t_end: "60days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_diffonly_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_diagedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -26,5 +26,4 @@ rayleigh_sponge: true
dt_save_state_to_disk: "10days"
dt: "100secs"
t_end: "30days"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_progedmf_diffonly_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_progedmf_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,4 @@ albedo_model: "RegressionFunctionAlbedo"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_clearsky_tvinsol_0M_slabocean"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ surface_setup: "DefaultExchangeCoefficients"
moist: "equil"
rad: "gray"
precip_model: "0M"
job_id: "longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M"
toml: [toml/longrun_aquaplanet_rhoe_equil_55km_nz63_gray_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,3 @@ dt_rad: "1hours"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "longrun_aquaplanet_rhoe_equil_clearsky_tvinsol_0M_slabocean"
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_bw_rhoe_equil_highres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ t_end: "100days"
dz_bottom: 30.0
h_elem: 16
precip_model: "0M"
job_id: "longrun_bw_rhoe_equil_highres"
moist: "equil"
diagnostics:
- short_name: [pfull, wa, va, rv, hus, ke]
Expand Down
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_bw_rhoe_highres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ initial_condition: "DryBaroclinicWave"
t_end: "100days"
z_elem: 45
dt: "400secs"
job_id: "longrun_bw_rhoe_highres"
diagnostics:
- short_name: [pfull, wa, va, rv, ke]
period: 1days
1 change: 0 additions & 1 deletion config/longrun_configs/longrun_hs_rhoe_dry_55km_nz63.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,4 @@ z_max: 55000.0
forcing: "held_suarez"
dt: "300secs"
rayleigh_sponge: true
job_id: "longrun_hs_rhoe_dry_55km_nz63"
toml: [toml/longrun_hs_rhoe_dry_55km_nz63.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "longrun_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,4 @@ precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
deep_atmosphere: true
job_id: "longrun_hs_rhoe_equil_55km_nz63_0M_deepatmos"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,3 @@ perturb_initstate: false
discrete_hydrostatic_balance: true
t_end: "800days"
dt_save_state_to_disk: "10days"
job_id: "longrun_sphere_hydrostatic_balance_rhoe"
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_bottom: 30.0
h_elem: 16
ode_algo: "SSP333"
precip_model: "0M"
job_id: "longrun_ssp_bw_rhoe_equil_highres"
moist: "equil"
apply_limiter: true
diagnostics:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ dz_bottom: 30.0
h_elem: 16
energy_upwinding: zalesak
precip_model: "0M"
job_id: "longrun_zalesak_tracer_energy_bw_rhoe_equil_highres"
moist: "equil"
apply_limiter: true
diagnostics:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,3 @@ dt_rad: "1hours"
prognostic_surface: "PrognosticSurfaceTemperature"
check_conservation: true
bubble: false
job_id: "aquaplanet_rhoe_equil_clearsky_tvinsol_0M_slabocean"
1 change: 0 additions & 1 deletion config/model_configs/box_density_current_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ config: "box"
vorticity_hyperdiffusion_coefficient: 0.6
scalar_hyperdiffusion_coefficient: 0.6
z_max: 6400.0
job_id: "box_density_current_test"
diagnostics:
- short_name: thetaa
period: 10secs
1 change: 0 additions & 1 deletion config/model_configs/box_hydrostatic_balance_rhoe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ config: "box"
hyperdiff: "false"
dt: "20secs"
perturb_initstate: false
job_id: "box_hydrostatic_balance_rhoe"
diagnostics:
- short_name: [wa, ua]
period: 1days
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,4 @@ precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
call_cloud_diagnostics_per_stage: true
job_id: "central_cloud_diag_gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,4 @@ moist: "equil"
precip_model: "0M"
rayleigh_sponge: true
forcing: "held_suarez"
job_id: "central_gpu_hs_rhoe_equil_55km_nz63_0M"
toml: [toml/longrun_hs_rhoe_equil_55km_nz63_0M.toml]
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ dt_save_state_to_disk: "2days"
regression_test: true
initial_condition: "MoistBaroclinicWave"
dt: "450secs"
job_id: "deep_sphere_baroclinic_wave_rhoe_equilmoist"
deep_atmosphere: true
moist: "equil"
diagnostics:
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_aquaplanet.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_aquaplanet
surface_setup: DefaultMoninObukhov
rad: clearsky
turbconv: diagnostic_edmfx
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_bomex_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: "diagnostic_edmfx_bomex_box"
initial_condition: "Bomex"
subsidence: "Bomex"
edmf_coriolis: "Bomex"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: "diagnostic_edmfx_bomex_stretched_box"
initial_condition: "Bomex"
subsidence: "Bomex"
edmf_coriolis: "Bomex"
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_dycoms_rf01_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_dycoms_rf01_box
initial_condition: DYCOMS_RF01
subsidence: DYCOMS
edmf_coriolis: DYCOMS_RF01
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_dycoms_rf01_explicit_box
initial_condition: DYCOMS_RF01
subsidence: DYCOMS
edmf_coriolis: DYCOMS_RF01
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_gabls_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_gabls_box
initial_condition: GABLS
edmf_coriolis: GABLS
surface_setup: GABLS
Expand Down
1 change: 0 additions & 1 deletion config/model_configs/diagnostic_edmfx_rico_box.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
job_id: diagnostic_edmfx_rico_box
initial_condition: Rico
subsidence: Rico
ls_adv: Rico
Expand Down
Loading

0 comments on commit e0fe1cd

Please sign in to comment.