New grids

[aekiss]

Before redoing the topography (#68), we should take the opportunity to check whether the grid can be improved.

e.g. consider

1. whether the tripolar points would be better located elsewhere, e.g. at lower latitude and/or further from the Gulf of Ob
2. whether a different location for the longitudinal seam would be more convenient for our purposes (is it inconvenient to have the seam in the Indian Ocean?)
3. whether the C-grid zonal velocity points should be on the equator as done in CESM3 providing symmetry for upwelling and waves (@gustavo-marques tells me they did this in CESM3 hoping for equatorial wave improvements but it didn't make much difference)
4. extending closer to the south pole to include ice shelf cavities as was done in the 1/20° panAntarctic config 1/20° topography mom6-panan#12 (comment)
5. refining grid in the Antarctic / coarsening in the Arctic Refine grid in Antarctic? #67
6. quantizing double-precision values so they are exactly representable in single precision (as we do in the vertical grid)

Does anything else spring to mind?

[access-hive-bot]

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/cosima-working-group-announce/238/73

[adfraser]

Hi Andrew,
2) Longitude seam: Other than at the peninsula, around 140 E has the narrowest meridional sea-ice extent - is that what you were thinking? But is it a concern to have the seam in "our turf" - East Antarctica? Would it be crazy to have the seam at the peninsula? Isn't the seam longitude somewhat set by the NH tripole locations? (take all with a grain of salt - I don't know much about models!)
4) definitely has my support!

[adele-morrison]

Good points to think about! Might be nice to bring this discussion up at a COSIMA weekly meeting?

[PaulSpence]

IInteresting questions. It is a rare opportunity to change the grid. Hard to know where and how much to invest in grid changes. Probably less modifications is best, considering the effort required to determine results. Item 4 above seems like a no-brainer modification though.

[adfraser]

Raising a new point here: @pwongpan and I realised that large tabular grounded iceberg D15 is not in the 0.25* grid (and presumably the .1 and 1.0 grids either).
Here are snapshots from 1997 (top) and 2024 (bottom) showing D15 (labelled as "grounded iceberg") against the 0.25 degree grid.

Even without cavities being implemented, I'd argue that D15 would be better treated as land pixels than open ocean pixels for the following reasons:

• It's easily resolved in the 0.25 and 0.1 degree grids. It's even 2 pixels in the 1 degree grid, I think.
• I am pretty sure it's grounded to the north, and likely occupies the majority of the water column for its entire length. So it's a major impediment to the passage of currents.
• It's been there for donkeys years (it's basically calved from the West Ice Shelf but remained in place).
• It's starting to break up but the main part is very much not looking likely to move any time soon.
• It's far more stable and long-lived (in the same place) than icebergs A23A and B9B. It's basically the most stable glaciological feature which probably should be included in the land mask, but isn't.

Can we consider prescribing this as land pixels for the OM3 grid? Obviously when cavities are implemented, it would make more sense to describe this as a cavity, but until then I think land is better than water. Just a discussion point - there may be ocean things I haven't thought of :)

[adele-morrison]

Out of interest, are there any grounded icebergs included in the current bathymetry? I think we just use bathymetry from GEBCO that doesn't have any icebergs in it?

[adfraser]

I suspect none. @dpath2o might know more

[ofa001]

Hi @adfraser @adele-morrison @dpath2o I would think it odd from a coupled model perspective to have the icebergs as land points in any new grid. For Dan's project it might be useful but he can put them in for that, but these grids will need to be used more widely. Upto now we havent been using icebergs in the grid, and we just had to redo the Antarctic because minor ice shelves weren't handled well in earlier bathymetry version. We need to improve things for the next iterations in access-om3/cm3

[adfraser]

Thanks @ofa001 - yeah I realise they will need to be used widely but I think including D15 could be a special case for the "base" grid. Just a thought.

[anton-seaice]

1. whether the tripolar points would be better located elsewhere, e.g. at lower latitude and/or further from the Gulf of Ob

The difference in area for some of these cells (as described in 126) seem fairly drastic, so this looks worth investigating further.

From Bi et al 2013:

In panel a), it looks like we would move the poles south-west to maximise the distance from the ocean. I suspect they will need to be 180° in longtitude apart for us not to break something.

2. whether a different location for the longitudinal seam would be more convenient for our purposes (is it inconvenient to have the seam in the Indian Ocean?)

In theory, moving this to the longitude to with the smallest length of ocean would reduce the amount of communication between MPI tasks but the effect would very marginal.

It would be nice to have Australia in the middle of our plots :)

3. whether the C-grid zonal velocity points should be on the equator as done in CESM3 providing symmetry for upwelling and waves (@gustavo-marques tells me they did this in CESM3 hoping for equatorial wave improvements but it didn't make much difference)

Its nice to have the equator on the edge of grid cells, just because its easier to understand I think (and maybe easier to analyse?)

4. extending closer to the south pole to include ice shelf cavities as was done in the 1/20° panAntarctic config 1/20° topography mom6-panan#12 (comment)

Sounds like we have consensus on doing this, but leaving the ice shelves landmasked for now.

5. refining grid in the Antarctic / coarsening in the Arctic Refine grid in Antarctic? #67

Coarsening in the Arctic could help with 1. ? But will refining the grid in the Antarctic force us into a shorter time-step also? The cells at the south (panel b above) are already somewhat out of square, which does increase CORRECTION: we don't want the cells at the Southern end to get too far out of square, becuase it increases the amount of communication needed between MPI tasks (to communicate quantities between processing blocks). We might be able to refine the shape of the processing blocks to avoid the extra communication though :)

Does anything else spring to mind?

I guess we will keep these refinements for 1 degree (Bi et al 2013):

In the meridional direction the grid spacing is nominally 1° resolution, with the
following three refinements: a) tripolar Arctic north of 65°N;
b) equatorial refinement to 1/3° between 10°S and 10°N; and
c) a Mercator (cosine-dependent) implementation for the
southern hemisphere, ranging from 0.25° at 78°S to 1° at 30°S.

We should make sure the new grid files are cf-complaint. CMS have a compliance checker which would help: http://climate-cms.wikis.unsw.edu.au/CF_checker

[ofa001]

Hi @anton-seaice In realtion to the Antarctic, the choice of grid there in the BI et al "auscom" grid used in access-om as well was to be "square" so that n/s resolution was close to the e-w resolution as the we went further south, it might look a bit skewed on the plot. I would support additional resolution as its our region of interest, but we still need to keep some resolution in the Arctic (for both ice and ocean processes). But its less of a priority.

[anton-seaice]

Thanks :) Ill correct my post .

[anton-seaice]

2. whether a different location for the longitudinal seam would be more convenient for our purposes (is it inconvenient to have the seam in the Indian Ocean?)

On further thought on this point, by definition the location of the longitudinal seam is the longitude of one of the tripoles. From the Murray 1995 paper:

The curves are constructed on a NH stereographic projection about foci, F1 and F2 , lying on either side of the N Pole (N), which is the origin of the projection (Fig. 9). The foci are situated at latitude phiF (...) and 90° east and west of the symmetry meridian, which is at longitude l0. The focal arc and the symmetry meridian define the x and y axes, respectively.

The key point being the symmetry meridian (90° offset in longitude from the tripoles), which is continuous across the orthogonal and stereographic sections of the grid:

The definition precludes us from defining the location of the longitudinal seam differently than the tripole, although there may not be a mathematical reason why the longitudinal seam and the tripoles need to be aligned in longitude.

[anton-seaice]

@adfraser Also just pointed out that moving the tripoles south will force the resolution in the Arctic to be lower, which is probably desirable for us.

[anton-seaice]

Apologies for the talking to myself: @pwongpan asks if we should consider IBSCO bathymetry in the Southern Ocean. It looks like the GEBCO_2023 has assimilated latest fairly up-to-date IBSCO already. From https://www.gebco.net/data_and_products/gridded_bathymetry_data/gebco_2023/:

The SRTM15+ base grid has been augmented with the gridded bathymetric data sets developed by the four Seabed 2030 Regional Centers to produce the GEBCO_2023 Grid. The Regional Centers have compiled gridded bathymetric data sets, largely based on multibeam data, for their areas of responsibility. These regional grids were then provided to the Global Center. ... For the polar regions, complete grids were provided due to the complexities of incorporating data held in polar coordinates.

The compilation of the GEBCO_2023 Grid from these regional data grids was carried out at the Global Centre, with the aim of producing a seamless global terrain model.

[adfraser]

As long as the new GEBCO has IBCSO2, not IBCSO1? IBCSO2 was a major update

[anton-seaice]

As long as the new GEBCO has IBCSO2, not IBCSO1? IBCSO2 was a major update

Yeah it does: see data_contributors and search Southern Ocean

[adfraser]

That's great then. I consider IBCSO2 the best available for the SO

[pwongpan]

Thanks all. I don't know how complicated it is, but it would be great to have a platform/cookbook where we could regenerate the bathymetry based on the new bathymetric observations (by icebreakers, Expendable Bathythermograph etc.) from the grid of interest.

[adele-morrison]

@pwongpan these tools exist and have been used for past models (ACCESS-OM2, panan): https://github.com/COSIMA/domain-tools

[anton-seaice]

4. extending closer to the south pole to include ice shelf cavities as was done in the 1/20° panAntarctic config

It is suggested in the ocean grid generator guide when doing this that the South Pole should be offset to maximise the distance from the Ice Shelves (which will lead to less tiny grid cells which could be ocean). The Ross Ice Shelf looks to extent past 85°S, so its probably worth considering.

Screenshort from nilas.org :)

[adele-morrison]

Hmm, yes, good point @anton-seaice. That would make analysis a pain doing zonal averaging though!!!

[anton-seaice]

Hmm, yes, good point @anton-seaice. That would make analysis a pain doing zonal averaging though!!!

It could/should be possible to start the 'displaced pole' section at say 78°S, so for most configurations it would all be within the landmask.

[ofa001]

Hudson Bay or Foxe Basin area its in north might have been adjusted as it is close to the tri-pole, but yes we need Red Sea, and Gulf, Black Sea is a tricky one depending on resolution as the link to Med is through a very narrow strait.

[anton-seaice]

Looks like the narrowness of the channel meant the gridcell-average of GEBCO was reduced to 20m by including regions of land.

This is slightly worrying non? Like the topo generation should be robust to this and only look at ocean cells ?

I had a quick look in gen_topo, but it would take some more time to figure out if it has awareness of land masks.

[aekiss]

@ezhilsabareesh8 I've fixed editTopo.py so we can use that for hand-edits.

[ezhilsabareesh8]

@ezhilsabareesh8 I've fixed editTopo.py so we can use that for hand-edits.

Thanks @aekiss, I will try to do the hand edits at crash locations.

More importantly, it looks like the new topography is missing these seas which are present in OM2:

• Mediterranean Sea
• Red Sea
• Baltic Sea
• Black Sea

I guess deseas is removing them?

Yes deseas is removing them. Here is the topography generated without deseas, will have a look at land mask difference.

[aekiss]

The current OM2 0.25 deg topo /g/data/vk83/experiments/inputs/access-om2/ocean/grids/bathymetry/global.025deg/2023.05.15/topog.nc was created by make_topog.sh from https://github.com/COSIMA/make_025deg_topo/blob/13c91173e37b5e02c772d22bfa352151207519e2/make_topog.sh

This uses editTopo to apply thousands of edits from topog_edits.txt before running deseas, and then applies some edits to an existing ocean_mask.nc, and then applies this to the topography. This particular sequence may be needed to retain the marginal seas.

I expect we'll want to retain many of the edits in topog_edits.txt, so that would be a good starting point.

[adele-morrison]

@aekiss are you saying we should dive straight in and deepen Denmark Strait etc now? You don’t think it’s worth having a look first to see if there’s an issue with the transport to see if it’s needed? I guess I’m optimistic some things might be improved with different model physics, and the updated GEBCO product? Perhaps at least we could check if the same edits in topog_edits.txt have also been made in the OM5 topography before diving in?

[aekiss]

It's a bit complicated. The current OM2 0.25 deg topo /g/data/vk83/experiments/inputs/access-om2/ocean/grids/bathymetry/global.025deg/2023.05.15/topog.nc was created by this script which

1. generates topography from GEBCO
2. applies 1847 cell edits from topog_edits.txt (NB: reference background topography shown in this image is the final topog.nc, not the initial one generated from GEBCO):
3. removes seas, does partial cells, sets min depth
4. Then this existing land mask /g/data/ik11/inputs/access-om2/input_20200530/mom_025deg/ocean_mask.nc (which fills in the ocean near the tripoles) is corrected south of 60S
5. The land mask is then edited using ocean_mask_edits.txt at these 20 points (NB: reference background mask shown in this image doesn't include Antarctic corrections)
6. Then, crucially, the ocean mask is applied to the topography. This fills in wet points (e.g. over large regions around the tripoles) and also creates wet points, e.g. channels (set to the minimum depth if I recall correctly). Non-advective cells are then fixed.

So the topography and land mask are modified (and modify each other) at multiple steps, with many of the largest topography changes actually being imposed via the land mask. We'll need to think about which of these changes we want to inherit into OM3. Many of the edits at straits etc are needed to correctly reproduce the depths of unresolved sills and I expect would be a better starting point than 0.25° averages of the new GEBCO. Filling in the ocean for small cells near the tripoles is also likely to be a good idea to get CFL stability with a decent timestep.

[access-hive-bot]

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/cosima-twg-meeting-minutes-2024/1734/19

[aekiss]

Looks like the narrowness of the channel meant the gridcell-average of GEBCO was reduced to 20m by including regions of land.

This is slightly worrying non? Like the topo generation should be robust to this and only look at ocean cells ?

I had a quick look in gen_topo, but it would take some more time to figure out if it has awareness of land masks.

@anton-seaice see COSIMA/domain-tools#38

[aekiss]

Looks like we'll need to modify deseas to avoid filling in marginal seas connected to the ocean by 1-cell-wide channels COSIMA/domain-tools#37

[ezhilsabareesh8]

@aekiss I have updated the domain-tools code to include a function that cuts off small T-cells of size less than a specified threshold, similar to the approach used in OM2. It removes the Foxe basin, North of Hudsons bay, where the simulation was crashing before. This was implemented based on the criteria outlined in the OM2 report here.

However, I am unclear about why the minimum T-cell size is set to 6 km for regions north of 60°N and 11.7 km for regions south of 60°S.

In addition, I have regridded the OM2 topography onto the new grid and plotted the mask differences. It appears that additional land has been added near the tripole region, particularly close to Kara Strait, when the T-cell size is limited. I also noticed that the Caspian Sea is not masked, but I believe this will be corrected once deaseas processing is applied.

Topography after cutting off T cells of size less than 6 km:

Land mask difference compared to OM2: (Blue: Ocean added, Yellow: Land added)

Kara Strait (after cutting off T cells):

Kara Strait (OM2):

[anton-seaice]

However, I am unclear about why the minimum T-cell size is set to 6 km for regions north of 60°N and 11.7 km for regions south of 60°S.

I'm not sure this is a criteria - I think its just due to the distance the cells are from a pole. e.g. Figure 3. in the OM2 technical report:

Also - this screenshot also shows that open ocean at the edge of the Kara strait (around 75E, 75N) has been removed in the OM2-025 grid. Maybe its ok to have added land where we have then ?

[ezhilsabareesh8]

I'm not sure this is a criteria - I think its just due to the distance the cells are from a pole. e.g. Figure 3. in the OM2 technical report:

I think the smaller T-cell sizes, less than 6 km, are likely confined to the North Pole due to the tripole grid. At the South Pole, the minimum cell thickness is around 11 km, the T cell distribution from OM2 report shows the difference in T-cell sizes between the two poles.

[aekiss]

The SH cutoff is due to Mercator scaling of dyt only being applied between 65S and 65N, so dyt is constant south of 65S. So the 11.7 km SH limit is due to the grid itself, not the topography.

[aekiss]

I think it would be good to have Kara Strait open, as in OM2 - maybe use a cutoff slightly smaller than 6km?

[ofa001]

I agree with @aekiss that keeping Kara Strait open may be a good idea it looked like a lot of additional land had been added to close it off, and we are now on a C-gris so it doesn't need to be as many points wide to have a realistic flow through.

[ezhilsabareesh8]

Thanks, @aekiss and @ofa001 I’ve refined the topography generation workflow and implemented a new sea-removal algorithm compatible with C-grid. I also filtered out T-cells smaller than 6 km and generated topography for two fill fractions: 0.1 and 0.5. Below is the updated workflow:

# Interpolate topography onto the horizontal grid:
/g/data/tm70/ek4684/domain-tools/topogtools gen_topo -i GEBCO_2024.nc -o topog_new.nc --hgrid ocean_hgrid.nc --tripolar --longitude-offset -100 

# Remove T-cells smaller than 6 km:
/g/data/tm70/ek4684/domain-tools/topogtools cut_off_T_cells -i topog_new.nc -o topog_new_cutoff.nc --cutoff 6 --hgrid ocean_hgrid.nc

# Apply fill for sea area fractions below 0.5:
/g/data/tm70/ek4684/domain-tools/topogtools fill_fraction -i topog_new_cutoff.nc -o topog_new_fillfraction.nc  --fraction 0.5 --grid_type C

# Remove seas:
/g/data/tm70/ek4684/domain-tools/topogtools deseas -i topog_new_fillfraction.nc -o topog_new_fillfraction_0.5_deseas_grid.nc --grid_type C

# Set minimum/maximum depth:
/g/data/tm70/ek4684/domain-tools/topogtools min_max_depth -i topog_new_fillfraction_0.5_deseas_grid.nc -o topog_new_fillfraction_deseas_mindepth.nc --level 4 --vgrid ocean_vgrid.nc --vgrid_type mom6

# Fix and check for non-advective cells:
/g/data/tm70/ek4684/domain-tools/topogtools fix_nonadvective -i topog_new_fillfraction_deseas_mindepth.nc -o topog_new_fillfraction_deseas_mindepth_fixnonadvective.nc --potholes --grid_type C --vgrid_type mom6
/g/data/tm70/ek4684/domain-tools/topogtools check_nonadvective -i topog_new_fillfraction_deseas_mindepth_fixnonadvective.nc --potholes --grid_type C --vgrid_type mom6

# Create land/sea mask:
cp topog_new_fillfraction_deseas_mindepth_fixnonadvective.nc topog.nc
/g/data/tm70/ek4684/domain-tools/topogtools mask -i topog.nc -o ocean_mask.nc
ncrename -O -v mask,kmt ocean_mask.nc kmt.nc

The additional land that was previously added near the Kara Strait has been corrected in the new workflow. However, changing the fill fraction leads to variations: with a fill fraction of 0.5, the Black Sea is removed; with a fill fraction of 0.1, the Black Sea remains, though additional small cells appear northwest of it compared to the OM2 topography.

Land mask difference b/w OM2 and new topo with 0.5 fill fraction (Black contour: Land added, White Contour: Sea added): As we can see Black sea omitted in the new topography.

Land mask difference b/w OM2 and new topo with 0.5 fill fraction (Black contour: Land added, White Contour: Sea added): Black sea included.

Comparison of OM2, 0.5 fill fraction and 0.1 fill fraction topo at Black sea:

Black sea is omitted when the fill fraction is 0.5 and Bosporus strait is open when fill fraction is 0.1 and Black sea is included.

Kara Strait: In all three cases, the Kara Strait remains open. However, when the fill fraction is set to 0.1, more tiny cells are added near the coastline of the Kara Sea.

Malacca and Sunda Strait: Open in all the three cases

Gibraltar Strait: Open in all the three cases. However, it is deeper in the OM2 topography compared to the new topography.

Hudsons Bay: In the new topography, the problematic Foxe Basin north of Hudson Bay—where crashes occurred—has been removed. However, the coastlines of Hudson Bay are shallower compared to those in the OM2 topography.

Let me know if there are specific basins or straits to focus on.

[aekiss]

this should be removed for C grids (if you use the latest version in the PR it should abort with error)

# Fix and check for non-advective cells:
/g/data/tm70/ek4684/domain-tools/topogtools fix_nonadvective -i topog_new_fillfraction_deseas_mindepth.nc -o topog_new_fillfraction_deseas_mindepth_fixnonadvective.nc --potholes --grid_type C --vgrid_type mom6
/g/data/tm70/ek4684/domain-tools/topogtools check_nonadvective -i topog_new_fillfraction_deseas_mindepth_fixnonadvective.nc --potholes --grid_type C --vgrid_type mom6

[ofa001]

Thanks @ezhilsabareesh8 for these plots, I like the 0.1 fill better in many of the straits though sometimes small islands disappear, it also created small coastal indentations, fjord shaped, around some of the Arctic coastlines you showed, which could easily trap ice whilst 0.5 was smoother and didn't show them, so it might be a case of looking closely at all the high latitude coastlines in both of these cases.

[aekiss]

Thanks for the careful documentation. I think it might be safest to use fill fraction of 0.5, as we've done previously, to avoid fiddly little embayments. In this case Black Sea requires a hand-edit to open Bosphorus.

Gibraltar Sill probably needs hand-editing to deepen it.

Glad that those various straits are now open. Shallower Hudson Bay if probably not a problem but we'll find out.

[ofa001]

Hi @aekiss @ezhilsabareesh8 after being at another meeting was reminded of 2 other straits, Red Sea and Persian Gulf that we often have issues with particularly in coupled models, Can you enhance how they look in the latest topographic C-grid version.

[ezhilsabareesh8]

Thanks @aekiss and @ofa001 , I have updated the workflow to omit fix_nonadvective and check_nonadvective steps. There is no difference in the outputs. In the 0.5 fill fraction case the black sea is completely removed, need to do the hand edits to include black sea and Bosphorus.

@ofa001 The Red sea and Persian Gulf seems to be fine in the new topo.

[ofa001]

Hi @ezhilsabareesh8, I think we still had to artificially widen the Red Sea for ACCESS-CM2-025 version, it may be in the notes, @aekiss will probably know where, its still quite narrow here. For the Persian Gulf is not the width so much but sill depth that we may have changed to increase mixing with the Indian Ocean otherwise they get very saline in coupled runs even at 0.25 resolution, its not clear if extra grid points with C-grid will help!

[ezhilsabareesh8]

@aekiss I made some hand edits in the topography to open the Bosphorus Strait, which preserves the Black Sea from being omitted by the deseas algorithm. I made two adjustments and set the land points to match the nearest depth values.

#    i    j  old  new
  1235  737  0.0  65.28469848632812
  1227  734  0.0  56.118499755859375

The updated topography now preserves the Black Sea; however, unlike in OM2, the Sea of Azov remains connected to the Black sea in the new topography. Would it be more appropriate to close the Strait of Kerch to omit the Sea of Azov?

[aekiss]

Thanks @ezhilsabareesh8, keeping the Sea of Azov is probably an improvement

[ofa001]

Hi @ezhilsabareesh8 and @aekiss, I also like having the Sea of Azov in there, I would only remove it if its shallowness introduced a problem in the runs.

[aekiss]

As @ezhilsabareesh8 pointed out, we'll need to fix the runoff #231 before we adjust the topography to minimise SSS restoring in marginal seas.

[ezhilsabareesh8]

Thanks, @aekiss, for documenting our discussion. Here’s a quick update on the new topography: the model with the updated topography with the recent edits and updated workflow has now been running successfully for 15 years without any crashes or truncation errors. However, the salinity restoring results remain very similar to those from the OM3 runs with the previous grid and topography. It might be best to address the runoff issues before making further edits to the topography.

SSS restoring (Solid lines: New Topo; Dashed lines: Old Topo)