Occasional KeyError: 'EARTH_RADIUS' because of missing *rsc: Run load_slc_geometry.py without --no-metadata_check option?

[falkamelung]

I am occasionally getting this error:

cat run_01_miaplpy_load_data_0__1.e
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/miniconda3/bin/load_slc_geometry.py", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/MiaplPy/src/miaplpy/load_slc_geometry.py", line 72, in main
    iDict = mut.read_subset_box(iDict)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/MiaplPy/src/miaplpy/objects/utils.py", line 1157, in read_subset_box
    pix_box = coord.bbox_geo2radar(geo_box)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/MintPy/src/mintpy/objects/coord.py", line 501, in bbox_geo2radar
    y, x, y_res, x_res = self.geo2radar(lat, lon, print_msg=print_msg)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/MintPy/src/mintpy/objects/coord.py", line 343, in geo2radar
    az_step = ut0.azimuth_ground_resolution(self.src_metadata)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work2/05861/tg851601/stampede2/code/rsmas_insar/tools/MintPy/src/mintpy/utils/utils0.py", line 232, in azimuth_ground_resolution
    Re = float(atr['EARTH_RADIUS'])
               ~~~^^^^^^^^^^^^^^^^
KeyError: 'EARTH_RADIUS'

The reason is that the *.rsc files are missing.

ls -l /merged/SLC/20160605
total 6
-rw-r--r-- 1 tg851601 G-820134 1012 Jul 30 01:02 20160605.slc.full.vrt
-rw-r--r-- 1 tg851601 G-820134 4148 Jul 30 01:02 20160605.slc.full.xml

The quick check is ls -l merged/SLC/*/*rsc.

I have not figured out why these files are missing.

Currently the run_01_miaplpy_load_data contains

load_slc_geometry.py --template miaplpy/miaplpyApp.cfg --project_dir . --work_dir ./miaplpy_SN_201606_201608  --no_metadata_check

When I run without --no_metadata_check option the missing *rsc files are created and it works fine. My question is whether we should remove --no_metadata_check in this line:

MiaplPy/src/miaplpy/miaplpyApp.py

Line 279 in 7de325f

run_commands = ['{a} load_slc_geometry.py {b} --no_metadata_check\n'.format(a=self.text_cmd.strip("'"), b=scp_args)]

[falkamelung]

I think I have figured out what the problem is. It checks whether inputs/slcStack.h5 exists before running prep_slc_isce.py. If it exists there is no need to run it again. So if you forget to remove your miaplpy folder but reruns ice, it will find inputs/slcStack.h5, and NOT rerun prep_slc_isce.py, Therefor it will throw an error and there are no r.rsc files

[mirzaees]

yes it might be because the slcStack was not generated properly in your previous run and when you run again, it checks for it's existence. Deleting the existing file to regenerate every time is not efficient, because this step takes long time. we should work on using VRT's instead!

[falkamelung]

As I suggest above, just removing --no_metadata_check form this line

MiaplPy/src/miaplpy/miaplpyApp.py

Line 279 in 7de325f

run_commands = ['{a} load_slc_geometry.py {b} --no_metadata_check\n'.format(a=self.text_cmd.strip("'"), b=scp_args)]

would solve the issue as far as I can see. I could issue a PR. Or do you mean doing the metadata_check would take too much time?