November 2019 Status

[gjscheer-ucd]

DWR PROD status

• install latest spatial cimis code base
• importing raw incoming data to GRASS
• test solar parameter calculation
• setup station data app key
• test ETo calculation for GOES17
• test ETo calculation for GOES15 format
• create raster output
• g.cimis-dwr errors
• fix file permission issue
• test CGI query
• transfer GOES16 ETo from 10/23/2018 to 12/4/2019 and create GOES15 Eto products
• transfer historical web output through 12/4/2019
• as of 11/27/2019 begin automated GOES17 solar calculation
• as of 12/4/2019 begin automated GOES17 ETo calculation

DWR prod strategy

• monitoring
• Automated Sciences patching 12/4/2019 before sunrise
• GOES17 data cutover to be the go live date of 12/5/2019
• 11/14/ firewall rules may affect spatial API
• prod smoke test 11/18 to 11/22

Status for g.cimis

• manual batch solar processing error
• solar calc without PGM files Issue 90
• final ssetr-G calc Issue 89
• bad image check is implemented? Issue 66
• other updates?
• state of DWR dev & test?

Status for cimis-goes-s

• d.mon crashes- Issue 94
• upgrading grass to 7.6 or 7.8 breaks r.solpos and r.heliosat compile - Issue 100

Todo

• George will update the repo on test and dev

[qjhart]

Prod - Data Assimilation

In order for prod to manage all data access, we need to get all the historical goes-15 data over. It's also fine to get all the goes-17 test/dev data as well, these can overlap. Transfer from the goes-15 production server is well defined in #93, although the tunnel issue (#42) is no longer required, you directly contact the goes-15 production server (mrsblapp20662.pr.water.ca.gov)

Issue #93 moved data to the testing server, and performed all the required changes to the goes-15 data, so on one hand we could simply use that data. The problem of course is that we have to use the GOES-15 cutoff date for our goes-15 data. That means we need to copy the data from goes-15 production, up until 2019-12-05.

The first step for this is to get the production data from GOES-16 with some rsyncs;

rsync --exclude=.tmp --exclude=.bash_history [email protected]:/apps/cimis/gdb/cimis/2019-09-?? ~cimis/gdb16/cimis/ -a -v
rsync --exclude=.tmp --exclude=.bash_history 
 [email protected]:/apps/cimis/gdb/cimis/2019-1?-?? ~cimis/gdb16/cimis/ -a -v

# start grass ~gdb15/cimis/cimis
cim16=~/gdb16/cimis;
cim15=~/gdb15/cimis;
save='ETo G Gc K Rnl Rs Rso Tdew Tm Tn Tx U2 ea es';
for d in $cim16/2018-10-??; do 
 day=$(basename $d); 
echo $day
 g.mapset --quiet -c dbase=~/gdb15 location=cimis mapset=$day; 
 rm -f ~/gdb15/cimis/TRANSFER;
 ln -s $d ~/gdb15/cimis/TRANSFER; 
 g.region -d; 
 for m in $save; do 
  if ( !  g.findfile element=cellhd file=$m > /dev/null); then
     r.mapcalc --overwrite --quiet expr="$m=$m@TRANSFER"; 
  else 
    echo $m@$day exists
  fi
 done;
done
rm -f ~/gdb15/cimis/TRANSFER;