NGIAB-DataStream Testing

[JordanLaserGit]

Should an automated workflow exist that tests the community deployment of DataStream with each new release of hydrofabric and/or NGIAB with the DataStream executions?

Primary motivation for this is to ensure coupling-safe updates, or at least identify how an update in NGIAB may break the DataStream deployment. Another motivation would be saving time avoiding manually updating and testing the DataStream AMI and executions for each NGIAB update. Also, this workflow may be a stepping stone to testing community contributed NextGen configuration improvements.

Some more info:

The DataStream AWS AMI is composed of the following:

• AWS Amazon Linux 2023 ARM base image
• ngen-datastream stream.sh (ideally main branch, but is currently the https://github.com/CIROH-UA/ngen-datastream/tree/v2.2_hydrofabric branch)
• ngen-datastream research datastream configuration files ( 💻 , 🌄 )
• forcingprocessor ( 🌄 ) and datastream ( 💻 , 🌄) docker images
• NGIAB image ( 💻 )
• merkdir image
• hydrofabric geopackages ( 🌄 )
• weights files ( 🌄 )

🌄 - updated each hydrofabric release
💻 - updated each NGIAB release

A workflow already exists in this repo to test the terraform and a sample datastream execution. This could be modified to test all executions currently stored in amazon event bridge, along with a user_data.sh script to build the AMI.