Spectral Synthesis for Satellite-to-Satellite Translation

This code corresponds to the following paper: https://ieeexplore.ieee.org/document/9462910

Citation

Vandal, T. J., McDuff, D., Wang, W., Duffy, K., Michaelis, A., & Nemani, R. R. (2021). Spectral Synthesis for Geostationary Satellite-to-Satellite Translation. IEEE Transactions on Geoscience and Remote Sensing.

Model

VAE-GAN Architecture forr unsupervised image-to-image translation with shared spectral reconstruction loss. Model is trained on GOES-16/17 and Himawari-8 L1B data processed by GeoNEX.

Dependencies

Python==3.7
Pytorch==1.5
Petastorm==0.9

Note: Functionality using PyTorch with MPI requires installation from source.

conda create --name geonex_torch1.5 python=3.7 pytorch=1.5 xarray numpy scipy pandas torchvision tensorboard opencv pyyaml jupyterlab matplotlib seaborn pyspark
conda install -c conda-forge pyhdf
pip install petastorm

Steps to Reproduce Experiments

Data

Find GOES-16/17 and Himawari-8 L1G products on the GeoNEX dataportal

Build Training Datasets

Data is parsed from GeoNEXL1G as sub-images and stored in a petastorm database using spark. We set max_files to 100 for testing only.

NEX environment

conda activate geonex_torch1.5
module -a use /u/analytix/tools/modulefiles
module load jdk/jdk8u202

cd data
python write_data_to_petastorm.py /nex/datapool/geonex/public/GOES16/GEONEX-L1G/ WRITE_DIRECTORY G16 --year 2018 --max_files 100
python write_data_to_petastorm.py /nex/datapool/geonex/public/GOES17/GEONEX-L1G/ WRITE_DIRECTORY G17 --year 2018 --max_files 100

Run Training script

Train model with a given configuration file with data and model parameter. See configs/Base-G16G17.yaml and configs/Base-G16G17H8.yaml as examples.

python train_net.py --config_file configs/Base-G16G17.yaml

Training can be visualized using tensorboard

tensorboard --logdir EXPERIMENT_DIRECTORY

Perform Inference

Work in progress
Current inference examples can be found in notebooks/

Known Challenges

This model estimated the lower bound of log-likelihood effectively causing reduced spatial resolution. The latent space is only appoximately cycle consistent. Recent developed in invertible methods (eg. AlignFlow) solves this problem deterministically with maximum likelihood.

Acknowledgements

This work was funded by the NASA Ames Research Center and NASA Earth eXchange (NEX). We acknowledge the network codes inherented from https://github.com/mingyuliutw/UNIT.