-
Notifications
You must be signed in to change notification settings - Fork 290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bugfix for Sentinel-2 radiance calculation #2896
Open
simonrp84
wants to merge
8
commits into
pytroll:main
Choose a base branch
from
simonrp84:Sen2_Radiance
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
d0f379b
Remove rogue unit assignment in MSI SAFE reader
simonrp84 39ed676
Add check for S2/MSI processing level to catch method needed for radi…
simonrp84 acf2088
Debugging MSI SAFE radiances
simonrp84 976b642
Bugfix for Sentinel-2 L1C radiance calculation, initial tests.
simonrp84 c1348f6
Update S2/MSI reader to use only the gridded SZA for radiance calcula…
simonrp84 4309738
Prepare S2/MSI reader for availability of L1B files, but raise error …
simonrp84 485ef66
Update S2/MSI tests for radiance calculations.
simonrp84 38a05d1
Add docs note to S2/MSI specifying that L1B data is not currently sup…
simonrp84 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -15,7 +15,7 @@ | |
# | ||
# You should have received a copy of the GNU General Public License along with | ||
# satpy. If not, see <http://www.gnu.org/licenses/>. | ||
"""SAFE MSI L1C reader. | ||
"""SAFE MSI L1C/L2A reader. | ||
|
||
The MSI data has a special value for saturated pixels. By default, these | ||
pixels are set to np.inf, but for some applications it might be desirable | ||
|
@@ -32,6 +32,10 @@ | |
|
||
https://sentinels.copernicus.eu/documents/247904/685211/S2-PDGS-TAS-DI-PSD-V14.9.pdf/3d3b6c9c-4334-dcc4-3aa7-f7c0deffbaf7?t=1643013091529 | ||
|
||
NOTE: At present, L1B data is not supported. If the user needs radiance data instead of counts or reflectances, these | ||
are retrieved by first calculating the reflectance and then working back to the radiance. L1B radiance data support | ||
will be added once the data is published onto the Copernicus data ecosystem. | ||
|
||
""" | ||
|
||
import logging | ||
|
@@ -59,13 +63,16 @@ | |
class SAFEMSIL1C(BaseFileHandler): | ||
"""File handler for SAFE MSI files (jp2).""" | ||
|
||
def __init__(self, filename, filename_info, filetype_info, mda, tile_mda, mask_saturated=True): | ||
def __init__(self, filename, filename_info, filetype_info, mda, tile_mda, | ||
mask_saturated=True): | ||
"""Initialize the reader.""" | ||
super(SAFEMSIL1C, self).__init__(filename, filename_info, | ||
filetype_info) | ||
del mask_saturated | ||
self._channel = filename_info["band_name"] | ||
self.process_level = filename_info["process_level"] | ||
if self.process_level not in ["L1C", "L2A"]: | ||
raise ValueError(f"Unsupported process level: {self.process_level}") | ||
self._tile_mda = tile_mda | ||
self._mda = mda | ||
self.platform_name = PLATFORMS[filename_info["fmission_id"]] | ||
|
@@ -83,7 +90,6 @@ | |
if proj is None: | ||
return | ||
proj.attrs = info.copy() | ||
proj.attrs["units"] = "%" | ||
proj.attrs["platform_name"] = self.platform_name | ||
return proj | ||
|
||
|
@@ -93,7 +99,21 @@ | |
if key["calibration"] == "reflectance": | ||
return self._mda.calibrate_to_reflectances(proj, self._channel) | ||
if key["calibration"] == "radiance": | ||
return self._mda.calibrate_to_radiances(proj, self._channel) | ||
# The calibration procedure differs for L1B and L1C/L2A data! | ||
if self.process_level in ["L1C", "L2A"]: | ||
# For higher level data, radiances must be computed from the reflectance. | ||
# By default, we use the mean solar angles so that the user does not need to resample, | ||
# but the user can also choose to use the solar angles from the tile metadata. | ||
# This is on a coarse grid so for most bands must be resampled before use. | ||
dq = dict(name="solar_zenith_angle", resolution=key["resolution"]) | ||
zen = self._tile_mda.get_dataset(dq, dict(xml_tag="Sun_Angles_Grid/Zenith")) | ||
tmp_refl = self._mda.calibrate_to_reflectances(proj, self._channel) | ||
return self._mda.calibrate_to_radiances(tmp_refl, zen, self._channel) | ||
#else: | ||
# For L1B the radiances can be directly computed from the digital counts. | ||
#return self._mda.calibrate_to_radiances_l1b(proj, self._channel) | ||
|
||
|
||
if key["calibration"] == "counts": | ||
return self._mda._sanitize_data(proj) | ||
if key["calibration"] in ["aerosol_thickness", "water_vapor"]: | ||
|
@@ -149,15 +169,15 @@ | |
|
||
def calibrate_to_reflectances(self, data, band_name): | ||
"""Calibrate *data* using the radiometric information for the metadata.""" | ||
quantification = int(self.root.find(".//QUANTIFICATION_VALUE").text) if self.process_level == "L1C" else \ | ||
quantification = int(self.root.find(".//QUANTIFICATION_VALUE").text) if self.process_level[:2] == "L1" else \ | ||
int(self.root.find(".//BOA_QUANTIFICATION_VALUE").text) | ||
data = self._sanitize_data(data) | ||
return (data + self.band_offset(band_name)) / quantification * 100 | ||
|
||
def calibrate_to_atmospheric(self, data, band_name): | ||
"""Calibrate L2A AOT/WVP product.""" | ||
atmospheric_bands = ["AOT", "WVP"] | ||
if self.process_level == "L1C": | ||
if self.process_level == "L1C" or self.process_level == "L1B": | ||
return | ||
elif self.process_level == "L2A" and band_name not in atmospheric_bands: | ||
return | ||
|
@@ -194,14 +214,37 @@ | |
@cached_property | ||
def band_offsets(self): | ||
"""Get the band offsets from the metadata.""" | ||
offsets = self.root.find(".//Radiometric_Offset_List") if self.process_level == "L1C" else \ | ||
offsets = self.root.find(".//Radiometric_Offset_List") if self.process_level[:2] == "L1" else \ | ||
self.root.find(".//BOA_ADD_OFFSET_VALUES_LIST") | ||
if offsets is not None: | ||
band_offsets = {int(off.attrib["band_id"]): float(off.text) for off in offsets} | ||
else: | ||
band_offsets = {} | ||
return band_offsets | ||
|
||
def solar_irradiance(self, band_name): | ||
"""Get the solar irradiance for a given *band_name*.""" | ||
band_index = self._band_index(band_name) | ||
return self.solar_irradiances[band_index] | ||
|
||
@cached_property | ||
def solar_irradiances(self): | ||
"""Get the TOA solar irradiance values from the metadata.""" | ||
irrads = self.root.find(".//Solar_Irradiance_List") | ||
if irrads is not None: | ||
solar_irrad = {int(irr.attrib["bandId"]): float(irr.text) for irr in irrads} | ||
else: | ||
solar_irrad = {} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can this be tested? |
||
return solar_irrad | ||
|
||
@cached_property | ||
def sun_earth_dist(self): | ||
"""Get the sun-earth distance from the metadata.""" | ||
sed = self.root.find(".//U") | ||
if sed is not None: | ||
return float(sed.text) | ||
return -1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not really found of this. Would it be possible to raise an error here and catch it downstream? |
||
|
||
@cached_property | ||
def special_values(self): | ||
"""Get the special values from the metadata.""" | ||
|
@@ -219,12 +262,23 @@ | |
"""Get the saturated value from the metadata.""" | ||
return self.special_values["SATURATED"] | ||
|
||
def calibrate_to_radiances(self, data, band_name): | ||
def calibrate_to_radiances_l1b(self, data, band_name): | ||
"""Calibrate *data* to radiance using the radiometric information for the metadata.""" | ||
physical_gain = self.physical_gain(band_name) | ||
data = self._sanitize_data(data) | ||
return (data + self.band_offset(band_name)) / physical_gain | ||
|
||
def calibrate_to_radiances(self, data, solar_zenith, band_name): | ||
"""Calibrate *data* to radiance using the radiometric information for the metadata.""" | ||
sed = self.sun_earth_dist | ||
if sed < 0.5 or sed > 1.5: | ||
raise ValueError(f"Sun-Earth distance is incorrect in the metadata: {sed}") | ||
solar_irrad_band = self.solar_irradiance(band_name) | ||
|
||
solar_zenith = np.deg2rad(solar_zenith) | ||
|
||
return (data / 100.) * solar_irrad_band * np.cos(solar_zenith) / (np.pi * sed * sed) | ||
|
||
def physical_gain(self, band_name): | ||
"""Get the physical gain for a given *band_name*.""" | ||
band_index = self._band_index(band_name) | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to leave that there as a placeholder. Soon the L1b data will be made available to users, and this section of code will then become useful :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we uncomment it then?