Build 10 / Super Resolution Level II Support (all data pre-2008)

[SteepAtticStairs]

This is not an issue that is super important compared to the issue of supporting data before 2016, because I think it is much more likely people will want to use data before 2016 than they will want to use data before 2008. However, for the sake of thoroughness, I am opening this issue.

I was doing some more research on the history of NEXRAD data, and I came upon this article referenced by a wikipedia page. I realized that this means that there was yet another change in NEXRAD data formatting, this time in 2008 - although there was no set date, that wikipedia page said the changes happed from "March to August 2008".

From what I can gather, it looks like this update was when the MSG31 format was added - and although I know almost nothing about the actual binary structures of these radar files, I do know that message 31 is rather important in their decoding.

All that to say, these pre-2008 files do not work with go-nexrad. Again, this is not as big of an issue as the pre-2016 radar file incompatibility issue was, but I am opining this issue for the sake of thoroughness. Here are the radar files that I am using to test which work with the current go-nexrad project (all of these files are links to their direct downloads):

DOES NOT WORK

• KABR20070101_000618.gz
• KABR20080604_141100.gz
• KABR20080604_201723_V04.gz
• KLIX20050828_181658.gz (I believe this is Hurricane Katrina)

WORKS

• KABR20080604_202927_V03.gz
• KBLX20090603_004417_V03.gz
• I think everything else onward from 2008

I noticed that all the files that worked had a V03 on the end of their filename, but the one file with V04 did not work. All files without any V0 do not work, and currently it is V06. Is this relevant at all?

EDIT: I opened KABR20080604_201723_V04.gz in NOAA's Weather and Climate Toolkit and it appears to be either corrupted or a file that does not contain any radar data, if this is helpful at all.

[bwiggs]

Thanks for the detailed report! Adding those sample files that don't work is very helpful.

I suspect there's going to be even more work to be done as we support files from farther and farther back.

[vcabbage]

Ran across the project and wanted to say it's very cool. I deal with this data in my day job and figured I'd drop a couple notes, I hope they're useful.

• As you've already discerned, build 10 added message 31 to support super-res scans (and dual polarization data but that was implemented later). Before build 10, scans were lower res and radial data was transmitted in message 1.
• Message 1 is relatively easy to parse, though it doesn't include as much metadata, such as station coordinates and elevation.
• There have been other changes over the years. Of the messages I work with, notable examples would be that message 5 used to be message 7 and in the really old data message 18 was reserved and not the same as message 18 today.
• For rendering images I don't think you'll need to do much more than support message 1, even for the oldest data.
• In my experience, the farther you go back in the archive the more partial/corrupt scans there are. KABR20080604_201723_V04.gz, for example, appears to be a partial scan.