pb2nc failing due to bufr archive library abort error

[malloryprow]

I am working with David Huber to get the METplus verification that runs in the GFS workflow on a new system called S4 (I think it is a University of Wisconsin machine, but I could be wrong). We ran into some issues with pb2nc. Here is what we are seeing in the log file (run with MET verbosity up to 5):

DEBUG 1: Creating NetCDF File: /scratch/users/dhuber/verif_global_standalone/tmpnwdev/verif_global.379229/grid2obs_step1/metplus_output/make_met_data_by_VALID/pb2nc/upper_air/prepbufr/prepbufr.gdas.2020080300.nc
DEBUG 1: Pre-processing BUFR File for variable names from /scratch/users/dhuber/verif_global_standalone/tmpnwdev/verif_global.379229/grid2obs_step1/data/prepbufr/prepbufr.gdas.2020080300
DEBUG 3: Blocking Bufr file (metadata) to: /scratch/users/dhuber/verif_global_standalone/tmpnwdev/verif_global.379229/grid2obs_step1/metplus_output/tmp/tmp_pb2nc_tbl_blk_95272_0

BUFR ARCHIVE LIBRARY ABORT***
BUFRLIB: DATELEN - INPUT ARGUMENT IS**** - IT MUST BE EITHER 8 OR 10
BUFR ARCHIVE LIBRARY ABORT***

I saw at https://dtcenter.org/met-online-tutorial-metv8-0/point-processing-tool/pb2nc that it mentions to try compiling MET with "--disable-block4", but this also gave the same error.

[JohnHalleyGotway]

@malloryprow I see you're having trouble running PrepBUFR files through MET's pb2nc tool. Unfortunately, I don't see any obvious problems/solutions in the log output you sent. So I don't have a quick fix to recommend.

If possible, granting someone from the METplus team access to the S4 machine to debug would be ideal, but I realize that's a longshot. Short of that, we could schedule a screen sharing session and try to debug together. And I think myself, @jprestop, and/or @hsoh-u could be good choices for that.

But stepping back, I'm wondering if the culprit is how MET is configured/compiled or the contents of that specific data file. Some questions to answer are:

• Does "make test" for MET, which does run the pb2nc tool, run without error? If not, it's definitely a configuration/compilation problem. If it does, then it might be an issue with this specific data file.

• Does the same command, using the same input data file, run fine on WCOSS... which is known to have a good installation?

If we determine that's it definitely a configuration/compilation problem, I'd want to know the details of the installation...

• MET s/w version number
• Compiler names and versions
• BUFRLIB version and what commands were used to compile it

You're right that the --disable-block4 option is a good one to try out. You'd rerun the "configure" command using "--disable-block4" followed by "make clean install test":

  --disable-block4        Disable use of BLOCK4 in the compilation. Use this
                          if you have trouble using PrepBufr files.

Too bad that didn't do the trick!

[malloryprow]

David passed the various logs on to Hera at /scratch1/NESDIS/nesdis-rdo2/David.Huber/for_mallory/install_logs. He said he is going to try to recreate the build with the same libraries on Hera.

[JohnHalleyGotway]

Thanks @malloryprow. I grabbed the logs and see in config.log that David did successfully set the --disable-block4 option which led to this line being printed in config.log:

configure:3801: -DBLOCK4 will not be used in the compiles

We'll definitely be able to debug any issues he finds on Hera. So please keep me posted.

[malloryprow]

Some progress has been made! I was able to get MET to build and get pb2nc to run successfully with using the HPC-stack libraries on Hera. The HPC-stack bufr installation seems to have multiple libraries (libbuf_4, libbuf_8, libbuf_d, libbuf_4_DA, libbuf_8_DA, and libbuf_d_DA). The initial buildings of MET were linking with libbuf_8. After testing with linking the other libraries, libbuf_4, libbuf_d, libbuf_4_DA, and libbuf_d_DA all lead to successful builds with successful runs of pb2nc. libbuf_8 and libbuf_8_DA lead to successful builds with failures in running pb2nc. I'm not sure exactly why there are these multiple libraries or what the differences are between them all.

I'm hoping that this success will translate to S4. David said he will try linking with libbuf_4 and let me know.

[malloryprow]

More good news! Linking with the -lbuf_4 lead to a successful run of pb2nc on S4 as well!

Julie also emailed me with this:

The following libraries are generated on the NCEP IBM Frost and Snow machines:
libbufr_4.a -- 4-byte reals, 4-byte integers, 64-bit executable compilation
libbufr_8.a -- 8-byte reals, 8-byte integers, 64-bit executable compilation
libbufr_d.a -- 8-byte reals, 4-byte integers, 64-bit executable compilation

I can cancel the meeting we had set up to debug things. I'm not sure with a discussion topic if I need to close it or resolve it in some manner, but we are all good now.

[JohnHalleyGotway]

@malloryprow thanks for the update. That is excellent news and is something we'll look out for in the future when there are multiple build variations of the BUFR library available.

Clicking the little circle checkmark on one of the discussion responses marks it as being a good "answer" to the original question. Either the person asking the question or the folks responding to from the METplus Support team can select an answer. And I'll do that now.