Error during the build: Python.h: No such file or directory

[anwe5]

Hi,

Last week, I opened a discussion here about the installation of MET on Levante. After we resolved that issue, I continued with the build as described here. Unfortunately, I encountered a new error message and don't know how to resolve it.

First, I run this line ./configure --prefix=`pwd` --enable-grib2 --enable-python --enable-modis --enable-mode_graphics --enable-lidar2nc. Then I type make install >& make_install.log &. In the make_install.log file, I get the following error message:

In file included from data_line.cc:34:
python_line.h:23:10: fatal error: Python.h: No such file or directory
 #include "Python.h"
          ^~~~~~~~~~
compilation terminated.
make[3]: *** [Makefile:712: libvx_util_a-data_line.o] Error 1
make[3]: Leaving directory '/home/b/b382378/MET/MET_releases/11.1.0/MET-11.1.1/src/basic/vx_util'
make[2]: *** [Makefile:370: install-recursive] Error 1
make[2]: Leaving directory '/home/b/b382378/MET/MET_releases/11.1.0/MET-11.1.1/src/basic'
make[1]: *** [Makefile:370: install-recursive] Error 1
make[1]: Leaving directory '/home/b/b382378/MET/MET_releases/11.1.0/MET-11.1.1/src'
make: *** [Makefile:428: install-recursive] Error 1
[1]+  Exit 2                  make install &> make_install.log

In the previous issue, there was an error with -lintl not being found, so we removed it from the MET_PYTHON_LD variable.

Attached, you can find the make_install.log file. Please let me know if you need any additional details or files.
make_install.log

Thank you in advance!

[JohnHalleyGotway]

Hello, I see you're having issues compiling MET, in particular, finding the Python.h. Thanks for sending along your make_install.log file. That provides some clues.

I see, in particular, that the compilation is failing for the source file named data_line.cc. I compared your compilation output to what I see from last night's nightly build for the main_v11.1 branch of MET. And comparing that single compilation line for data_line.cc, I see a pretty big difference.

In your output, I see these two compilation options listed in order: -fopenmp -DBLOCK4
Whereas in our nightly build output, there's a whole lot more that occurs between those two options:

-fopenmp -I/nrit/ral/netcdf-4.9.2/gnu-12.2.0/include \
-I/nrit/ral/hdf5-1.14.2/include \
-I/nrit/ral/nceplibs-1.4.0/g2c-1.6.2/include \
-DMET_PYTHON_BIN_EXE="\"/nrit/ral/met-python3/bin/python3.10\"" \
-I../../basic/vx_log -I../../basic/vx_util \
-I/nrit/ral/met-python3/include/python3.10 \
-I/usr/include -I/nrit/ral/hdf4-4.2.16-2/include/hdf \
-I/nrit/ral/hdf-eos2-3.0/include -I/usr/include/cairo \
-I/usr/include/freetype2 \
-DBLOCK4

And note that all those extra -I options tell the compiler where to find the header files it needs. In particular, -I/nrit/ral/met-python3/include/python3.10 tells it where to find Python.h. Not having a similar setting, your compiler isn't able to locate the header files it needs.

So this is very likely just an issue in how you have MET configured. To debug that further, please send me the config.log file you'll find in the top-level MET directory. Whenever you run the configure command there, it generates this config.log file to provide information about the environment. For comparison, here's
config.log from our main_v11.1 branch nightly build. Please feel free to compare to your own output to see any obvious issues.

In particular, compiling MET requires setting many environment variables to tell it where to find it's dependent libraries. Run ./configure --help to see information about them. And if it helps, here's a link to the file that defines all of these for our development environment:
https://github.com/dtcenter/MET/blob/main_v11.1/internal/scripts/environment/development.seneca
You would not need to set MET_DEVELOPMENT, but note that the MET_PYTHON_{etc} settings relate to the python embedding support.

Hope that helps.

[anwe5]

Hi, thanks for your response!

Here's the config.log file:
config.log

Comparing both config.log files, I discovered that I am using a different gcc version (gcc version 8.5.0 vs. gcc version 12.2.0) . I also noticed other differences, but I didn't unterstand their implications. For example, the "running config.status" part appears at very different locations in the files and some of the cache variables are missing values.

Also, here is my installation file so you can see how I set the environment variables.
install_met_env_levante.txt

[jprestop]

Hi @anwe5. Looking at your config.log file, I see that config.log is not picking up the values that are specified in your install_met_env_levante.txt file.

MET_PYTHON_BIN_EXE=''
MET_PYTHON_CC=''
MET_PYTHON_LD=''
MET_NETCDFINC=''
MET_NETCDFLIB=''
MET_HDF5=''
MET_BUFRLIB=''
MET_GSL=''

However, I'm not sure why that would be. I have not seen this behavior before. Please run ./compile_MET_all.sh install_met_env_levante.txt and send the screen output, so that I can take a look and see if there is anything obvious going on.

[anwe5]

Hi @jprestop,

here is the screen output.
screen_output.txt

[jprestop]

Hi @anwe5. Thank you for passing along the screen output. Everything looked fine to me. However, I did note something odd. The configure command in your screen output was:

./configure --prefix=/home/b/b382378/MET/MET_releases/11.1.0/ --bindir=/home/b/b382378/MET/MET_releases/11.1.0/bin BUFRLIB_NAME=-lbufr_4 GRIB2CLIB_NAME=-lgrib2c --enable-grib2 --enable-python > configure.log 2>&1

which is all exactly what we would expect to see based on your install_met_env_levante.txt file. But your config.log file shows the configure command as being:

./configure --prefix=/home/b/b382378/MET/MET_releases/11.1.0/MET-11.1.1 --enable-grib2 --enable-python --enable-modis --enable-mode_graphics --enable-lidar2nc

which is missing --bindir, BUFRLIB_NAME, and GRIB2CLIB_NAME and has the addition of the "MET-11.1.1" directory in the path for prefix and is not what we would expect to see based on your install_met_env_levante.txt file.

Please remove the "MET-11.1.1" subdirectory and rerun ./compile_MET_all.sh install_met_env_levante.txt. Please let us know if that is successful. If it is not, please send along the config.log and met.make.log that will be in the newly created "MET-11.1.1" subdirectory and please pass along your screen_output.txt file from the new run.

[jprestop]

@anwe5 Great! Thank you for letting us know. If you don't have any further questions at this time, please go ahead and click the "Mark as answer" button for the answer that you feel best answers your question. Then, I will LOCK this conversation to prevent future posts. Our team has decided that once discussions have been answered, and the answer has been confirmed, we’ll lock them. We want to encourage users to ask new questions in new discussions rather than posting to old ones. Hopefully that’ll make the questions/answers easier for other users to follow. So if/when more issues/questions arise, please feel free to start a new discussion, as you have already.

[anwe5]

Sorry for the delay in updating you. The issue has been resolved. Thank you!