FATAL ERROR (SEGFAULT): when python embedding #2660

Wesley-J-Davis · 2024-08-23T15:38:52Z

Wesley-J-Davis
Aug 23, 2024

I am trying to use python embedding to run grid_stat and I keep getting this SEGFAULT message.

(envname) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity exec -e --bind $(pwd) --bind $WORKDIR /p/work1/daviswj/MET_12_0_0_BASE/met_v12.0.0-beta1.simg plot_data_plane PYTHON_NUMPY tair.ps 'name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -title "air_temperature_NAAPS2" -v 10
DEBUG 1: Start plot_data_plane by daviswj(927863) at 2024-08-23 15:13:33Z cmd: /usr/local/bin/plot_data_plane PYTHON_NUMPY tair.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -title air_temperature_NAAPS2 -v 10
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object of type "FileType_Python_Numpy".
DEBUG 3: Running user's python script (/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2).
DEBUG 3: Initializing MET compile time python instance: /usr/local/bin/python3
FATAL ERROR (SEGFAULT): Process 47876 got signal 11 @ local time = 2024-08-23 15:13:37Z
FATAL ERROR (SEGFAULT): Look for a core file in /p/work1/daviswj/MONET-MET-INTEGRATION
FATAL ERROR (SEGFAULT): Process command line: /usr/local/bin/plot_data_plane PYTHON_NUMPY tair.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -title air_temperature_NAAPS2 -v 10
Segmentation fault

As I understand it, segmentation fault refers to the container trying to access something in memory it isn't allowed to. I've run into this error several times now and not just the grid_stat utility. I had the issue with plot_point_obs as well. Though I was able to get around that one by using ascii2nc to convert my point obs to a .nc file and then plotting that.

The goal of my work is to read in an aerosol model in netcdf format. My strategy at first was to use xarray to read the netcdf and then pull out the data as a numpy array. Code is as follows (slightly truncated):

ds = xr.open_dataset(f"/p/work1/daviswj/MONET-MET-INTEGRATION/fcst/{file_name}")
met_data = ds[var].isel(time=0).isel(sigma_layer=0).to_numpy()
var_data = ds[var].isel(time=0).isel(sigma_layer=0)
attrs = {
'valid': ds.time.dt.strftime("%Y%m%d_%H%M%S").item(),
'init': ds.atime.dt.strftime("%Y%m%d_%H%M%S").item(),
'lead': int((ds.time-ds.atime).dt.total_seconds()/3600),
'accum': '000000',
'name': var_data.attrs['standard_name'],
'long_name': var_data.attrs['long_name'],
'level': 'Surface',
'units': var_data.attrs['units'],
'grid': {
'type': 'LatLon',
'name': 'NAAPS2',
'Nx': len(ds.longitude),
'Ny': len(ds.latitude),
'lat_ll': llx,
'lon_ll': lly,
}
}

After some troubleshooting I became suspicious that the python version I was using in my environment was a couple versions ahead of the one used to compile my singularity container. My next troubleshooting step was to save met_data and attrs using pickle and then read those pickle files in using python3.9 instead of the 3.10 version that I'd been using. That still gave me the segmentation fault so for the time being I'm fresh out of ideas.

I've saved my directory structure to /incoming/irap/met_help/DAVIS_DATA/DAVIS_DATA_FULL.tar

There is another file there called DAVIS_DATA but I hit control+c trying to copy the text, interrupting the transfer instead, use DAVIS_DATA_FULL to access everything. The MET version I'm running with is 12.0.0 in a singularity container, I'll pass that along too if you need.

Thank you.

Answered by DanielAdriaansen

Aug 23, 2024

OK, agreed probably a container or environment issue.

In your first error, I saw this:

DEBUG 3: Initializing MET compile time python instance: /usr/local/bin/python3

And in your last test with 11.0.1, I see this:

File "/usr/local/lib/python3.6/site-packages/xarray/core/common.py", line 229, in getattr
"{!r} object has no attribute {!r}".format(type(self).name, name)

Does that mean when using 12.0.0-beta1 you're also using Python 3.6? If so, I would encourage you to upgrade Python to 3.10.4+ and re-compile MET against it and see if that helps before we go any farther. The current documentation details requirements for Python embedding here:
https://met.readthedocs.io/en/latest/Users_Guid…

View full answer

DanielAdriaansen · 2024-08-23T15:54:33Z

DanielAdriaansen
Aug 23, 2024
Collaborator

I @Wesley-J-Davis, I downloaded the tar file but don't see this script:
/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py

Can you share that script? I will start by testing plot_data_plane first. Thanks!

13 replies

Wesley-J-Davis Aug 23, 2024
Author

that file should be under the fcst folder by the way

/fcst/2023012000_000_000_nap045r3x3_000.nc

DanielAdriaansen Aug 23, 2024
Collaborator

Thanks! I was able to get this script to work:

import xarray as xr
import numpy as np

file = "/home/dadriaan/projects/disc2660/fcst/2023012000_000_000_nap045r3x3_000.nc"
var = 'tair'
ds = xr.open_dataset(file)
#ds = xr.open_dataset(f"/p/work1/daviswj/MONET-MET-INTEGRATION/fcst/{file_name}")

llx = ds.isel(latitude=0).latitude.values
lly = ds.isel(longitude=0).longitude.values

print(ds.longitude.values)
print(ds.longitude.min().values)

met_data = ds[var].isel(time=0).isel(sigma_layer=0).to_numpy()

# Flip the data across the equator
met_data = met_data[::-1]

var_data = ds[var].isel(time=0).isel(sigma_layer=0)
attrs = {
'valid': ds.time.dt.strftime("%Y%m%d_%H%M%S").item(),
'init': ds.atime.dt.strftime("%Y%m%d_%H%M%S").item(),
'lead': str(int((ds.time-ds.atime).dt.total_seconds()/3600)),
'accum': '000000',
'name': var_data.attrs['standard_name'],
'long_name': var_data.attrs['long_name'],
'level': 'Surface',
'units': var_data.attrs['units'],
'grid': {
'type': 'LatLon',
'name': 'NAAPS2',
'Nlon': len(ds.longitude),
'Nlat': len(ds.latitude),
'lat_ll': float(llx),
'lon_ll': float(lly),
'delta_lat':0.3,
'delta_lon':0.3,
}

However, the graphic is a little off and I think it has to do with the correct lower left latitude and longitude for the grid definition. Here is my graphic:

Do you know the correct lower left latitude and longitude?

The command I used was:
plot_data_plane PYTHON_NUMPY test.ps 'name="test_naaps_script.py";' -v 5

MET version 12.0.0-beta2.

Wesley-J-Davis Aug 23, 2024
Author

hmm, I copied over your exact python reader and command and I'm still getting the segmentation fault.

I'm working with met v12.0.0beta1 via a singularity container. Feels like this is less of a MET issue and more of a container one now since you were able to get it to work no problem.

I tried the same thing on a MET v11.0.1 container and I get this error:

(python3.9) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity exec -e ../MET_11_0_1_BASE/met_v11.0.1.simg plot_data_plane PYTHON_NUMPY NAAPS2.ps 'name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -v 5
DEBUG 1: Start plot_data_plane by daviswj(927863) at 2024-08-23 19:53:47Z cmd: /usr/local/bin/plot_data_plane PYTHON_NUMPY NAAPS2.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -v 5
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object of type "FileType_Python_Numpy".
DEBUG 3: Running user's python script (/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2).
Traceback (most recent call last):
File "/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py", line 12, in
met_data = ds[var].isel(time=0).isel(sigma_layer=0).to_numpy()
File "/usr/local/lib/python3.6/site-packages/xarray/core/common.py", line 229, in getattr
"{!r} object has no attribute {!r}".format(type(self).name, name)
AttributeError: 'DataArray' object has no attribute 'to_numpy'
WARNING:
WARNING: python_dataplane() -> an error occurred importing module "/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2"
WARNING:
ERROR :
ERROR : plot_data_plane -> trouble getting field "name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";" from file "PYTHON_NUMPY"
ERROR :

I found it strange that even having set MET_PYTHON_EXE to a 3.9 version of python it still used python3.6 in the invocation.

My next step was to move down to a python 3.6 environment

(python3.6) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity exec -e --bind $WORKDIR ../MET_11_0_1_BASE/met_v11.0.1.simg plot_data_plane PYTHON_NUMPY NAAPS2.ps 'name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -v 5
DEBUG 1: Start plot_data_plane by daviswj(927863) at 2024-08-23 20:01:07Z cmd: /usr/local/bin/plot_data_plane PYTHON_NUMPY NAAPS2.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -v 5
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object of type "FileType_Python_Numpy".
DEBUG 3: Running user's python script (/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2).
Traceback (most recent call last):
File "/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py", line 12, in
met_data = ds[var].isel(time=0).isel(sigma_layer=0).to_numpy()
File "/usr/local/lib/python3.6/site-packages/xarray/core/common.py", line 229, in getattr
"{!r} object has no attribute {!r}".format(type(self).name, name)
AttributeError: 'DataArray' object has no attribute 'to_numpy'
WARNING:
WARNING: python_dataplane() -> an error occurred importing module "/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2"
WARNING:
ERROR :
ERROR : plot_data_plane -> trouble getting field "name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";" from file "PYTHON_NUMPY"
ERROR :

Got rid of the segmentation fault but it's still erroring out

DanielAdriaansen Aug 23, 2024
Collaborator

OK, agreed probably a container or environment issue.

In your first error, I saw this:

DEBUG 3: Initializing MET compile time python instance: /usr/local/bin/python3

And in your last test with 11.0.1, I see this:

File "/usr/local/lib/python3.6/site-packages/xarray/core/common.py", line 229, in getattr
"{!r} object has no attribute {!r}".format(type(self).name, name)

Does that mean when using 12.0.0-beta1 you're also using Python 3.6? If so, I would encourage you to upgrade Python to 3.10.4+ and re-compile MET against it and see if that helps before we go any farther. The current documentation details requirements for Python embedding here:
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#compiling-met-for-python-embedding

Your latest error is because the Xarray version in your Python 3.6 does not have a to_numpy() method for DataArray's (it's probably old), but maybe a clue why the test with 12.0.0-beta1 is failing.

Answer selected by Wesley-J-Davis

Wesley-J-Davis Aug 23, 2024
Author

I'm getting the segmentation fault with python 3.10.14 too unfortunately, hmm.

Well we know the problem isn't MET itself so back to the drawing board. Thanks for helping out.

DanielAdriaansen Aug 23, 2024
Collaborator

Well we may still be able to help-

Can you send the full output from this test that you did:

hmm, I copied over your exact python reader and command and I'm still getting the segmentation fault.

So this would be using 12.0.0-beta1 in your container with plot_data_plane and my Python script that I sent?

Wesley-J-Davis Aug 26, 2024
Author

yea that's correct.

I got after it again this morning and I'm still getting the same issues.

this works

singularity exec /p/work1/daviswj/MET_11_0_1_BASE/met_v11.0.1.simg plot_data_plane
PYTHON_NUMPY
NAAPS2.ps
'name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -v 5

this does not

singularity exec /p/work1/daviswj/MET_12_0_0_BASE/met_v12.0.0-beta1.simg
plot_data_plane
PYTHON_NUMPY
NAAPS2.ps
'name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -v 5

I'ts this line where they diverge:

DEBUG 4: Reading temporary Python dataplane file: /tmp/tmp_met_nc_204366_0

Using version 11 the next info that gets displayed is about grid information, but with version 12, I get a segmentation fault.

FATAL ERROR (SEGFAULT): Process 204366 got signal 11 @ local time = 2024-08-26 18:56:07Z
FATAL ERROR (SEGFAULT): Look for a core file in /p/work1/daviswj/MONET-MET-INTEGRATION
FATAL ERROR (SEGFAULT): Process command line: /usr/local/bin/plot_data_plane PYTHON_NUMPY NAAPS2.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -v 5
Segmentation fault

I had a similar issue with plot_point_obs as weill in the version 12 container. I wasn't able to get that utility working with python embedding but ascii2nc worked fine with python embedding so I used the ascii2nc output to feed plot_point_obs and that was successful.

Wesley-J-Davis Aug 26, 2024
Author

Well we may still be able to help-

Can you send the full output from this test that you did:

hmm, I copied over your exact python reader and command and I'm still getting the segmentation fault.

So this would be using 12.0.0-beta1 in your container with plot_data_plane and my Python script that I sent?

(python3.9) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity exec /p/work1/daviswj/MET_12_0_0_BASE/met_v12.0.0-beta1.simg plot_data_plane PYTHON_NUMPY NAAPS2.ps 'name="/p/work1/daviswj/MONET-ME
T-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py";' -v 5
DEBUG 1: Start plot_data_plane by daviswj(927863) at 2024-08-26 18:56:05Z cmd: /usr/local/bin/plot_data_plane PYTHON_NUMPY NAAPS2.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -v 5
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object of type "FileType_Python_Numpy".
DEBUG 3: Calling /p/home/daviswj/.conda/envs/python3.9/bin/python to run user's python script (/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2).
DEBUG 4: Writing temporary Python dataplane file:
DEBUG 4: /p/home/daviswj/.conda/envs/python3.9/bin/python /usr/local/share/met/python/pyembed/write_tmp_dataplane.py /tmp/tmp_met_nc_204366_0 /p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2
Python Script: '/usr/local/share/met/python/pyembed/write_tmp_dataplane.py'
User Command: '/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2'
Temporary File: '/tmp/tmp_met_nc_204366_0'
[[242.72017 242.71881 242.71745 ... 242.72289 242.72153 242.72153]
[242.82454 242.82182 242.82047 ... 242.83131 242.82996 242.82724]
[243.06174 243.05766 243.05225 ... 243.07664 243.07123 243.06715]
...
[240.58672 240.58536 240.58536 ... 240.58943 240.58943 240.58807]
[240.95268 240.95404 240.9554 ... 240.94861 240.94997 240.95132]
[240.6057 240.60704 240.60704 ... 240.60162 240.60298 240.60434]]
{'valid': '20230120_000000', 'init': '20230120_000000', 'lead': '0', 'accum': '000000', 'name': 'air_temperature', 'long_name': 'air_temperature', 'level': 'Surface', 'units': 'K', 'grid': {'type': 'LatLon', 'name': 'NAAPS2', 'Nlon': 1080, 'Nlat': 540, 'lat_ll': -89.83333587646484, 'lon_ll': -179.8333282470703, 'delta_lat': 0.3, 'delta_lon': 0.3}}
DEBUG 3: Initializing MET compile time python instance: /usr/local/bin/python3
DEBUG 4: Reading temporary Python dataplane file: /tmp/tmp_met_nc_204366_0
FATAL ERROR (SEGFAULT): Process 204366 got signal 11 @ local time = 2024-08-26 18:56:07Z
FATAL ERROR (SEGFAULT): Look for a core file in /p/work1/daviswj/MONET-MET-INTEGRATION
FATAL ERROR (SEGFAULT): Process command line: /usr/local/bin/plot_data_plane PYTHON_NUMPY NAAPS2.ps name="/p/work1/daviswj/MONET-MET-INTEGRATION/lib/NAAPS_PYTHON_READER_2.py"; -v 5
Segmentation fault

Wesley-J-Davis · 2024-08-28T15:16:08Z

Wesley-J-Davis
Aug 28, 2024
Author

Here's the python version checking:

Setting python3.9 and checking version

(python3.9) (base) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> python
Python 3.9.19 (main, Mar 21 2024, 17:11:28) 
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> print(numpy.__version__)
1.24.3
>>> 
(python3.9) (base) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> which python
/p/home/daviswj/.conda/envs/python3.9/bin/python

Shelling into the met 12 container to look for /usr/local/bin/python3

(python3.9) (base) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> /usr/local/bin/python3
bash: /usr/local/bin/python3: No such file or directory
(python3.9) (base) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity shell /p/work1/dav
iswj/MET_12_0_0_BASE/met_v12.0.0-beta1.simg
Singularity> /usr/local/bin/python3
Python 3.10.4 (main, Sep  6 2023, 18:25:14) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Could not open PYTHONSTARTUP
FileNotFoundError: [Errno 2] No such file or directory: '/etc/pythonstart'
>>> import numpy
>>> print(numpy.__version__)
1.25.2

shelling into the met11 container to do the same thing

(python3.9) (base) daviswj@narwhal13:/p/work1/daviswj/MONET-MET-INTEGRATION> singularity shell /p/work1/dav
iswj/MET_11_0_1_BASE/met_v11.0.1.simg 
Singularity> /usr/local/bin/python3
bash: /usr/local/bin/python3: No such file or directory
Singularity> python
Python 2.7.5 (default, Jun 28 2022, 15:30:04) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Could not open PYTHONSTARTUP
IOError: [Errno 2] No such file or directory: '/etc/pythonstart'
>>> import numpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named numpy

0 replies

Wesley-J-Davis · 2024-08-28T15:17:41Z

Wesley-J-Davis
Aug 28, 2024
Author

exact contents of NAAPS_PYTHON_READER_2.py

import xarray as xr
import numpy as np
import sys

var = str(sys.argv[1])
file = "/p/work1/daviswj/MONET-MET-INTEGRATION/fcst/2023012000_000_000_nap045r3x3_000.nc"
#var = 'tair'
ds = xr.open_dataset(file)
print (ds)
#ds = xr.open_dataset(f"/p/work1/daviswj/MONET-MET-INTEGRATION/fcst/{file_name}")

llx = ds.isel(latitude=0).latitude.values
lly = ds.isel(longitude=0).longitude.values
print(ds.latitude.values[-1])
print(ds.longitude.values[-1])

met_data = ds[var].isel(time=0).isel(sigma_layer=0).to_numpy()

# Flip the data across the equator
met_data = met_data[::-1]

var_data = ds[var].isel(time=0).isel(sigma_layer=0)

attrs = {
'valid': ds.time.dt.strftime("%Y%m%d_%H%M%S").item(),
'init': ds.atime.dt.strftime("%Y%m%d_%H%M%S").item(),
'lead': str(int((ds.time-ds.atime).dt.total_seconds()/3600)),
'accum': '000000',
'name': var_data.attrs['standard_name'],
'long_name': var_data.attrs['long_name'],
'level': 'Surface',
'units': var_data.attrs['units'],
'grid': {
'type': 'LatLon',
'name': 'NAAPS2',
'Nlat': len(ds.latitude),
'Nlon': len(ds.longitude),
'lat_ll': float(ds.latitude.values[0]),
'lon_ll': float(ds.longitude.values[0]),
'delta_lat':180./len(ds.latitude),
'delta_lon':360./len(ds.longitude),
}
}
print (met_data)
print (attrs)

8 replies

Wesley-J-Davis Aug 28, 2024
Author

By the way, is the new version coming online soon?

Wesley-J-Davis Aug 28, 2024
Author

Thanks for all of the info! I worked with @JohnHalleyGotway this morning for a bit to try and dig in some more. We discovered an issue with the MET-12.0.0-beta1 Docker image file, and were unsuccessful using that to run your Python embedding script.

Instead, we switched to the latest MET-12.0.0-beta5 image and were able to run the script successfully. Could you try switching to this image instead? Here is a link to the Docker image we used: https://hub.docker.com/layers/dtcenter/met/12.0.0-beta5/images/sha256-b39b97f7c8fe1e66e78a0ac222e2b01534181d4d0ce44c07ec0d9b24841ad981?context=explore.

Also, I am not sure whether Singularity is inheriting the value of MET_PYTHON_EXE that you are setting prior to running Singularity, but it is probably best practice to make sure that if you need to use MET_PYTHON_EXE, it points to a Python inside the container and not on your local system, to avoid potential conflicts with shared libraries.

And finally, I will note that your Python script as it is should work without using MET_PYTHON_EXE, and therefore I'd recommend just running MET without setting that, and only switch to using MET_PYTHON_EXE if the need arises later on with special Python library requirements not available in the Python version used to compile MET.

Please let us know if any of these suggestions help or are successful on your end. Thanks!

I'm sure the plot_data_plane graphic would work but I have been having this same issue with plot_point_obs as well. I was able to get around that by using ascii2nc on my point_ob data and then running that through plot_point_obs and point_stat. My point observations python embedding for certain has python libraries that weren't in the MET build since it's using monet and monetio from https://github.com/noaa-oar-arl.

Just so you're aware, point_stat was failing in my instance too. I was trying to python embed both the forecast and the obs and version 12 would give the same segmentation error running that utility as with plot_data_plane. Version 11 didn't have that same issue.

DanielAdriaansen Aug 29, 2024
Collaborator

By the way, is the new version coming online soon?

If you mean version 12.0.0, it looks like based on our current projection, December 11th 2024:
https://github.com/orgs/dtcenter/projects/74/settings/fields/35866684.

Final beta (6) in October, release candidate in November, and coordinated release in December is the plan at the moment.

jprestop Aug 29, 2024
Maintainer

@DanielAdriaansen, thanks for pointing @Wesley-J-Davis to the Cycle settings for the METplus release schedule. I wanted to point out that we've also started including this information in our METplus User's Guide in the METplus Release Information section of our "develop" version (version selector in the lower left bottom) of the documentation, in case that is helpful.

DanielAdriaansen Aug 29, 2024
Collaborator

I'm sure the plot_data_plane graphic would work but I have been having this same issue with plot_point_obs as well. I was able to get around that by using ascii2nc on my point_ob data and then running that through plot_point_obs and point_stat. My point observations python embedding for certain has python libraries that weren't in the MET build since it's using monet and monetio from https://github.com/noaa-oar-arl.

Just so you're aware, point_stat was failing in my instance too. I was trying to python embed both the forecast and the obs and version 12 would give the same segmentation error running that utility as with plot_data_plane. Version 11 didn't have that same issue.

Right, the error we are seeing with 12.0.0-beta1 would affect Python embedding regardless of which tool or fcst/obs you're using it for. Your solution for getting around the Python embedding using ascii2nc before plot_point_obs is a good idea for now until we can get the Python embedding working. Thanks for pointing out that for the obs, you'll need Python modules and thus MET_PYTHON_EXE, which hopefully will not be a problem with a new 12.0.0-beta version if possible.

It also makes sense that point_stat would be failing as well given what we observed. when using beta1. I am hopeful you'll be able to switch to beta5! Let us know how it goes. Thanks!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FATAL ERROR (SEGFAULT): when python embedding #2660

{{title}}

Replies: 3 comments 21 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

FATAL ERROR (SEGFAULT): when python embedding #2660

Wesley-J-Davis Aug 23, 2024

Replies: 3 comments · 21 replies

DanielAdriaansen Aug 23, 2024 Collaborator

Wesley-J-Davis Aug 23, 2024 Author

DanielAdriaansen Aug 23, 2024 Collaborator

Wesley-J-Davis Aug 23, 2024 Author

DanielAdriaansen Aug 23, 2024 Collaborator

Wesley-J-Davis Aug 23, 2024 Author

DanielAdriaansen Aug 23, 2024 Collaborator

Wesley-J-Davis Aug 26, 2024 Author

Wesley-J-Davis Aug 26, 2024 Author

Wesley-J-Davis Aug 28, 2024 Author

Wesley-J-Davis Aug 28, 2024 Author

Wesley-J-Davis Aug 28, 2024 Author

Wesley-J-Davis Aug 28, 2024 Author

DanielAdriaansen Aug 29, 2024 Collaborator

jprestop Aug 29, 2024 Maintainer

DanielAdriaansen Aug 29, 2024 Collaborator

Wesley-J-Davis
Aug 23, 2024

Replies: 3 comments 21 replies

DanielAdriaansen
Aug 23, 2024
Collaborator

Wesley-J-Davis Aug 23, 2024
Author

DanielAdriaansen Aug 23, 2024
Collaborator

Wesley-J-Davis Aug 23, 2024
Author

DanielAdriaansen Aug 23, 2024
Collaborator

Wesley-J-Davis Aug 23, 2024
Author

DanielAdriaansen Aug 23, 2024
Collaborator

Wesley-J-Davis Aug 26, 2024
Author

Wesley-J-Davis Aug 26, 2024
Author

Wesley-J-Davis
Aug 28, 2024
Author

Wesley-J-Davis
Aug 28, 2024
Author

Wesley-J-Davis Aug 28, 2024
Author

Wesley-J-Davis Aug 28, 2024
Author

DanielAdriaansen Aug 29, 2024
Collaborator

jprestop Aug 29, 2024
Maintainer

DanielAdriaansen Aug 29, 2024
Collaborator