relion 5 pick tomograms: RuntimeError: no validator found for <class 'numpy.ndarray'>, see arbitrary_types_allowed in Config

[RichStone99]

Installed current relion 5 on centOS 9. Am trying the subtomogram averaging tutorial with the HIV VLP data set. Worked well up to "Reconstruct tomograms". Tomograms have been generated and look fine. But when trying to launch "Pick tomograms", napari didn't open and got the following error message. How can we fix it? Seems to relate to pydantic and/or tomography_python_programs.
Btw, napari ran well during, for example, the "Exclude tilt-images" step and inspection of generated tomo in the "Reconstruct tomograms" step. IMOD was used in "Reconstruct tomograms".

Traceback (most recent call last):
File "", line 1, in
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/init.py", line 1, in
from .particles import pick_particles
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/particles.py", line 7, in
from napari_threedee.annotators import PointAnnotator
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/init.py", line 1, in
from . import data_models
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/init.py", line 1, in
from .points import N3dPoints
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/points.py", line 8, in
from napari_threedee.annotators.base import N3dDataModel
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/init.py", line 5, in
from .surfaces import SurfaceAnnotator
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/surfaces/init.py", line 1, in
from .annotator import SurfaceAnnotator
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/surfaces/annotator.py", line 12, in
from morphosamplers.surface_spline import GriddedSplineSurface
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/init.py", line 12, in
from morphosamplers.models import Path, Sphere, Surface, Dipole
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/models/init.py", line 1, in
from .path import Path
File "/home/tangl/sw/miniforge3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/models/path.py", line 7, in
class Path(MorphoModel):
File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.new
File "pydantic/fields.py", line 504, in pydantic.fields.ModelField.infer
File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.init
File "pydantic/fields.py", line 555, in pydantic.fields.ModelField.prepare
File "pydantic/fields.py", line 829, in pydantic.fields.ModelField.populate_validators
File "pydantic/validators.py", line 765, in find_validators
RuntimeError: no validator found for <class 'numpy.ndarray'>, see arbitrary_types_allowed in Config

[labzw]

Having exactly the same error. I feel this might be a bug in morphosamplers. It seems natively Pydantic does not support ndarray.

Solved or bypassed this issue by adding

class Config:
        arbitrary_types_allowed = True

in dipole.py path.py sphere.py surface.py files under /YOUR/CONDA/PATH/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/models

[alisterburt]

@RichStone99 @labzw apologies for this - could you please run pip show morphosamplers and pip show pydantic in your RELION 5 conda environment and tell me the output? We recently moved to only supporting pydantic>=2 which may be the root cause of this issue for you

[CACNYC]

@alisterburt I have an identical error to @RichStone99 @labzw :
Traceback (most recent call last):
File "",
, line 1, in < module»
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/_init_py", line 1, in from particles import pick_particles
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/particles.py", line 7, in from napari_threedee.annotators import PointAnnotator
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/init.py", line 1, in from. import data models
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/_init_py", line 1, in from points import N3dPoints
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/points.py", line 8, in from napari threedee.annotators.base import N3dDataModel
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/_init.py", line 5, in
from surfaces import SurfaceAnnotator
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/surfaces/_init_py", line 1, in < module> from annotator import SurfaceAnnotator
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/annotators/surfaces/annotator.py", line 12, in from morphosamplers.surface_spline import GriddedSplineSurface
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/_init_py", line 12, in from morphosamplers.models import Path, Sphere, Surface, Dipole
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/models/_init_py", line 1, in from path import Path
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/morphosamplers/models/path.py", line 7, in class Path(MorphoModel):
File "pydantic/main.py", line 197, in pydantic.main.ModelMetaclass.new File "pydantic/fields.py", line 504, in pydantic.fields.ModelField.infer File "pydantic/fields.py", line 434, in pydantic.fields.ModelField.__init File "pydantic/fields.py", line 555, in pydantic.fields.ModelField prepare File "pydantic/fields.py", line 829, in pydantic.fields.ModelField.populate_validators File "pydantic/validators.py", line 765, in find_validators
RuntimeError: no validator found for <class 'numpy.ndarray >, see 'arbitrary_types_allowed* in Config

Output of pip show morphosamplers:
(relion-5.0) [spuser@spgpu TS_01_ projecti$ pip show morphosamplers
Name: morphosamplers
Version: 0.0.15
Summary: A library for sampling image data along morphological objects such as splines and surfaces.
Home-page: https: //github.com/kevinyamauchi/morphosamplers
Author: Kevin Yamauchi
Author-email: [email protected]
License: BSD 3-Clause License
Location: /opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages
Requires: einops, numpy, psygnal, pydantic, pydantic-compat, scipy, typing-extensions
•Required-by: napari-threedee, relion

Output of pip show pydantic:
(relion-5.0) [spuser@spgpu TS_01 projecti$ pip show pydantic
-Name: pydantic
Version: 1.10.18
Summary: Data validation and settings management using python type hints
Home-page: https: //github.com/pydantic/pydantic
rAuthor: Samuel Colvin
l Author-email: [email protected]
License: MIT
Location: /opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages
Requires: typing-extensions
Required-by: app-model, mdocfile, morphosamplers, napari, napari-threedee, npez, pydantic-compat

Ah, then pydantic 1.x is the culprit?

Thanks for your help Alister!

[CACNYC]

Hmm, it looks like I have a napari version constraint with napari 0.4.18

[spuser@spgpu TS_01_project]$ pip install --upgrade pydantic
Defaulting to user installation because normal site-packages is not writeable Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pydantic in /opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages (1.10.18)
Collecting pydantic
Downloading pydantic-2.9.2-py3-none-any.whl.metadata (149 kB)
Collecting annotated-types>=0.6.0 (from pydantic)
Downloading annotated_types-0.7.0-py-none-any. whl.metadata (15 kB)
Collecting pydantic-core==2.23.4 (from pydantic)
Downloading pydantic_core-2.23.4-cp310-c310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Requirement already satisfied: typing-extensions>=4.6.1 in /opt/miniconda3/envs/relion-5.0/Lib/python3.10/site-packages (from pydantic) (4.12.2)
Downloading pydantic-2.9.2-py3-none-any.whl (434 kB)
DownLoading pydantic_core-2.23,4-cp310-cp310-manyLinux_2_17_x86_64.manylinux2014_x86_64.wh1 (2.1 MB)
Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)
Installing collected packages: pydantic-core, annotated-types, pydantic
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
napari 0.4.18 requires pydantic<2,>=1.9.0, but you have pydantic 2.9.2 which is incompatible.
Successfully installed annotated-types-0.7.0 pydantic-2.9.2 pydantic-core-2.23.4

In relion5:
Traceback (most recent call last):
File "", line 1, in ‹module>
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/_init_py". line 1, in from particles import pick_particles
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/pick/particles.py", line 7, in from napari threedee.annotators import PointAnnotator
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/_init_py", line 1, in from. import data models
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/_init_py", line 1, in < module> from points import N3dPoints
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari_threedee/data_models/points.py", line 3, in import napari.layers
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/init.py", line 9, in < module > from napari.layers.base import Layer
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/base/_init_py", line 1, in from napari.layers.base._base_constants import ActionType
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/base/_base_constants.py", line 4, in < module> from napari.utils.misc import StringEnum
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/_init_py", line 2, in from napari.utils.colormaps import Colormap
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/colormaps/_init-py", line 2, in from napari.utils.co/ormaps.colormap import Colormap
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/colormaps/colormap.py", line 9, in from napari.utils.events import EventedModel
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/events/_init_py", line 17, in from napari.utils.events.evented_model import EventedModel
File "/opt/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/events/evented_model.py", line 66, in class EventedMetaclass(main.ModelMetaclass):
File "/home/spuser/.local/lib/python3.10/site-packages/pydantic/_migration.py", line 306, in wrapper raise AttributeError(f'module {module!r} has no attribute {name!r}") AttributeError: module 'pydantic.main' has no attribute "ModelMetaclass"

[bogdantoader]

Hello, downgrading to the following versions in the conda environment should fix the issue:

morphosamplers.version
'0.0.13'
pydantic.version
'1.10.18'
napari.version
'0.4.18'

[CACNYC]

Hello, downgrading to the following versions in the conda environment should fix the issue:

morphosamplers.version
'0.0.13'
pydantic.version
'1.10.18'
napari.version
'0.4.18'

Thanks Bogdan, I can confirm that rolling back morphosamplers to version 0.0.13, combined with pydantic 1.10.18 and napari 0.4.18, allowed both the Exclude tilt and Pick tomograms job types to run and complete successfully.

[labzw]

Update:

Remove and recreate conda environment with latest environment.yml works!

So after I rebuild the latest version and roll back morphosamplers to 0.0.13, I am getting this error now.......

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/exclude_tilt_images/__init__.py", line 1, in <module>
    from ._cli import cli
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/exclude_tilt_images/_cli.py", line 8, in <module>
    from .relion_tilt_image_excluder import RelionTiltImageExcluderWidget
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/exclude_tilt_images/relion_tilt_image_excluder.py", line 5, in <module>
    from .._qt.tilt_image_excluder import TiltImageExcluderWidget
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/_qt/tilt_image_excluder.py", line 8, in <module>
    from .components.tilt_image_selection_widget import TiltImageSelectionWidget
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/_qt/components/tilt_image_selection_widget.py", line 12, in <module>
    class TiltImageSelectionWidget(QWidget):
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/tomography_python_programs/_qt/components/tilt_image_selection_widget.py", line 17, in TiltImageSelectionWidget
    viewer: napari.Viewer,
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/_lazy.py", line 45, in __getattr__
    submod = import_module(
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/viewer.py", line 8, in <module>
    from napari.components.viewer_model import ViewerModel
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/components/__init__.py", line 19, in <module>
    from napari.components.layerlist import LayerList
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/components/layerlist.py", line 9, in <module>
    from napari.layers import Layer
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/__init__.py", line 12, in <module>
    from napari.layers.points import Points
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/points/__init__.py", line 1, in <module>
    from napari.layers.points import _points_key_bindings
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/points/_points_key_bindings.py", line 4, in <module>
    from napari.layers.points.points import Points
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/points/points.py", line 38, in <module>
    from napari.layers.utils.text_manager import TextManager
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/utils/text_manager.py", line 29, in <module>
    class TextManager(EventedModel):
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/layers/utils/text_manager.py", line 83, in TextManager
    string: StringEncoding = ConstantStringEncoding(constant='')
  File "/usr/bin/miniconda3/envs/relion-5.0/lib/python3.10/site-packages/napari/utils/events/evented_model.py", line 242, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for ConstantStringEncoding
constant
  Unable to avoid copy while creating an array as requested.
If using `np.array(obj, copy=False)` replace it with `np.asarray(obj)` to allow a copy when needed (no behavior change in NumPy 1.x).
For more details, see https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword. (type=value_error)