error on tutorial Evolution of sequence, structure and dynamics with Evol and SignDy

[Armanda87]

Good morning
i found another error in the tutorial "Evolution of sequence, structure and dynamics with Evol and SignDy" on the step "Step 2: Mode ensemble"

Thanks and i hope you can help me again

saveModeEnsemble(gnms, 'PBP-I')

the error "ValueError Traceback (most recent call last)
Cell In[49], line 1
----> 1 saveModeEnsemble(gnms, 'PBP-I')

File ~/miniconda3/envs/py39/lib/python3.8/site-packages/ProDy-2.4.1-py3.8-linux-x86_64.egg/prody/dynamics/signature.py:1887, in saveModeEnsemble(mode_ensemble, filename, atoms, **kwargs)
1884 filename += '.npz'
1886 ostream = openFile(filename, 'wb', **kwargs)
-> 1887 np.savez_compressed(ostream, **attr_dict)
1888 ostream.close()
1890 return filename

File <array_function internals>:200, in savez_compressed(*args, **kwargs)

File ~/miniconda3/envs/py39/lib/python3.8/site-packages/numpy/lib/npyio.py:686, in savez_compressed(file, *args, **kwds)
623 @array_function_dispatch(_savez_compressed_dispatcher)
624 def savez_compressed(file, *args, **kwds):
625 """
626 Save several arrays into a single file in compressed .npz format.
627
(...)
684
685 """
--> 686 _savez(file, args, kwds, True)

File ~/miniconda3/envs/py39/lib/python3.8/site-packages/numpy/lib/npyio.py:716, in _savez(file, args, kwds, compress, allow_pickle, pickle_kwargs)
714 for key, val in namedict.items():
715 fname = key + '.npy'
--> 716 val = np.asanyarray(val)
717 # always force zip64, gh-10776
718 with zipf.open(fname, 'w', force_zip64=True) as fid:

ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (85,) + inhomogeneous part."

[jamesmkrieger]

what version of numpy are you using?

[Armanda87]

numpy 1.23.5 pypi_0 pypi

[jamesmkrieger]

I've just tried in environments with two different numpy versions using another ensemble that I have and that worked in both cases. I also tried both the latest development master branch and v2.4.1 from the tag and both worked in the latter environment with the higher numpy version

python 3.8.13 h12debd9_0
numpy 1.18.4 pypi_0 pypi

python 3.9.18 h955ad1f_0
numpy 1.26.2 pypi_0 pypi

In [1]: from prody import *

In [2]: dali_ens = loadEnsemble('all_ensemble.ens.npz')[:5]

In [3]: ag = dali_ens.getAtoms().copy()

In [4]: dali_ens.setAtoms(ag.select('chain A'))

In [5]: gnms = calcEnsembleENMs(dali_ens, model='GNM', trim='reduce')
   ...: gnms
@> Matching 20 modes across 5 modesets... [ 80%] 1sOut[5]: <ModeEnsemble: 5 modesets (20 modes, 949 atoms)>

In [6]: saveModeEnsemble(gnms, 'PBP-I')
/home/jkrieger/software/miniconda/envs/prody-github/lib/python3.9/site-packages/numpy/lib/npyio.py:713: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  val = np.asanyarray(val)
Out[6]: 'PBP-I.modeens.npz'

I've also just tried this in a new conda environment with python 3.8 and installing prody from pypi/pip, getting numpy 1.23.5 too, and that also works. I've also tried in this environment with the first 10 pdb_ids from the filtered dali_rec and that worked fine too:

In [1]: from prody import *

In [2]: dali_rec = searchDali('3H5V','A')
   ...: dali_rec
Connecting to Dali for search results...Out[2]: <prody.database.dali.DaliRecord at 0x7fb9dca19190>

In [3]: dali_rec
Out[3]: <prody.database.dali.DaliRecord at 0x7fb9dca19190>

In [4]: dali_rec.isSuccess
Out[4]: True

In [5]: pdb_ids = dali_rec.filter(cutoff_len=0.7, cutoff_rmsd=1.0, cutoff_Z=30)

In [6]: mappings = dali_rec.getMappings()

In [7]: ags = parsePDB(pdb_ids, subset='ca')
   ...: len(ags)
@> Retrieving 2wjxB... [  9%] 430s^C^C
KeyboardInterrupt


In [8]: len(pdb_ids)
Out[8]: 244

In [9]: pdb_ids = pdb_ids[:10]

In [10]: ags = parsePDB(pdb_ids, subset='ca')
    ...: len(ags)
@> Retrieving 6ruqD... [ 90%] 3sOut[10]: 10

In [11]: dali_ens = buildPDBEnsemble(ags, mapping=mappings, seqid=20, labels=pdb_ids)
    ...: dali_ens
@> Mapping 5l1fB_ca to the reference... [ 10%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 4u5cB_ca to the reference... [ 20%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 5l1eB_ca to the reference... [ 30%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 5l1hB_ca to the reference... [ 40%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 5l1gB_ca to the reference... [ 50%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 7rz9C_ca to the reference... [ 70%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 4u1wD_ca to the reference... [ 80%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> Mapping 6ruqD_ca to the reference... [ 90%] 1s@> WARNING no atommaps were available. Consider adjusting accepting criteria
@> WARNING 8 structures cannot be mapped.
Out[11]: <PDBEnsemble: Unknown (2 conformations; 376 atoms)>

In [12]: saveEnsemble(dali_ens, 'PBP-I')
Out[12]: 'PBP-I.ens.npz'

In [13]: dali_ens = loadEnsemble('PBP-I.ens.npz')

In [14]: gnms = calcEnsembleENMs(dali_ens, model='GNM', trim='reduce')
    ...: gnms
@> Matching 20 modes across 2 modesets... [ 50%] 1sOut[14]: <ModeEnsemble: 2 modesets (20 modes, 376 atoms)>

In [15]: saveEnsemble(dali_ens, 'PBP-I')
Out[15]: 'PBP-I.ens.npz'

In [16]: dali_ens
Out[16]: <PDBEnsemble: Unknown (2 conformations; 376 atoms)>

In [17]: saveModeEnsemble(gnms, 'PBP-I')
/home/jkrieger/software/miniconda/envs/py38/lib/python3.8/site-packages/numpy/lib/npyio.py:716: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  val = np.asanyarray(val)
Out[17]: 'PBP-I.modeens.npz'

So, I'm not sure what's wrong.

@atbogetti, any ideas?

[atbogetti]

Hello @Armanda87 and @jamesmkrieger, I will look into this shortly and write back with any findings.

[atbogetti]

I am actually getting the same error as you, @Armanda87. @jamesmkrieger I cannot get it to work for me using the same code snippet in the tutorial and the one you provided. I am using the most recent prody installed from source with python 3.11 and numpy 1.23.5.

[atbogetti]

I changed this line (

ProDy/prody/dynamics/signature.py

Line 1887 in 412c8fa

np.savez_compressed(ostream, **attr_dict)

):

np.savez_compressed(ostream, **attr_dict) to np.savez_compressed(ostream, attr_dict) and it doesn't give me the error anymore. Is there anyway to check if my npz file was saved correctly @jamesmkrieger ?

[jamesmkrieger]

I am actually getting the same error as you, @Armanda87. @jamesmkrieger I cannot get it to work for me using the same code snippet in the tutorial and the one you provided. I am using the most recent prody installed from source with python 3.11 and numpy 1.23.5.

Why python 3.11?

It’s unlikely that ProDy is supported for that. We run checks up to 3.10

The bug report also mentions files in python 3.8 directories

Nevertheless, it’s good you can reproduce the error

[jamesmkrieger]

I changed this line (

ProDy/prody/dynamics/signature.py

Line 1887 in 412c8fa

np.savez_compressed(ostream, **attr_dict)

):
np.savez_compressed(ostream, **attr_dict) to np.savez_compressed(ostream, attr_dict) and it doesn't give me the error anymore. Is there anyway to check if my npz file was saved correctly @jamesmkrieger ?

Load it back and see if you can get reasonable results from all the subsequent analyses and probably some prior things in the tutorials too

[jamesmkrieger]

You’ll also have to see if that fix works in older python versions