Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError: cannot pickle 'PyCapsule' object when adding intercept #301

Closed
hummuscience opened this issue Oct 17, 2023 · 6 comments
Closed

TypeError: cannot pickle 'PyCapsule' object when adding intercept #301

hummuscience opened this issue Oct 17, 2023 · 6 comments
Assignees
@digicosmos86

Comments

@hummuscience
Copy link

hummuscience commented Oct 17, 2023 •
edited
Loading

Describe the bug
A clear and concise description of what the bug is.

HSSM version
0.1.5 with python 3.9 and latest dev version with python 3.10

To Reproduce


import hssm

cav_data = hssm.load_data("cavanagh_theta")

The following produces an error:


model_side = hssm.HSSM(
    data=cav_data,
    model="ddm",
    a = 1,
    hierarchical = False,
    include=[
	{
	    "name": "v",  # Drift rate
	    "prior": {
                "Intercept": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
                }
	    },
	    "formula": "v ~ 1 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
	{
	    "name": "z",  # Starting point
	    "prior": {
                "Intercept": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
                }
	    },
	    "formula": "z ~ 1 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
	{
	    "name": "t",  # Starting point
	    "prior": {
                "Intercept": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
                } 
	    },
	    "formula": "t ~ 1 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
    ],
)


/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
Traceback (most recent call last):
  File "<string>", line 17, in __PYTHON_EL_eval
  File "<string>", line 160, in <module>
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/hssm/hssm.py", line 268, in __init__
    self.model = bmb.Model(
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/bambi/models.py", line 160, in __init__
    design = fm.design_matrices(
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/matrices.py", line 556, in design_matrices
    design = DesignMatrices(description, data, env)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/matrices.py", line 54, in __init__
    self.model.eval(data, env)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/terms.py", line 1258, in eval
    self.add_extra_terms(encodings, data, env)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/terms.py", line 1240, in add_extra_terms
    extra_term = create_extra_term(term, subencoding, data, env)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/terms.py", line 1294, in create_extra_term
    extra_term = Term(*deepcopy(components))
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 206, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/muad/anaconda3/envs/hssm10/lib/python3.10/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle 'PyCapsule' object

But if I remove the intercept, no error, only the usual warnings about categorical predictors:


model_side = hssm.HSSM(
    data=cav_data,
    model="ddm",
    a = 1,
    hierarchical = False,
    include=[
	{
	    "name": "v",  # Drift rate
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "v ~ 0 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
	{
	    "name": "z",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "z ~ 0 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
	{
	    "name": "t",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "t ~ 0 + (stim|subj_idx) + (1|subj_idx) + C(dbs):C(conf)",
	    "link": "identity",
	},
    ],
)



/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):
/home/muad/anaconda3/envs/hssm10/lib/python3.10/site-packages/formulae/terms/call.py:108: FutureWarning: is_categorical_dtype is deprecated and will be removed in a future version. Use isinstance(dtype, CategoricalDtype) instead
  elif is_string_dtype(x) or is_categorical_dtype(x) or isinstance(x, CategoricalBox):

edit: removed the intercept prior in the second model (without the intercept), even though it didnt make a difference
@hummuscience
Copy link
Author

It seems to be connected to the interaction term. When I remove it, there are no errors:


model_side = hssm.HSSM(
    data=cav_data,
    model="ddm",
    a = 1,
    hierarchical = False,
    include=[
	{
	    "name": "v",  # Drift rate
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "v ~ 1 + (stim|subj_idx) + (1|subj_idx)",
	    "link": "identity",
	},
	{
	    "name": "z",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "z ~ 1 + (stim|subj_idx) + (1|subj_idx)",
	    "link": "identity",
	},
	{
	    "name": "t",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "t ~ 1 + (stim|subj_idx) + (1|subj_idx)",
	    "link": "identity",
	},
    ],
)

@hummuscience
Copy link
Author 

model_side = hssm.HSSM(
    data=cav_data,
    model="ddm",
    a = 1,
    hierarchical = False,
    include=[
	{
	    "name": "v",  # Drift rate
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "v ~ 1 + C(dbs)*C(conf)",
	    "link": "identity",
	},
	{
	    "name": "z",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "z ~ 1 + C(dbs)*C(conf)",
	    "link": "identity",
	},
	{
	    "name": "t",  # Starting point
	    "prior": {
		    "name": "Normal",
		    "sigma": 1,
		    "mu": 0.0,
	    },
	    "formula": "t ~ 1 + C(dbs)*C(conf)",
	    "link": "identity",
	},
    ],
)

This however, seem to work

@digicosmos86
Copy link
Collaborator

Thanks for letting us know! I tested the code myself and got a different but very similar error. It seems this happens when bambi is trying to create design matrices. It uses deepcopy which involves pickling some objects, but not object types are supported. Will create an issue for bambi developers

@digicosmos86 digicosmos86 mentioned this issue Oct 17, 2023
Open
@digicosmos86 digicosmos86 self-assigned this Oct 17, 2023
@tomicapretto
Copy link
Collaborator

@Cumol have a look at bambinos/bambi#738 (comment) :)

@digicosmos86 digicosmos86 mentioned this issue Oct 20, 2023
Closed
@digicosmos86
Copy link
Collaborator

@Cumol I suppose we can close this now? Please feel free to reopen this if you have any further questions

@tomicapretto
Copy link
Collaborator

@digicosmos86 yes I think we can close. This is a problem at the formulae level. I'll open an issue now.

@tomicapretto tomicapretto mentioned this issue Oct 25, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@digicosmos86 digicosmos86

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hummuscience @digicosmos86 @tomicapretto