CFA and SEM with PyMC #698

NathanielF · 2024-09-01T22:18:08Z

CFA and SEM with PyMC

Related to this issue here I'm adding a PR to demonstrate the functionality of CFA and SEM models using PyMC
#695

Notebook follows style guide https://docs.pymc.io/en/latest/contributing/jupyter_style.html
PR description contains a link to the relevant issue:
- a tracker one for existing notebooks (tracker issues have the "tracker id" label)
- or a proposal one for new notebooks
Check the notebook is not excluded from any pre-commit check: https://github.com/pymc-devs/pymc-examples/blob/main/.pre-commit-config.yaml

Helpful links

https://github.com/pymc-devs/pymc-examples/blob/main/CONTRIBUTING.md

📚 Documentation preview 📚: https://pymc-examples--698.org.readthedocs.build/en/698/

Signed-off-by: Nathaniel <[email protected]>

review-notebook-app · 2024-09-01T22:18:12Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

Signed-off-by: Nathaniel <[email protected]>

NathanielF · 2024-09-08T16:22:08Z

Think this is ready for review @drbenvincent , @cluhmann , @AlexAndorra . Any feedback/pushback welcome.

NathanielF · 2024-09-16T09:32:06Z

Giving this another nudge!

Thanks!

drbenvincent · 2024-09-17T17:34:19Z

FYI: I'm totally overloaded at the moment, so going to struggle with a review. Will see if things ease up a bit next week.

NathanielF · 2024-09-17T20:14:34Z

No worries.

review-notebook-app · 2024-09-25T03:08:46Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:46Z
----------------------------------------------------------------

Perhaps add reference to psychometrics in the title to make it more specific, and aid in discoverability.

NathanielF commented on 2024-09-25T11:38:08Z
----------------------------------------------------------------

Good idea.

review-notebook-app · 2024-09-25T03:08:47Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:47Z
----------------------------------------------------------------

nitpick: data set should not have a hyphen; data set or dataset, no?

NathanielF commented on 2024-09-25T11:38:41Z
----------------------------------------------------------------

changed.

review-notebook-app · 2024-09-25T03:08:48Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:47Z
----------------------------------------------------------------

Maybe hyperlink SEM and CFA to some introductory materials on the topics (even a Wikipedia link would be good)

NathanielF commented on 2024-09-25T11:38:54Z
----------------------------------------------------------------

Added links to wikipedia.

review-notebook-app · 2024-09-25T03:08:49Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:48Z
----------------------------------------------------------------

There are lots of UserWarnings in the notebook. Consider filtering them out.

review-notebook-app · 2024-09-25T03:08:49Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:49Z
----------------------------------------------------------------

The last line of the equation set contains a typo -- should be \psi_n rather than \psi_3

NathanielF commented on 2024-09-25T11:39:29Z
----------------------------------------------------------------

Fixed. Thanks, good catch!

review-notebook-app · 2024-09-25T03:08:50Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:50Z
----------------------------------------------------------------

Line #29.        chol, _, _ = pm.LKJCholeskyCov("chol_cov", n=2, eta=2, sd_dist=sd_dist, compute_corr=True)

Why do you set compute_corr=True and discard the correlations?

NathanielF commented on 2024-09-25T11:40:37Z
----------------------------------------------------------------

It's not used here in this model, but useful to have computed for analysing the correlations between the constructs post model fit. I demonstrate this below for the full measurement model....

review-notebook-app · 2024-09-25T03:08:51Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:50Z
----------------------------------------------------------------

Cool plots!

NathanielF commented on 2024-09-25T11:45:43Z
----------------------------------------------------------------

Thanks!

review-notebook-app · 2024-09-25T03:08:52Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:51Z
----------------------------------------------------------------

Line #81.            draws=10000,

Why so many draws? Bad autocorrelation?

NathanielF commented on 2024-09-25T11:42:40Z
----------------------------------------------------------------

Yes, it seemed to help generally boost the ESS. The model samples pretty fast so it was easy enough to just boost the samples.

review-notebook-app · 2024-09-25T03:08:53Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:52Z
----------------------------------------------------------------

typo: reflected

NathanielF commented on 2024-09-25T11:42:50Z
----------------------------------------------------------------

Fixed.

review-notebook-app · 2024-09-25T03:08:53Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:53Z
----------------------------------------------------------------

typo in first sentence: individual

NathanielF commented on 2024-09-25T11:42:59Z
----------------------------------------------------------------

Fixed.

review-notebook-app · 2024-09-25T03:08:54Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:53Z
----------------------------------------------------------------

Typo: each

NathanielF commented on 2024-09-25T11:43:11Z
----------------------------------------------------------------

Fixed

review-notebook-app · 2024-09-25T03:08:55Z

View / edit / reply to this conversation on ReviewNB

fonnesbeck commented on 2024-09-25T03:08:54Z
----------------------------------------------------------------

What about measures of relative fit among the models? What do the LOO/WAIC values look like, and are they helpful in selecting the appropriate level of complexity?

NathanielF commented on 2024-09-25T11:45:26Z
----------------------------------------------------------------

Added the global fit comparison for the SEM and full measurement models. The LOO metrics are quite close, but the full measurement model "wins" on this score. I added a note to say we need to consider the value of the trade-off between expressive power of the SEMs and the questions we can answer in the context of simple global fit comparisons.

Signed-off-by: Nathaniel <[email protected]>

NathanielF · 2024-09-25T11:38:09Z

Good idea.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:38:43Z

changed.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:38:55Z

Added links to wikipedia.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:39:30Z

Fixed. Thanks, good catch!

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:40:38Z

It's not used here in this model, but useful to have computed for analysing the correlations between the constructs post model fit. I demonstrate this below for the full measurement model....

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:42:41Z

Yes, it seemed to help generally boost the ESS. The model samples pretty fast so it was easy enough to just boost the samples.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:42:51Z

Fixed.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:43:01Z

Fixed.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:43:12Z

Fixed

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:45:27Z

Added the global fit comparison for the SEM and full measurement models. The LOO metrics are quite close, but the full measurement model "wins" on this score. I added a note to say we need to consider the value of the trade-off between expressive power of the SEMs and the questions we can answer in the context of simple global fit comparisons.

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:45:44Z

Thanks!

View entire conversation on ReviewNB

NathanielF · 2024-09-25T11:48:31Z

Thanks @fonnesbeck I think i've addressed most of the points you raised. But let me know if you think I should add anything else.

Note I also re-parameterised the tau terms in the final SEM model which seemed to fix the divergences i was getting.

fonnesbeck

LGTM!

NathanielF · 2024-09-25T14:26:42Z

Sweet!

NathanielF · 2024-09-25T17:27:26Z

Think you'll have to merge that for me @fonnesbeck, i don't have that power.

NathanielF · 2024-09-25T17:43:04Z

Woo! Thanks @fonnesbeck

* First commit CFA and SEM Signed-off-by: Nathaniel <[email protected]> * test pre-commit hooks Signed-off-by: Nathaniel <[email protected]> * updated for pre-commit Signed-off-by: Nathaniel <[email protected]> * updated git index Signed-off-by: Nathaniel <[email protected]> * update with full run Signed-off-by: Nathaniel <[email protected]> * add some text write up and a pair plot Signed-off-by: Nathaniel <[email protected]> * update with full run and text Signed-off-by: Nathaniel <[email protected]> * Run with explained variance calculations Signed-off-by: Nathaniel <[email protected]> * Add sem structure section Signed-off-by: Nathaniel <[email protected]> * full run with lavaan syntax noted Signed-off-by: Nathaniel <[email protected]> * update with draft conclusion Signed-off-by: Nathaniel <[email protected]> * tigher writing Signed-off-by: Nathaniel <[email protected]> * tidying Signed-off-by: Nathaniel <[email protected]> * emphasis on exchangeability Signed-off-by: Nathaniel <[email protected]> * update plot sizes Signed-off-by: Nathaniel <[email protected]> * add pearl reference Signed-off-by: Nathaniel <[email protected]> * updating with Chris's comments and better parameterisation Signed-off-by: Nathaniel <[email protected]> --------- Signed-off-by: Nathaniel <[email protected]>

First commit CFA and SEM

ba79db8

Signed-off-by: Nathaniel <[email protected]>

NathanielF added 15 commits September 1, 2024 23:30

test pre-commit hooks

194a6f5

Signed-off-by: Nathaniel <[email protected]>

updated for pre-commit

8083c9c

Signed-off-by: Nathaniel <[email protected]>

updated git index

5cb1c42

Signed-off-by: Nathaniel <[email protected]>

update with full run

6f6d880

Signed-off-by: Nathaniel <[email protected]>

add some text write up and a pair plot

e9a5beb

Signed-off-by: Nathaniel <[email protected]>

update with full run and text

d15639a

Signed-off-by: Nathaniel <[email protected]>

Run with explained variance calculations

47da13d

Signed-off-by: Nathaniel <[email protected]>

Add sem structure section

62d788b

Signed-off-by: Nathaniel <[email protected]>

full run with lavaan syntax noted

64799e9

Signed-off-by: Nathaniel <[email protected]>

update with draft conclusion

a69f76d

Signed-off-by: Nathaniel <[email protected]>

tigher writing

aeda1be

Signed-off-by: Nathaniel <[email protected]>

tidying

198c814

Signed-off-by: Nathaniel <[email protected]>

emphasis on exchangeability

8ec7863

Signed-off-by: Nathaniel <[email protected]>

update plot sizes

eb4ed98

Signed-off-by: Nathaniel <[email protected]>

add pearl reference

2247d19

Signed-off-by: Nathaniel <[email protected]>

NathanielF marked this pull request as ready for review September 8, 2024 16:21

NathanielF requested review from cluhmann, drbenvincent and AlexAndorra September 8, 2024 16:22

updating with Chris's comments and better parameterisation

a0f13b1

Signed-off-by: Nathaniel <[email protected]>

fonnesbeck self-requested a review September 25, 2024 13:40

fonnesbeck approved these changes Sep 25, 2024

View reviewed changes

fonnesbeck merged commit ba7a936 into pymc-devs:main Sep 25, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CFA and SEM with PyMC #698

CFA and SEM with PyMC #698

NathanielF commented Sep 1, 2024 •

edited

Loading

review-notebook-app bot commented Sep 1, 2024

NathanielF commented Sep 8, 2024

NathanielF commented Sep 16, 2024

drbenvincent commented Sep 17, 2024

NathanielF commented Sep 17, 2024

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

fonnesbeck left a comment

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

CFA and SEM with PyMC #698

CFA and SEM with PyMC #698

Conversation

NathanielF commented Sep 1, 2024 • edited Loading

CFA and SEM with PyMC

Helpful links

review-notebook-app bot commented Sep 1, 2024

NathanielF commented Sep 8, 2024

NathanielF commented Sep 16, 2024

drbenvincent commented Sep 17, 2024

NathanielF commented Sep 17, 2024

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

review-notebook-app bot commented Sep 25, 2024 • edited Loading

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

fonnesbeck left a comment

Choose a reason for hiding this comment

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 25, 2024

NathanielF commented Sep 1, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading

review-notebook-app bot commented Sep 25, 2024 •

edited

Loading