Interpretation of RSA results

[max-kat]

Dear RSA community,

I am new to RSA and have some questions regarding how to do and also interpret certain aspects of a 'typical' RSA analysis and would appreciate it if you could guide me in the right direction :)

Study idea:
We have a 2 (perception v imagery) x 2 (art v faces) x 2 (high v low experience) and we want to know i) if high experiences across perception and imagery are similar and ii) how the data is best described using a weighted model

1. I have the data (unstandardized beta-maps) of 30 participants and their 8 conditions (unfortunately noise-normalization is not possible as not all participants have more than 1 run due to specifics of the design)
2. Reading https://www.diedrichsenlab.org/pubs/Walther_Neuroimage_2016.pdf I think we should use correlation as we are interested in the shape and not the strength of activations. Or is there something I am missing here to consider?
   rdms = rsr.calc_rdm(data_all, descriptor='conds', method = 'correlation')
3. I created 3 categorical model rdms (perception/face/high). As far as I understand: a dissimilarity of 0 indicates perfect similarity --> so the models should have 0 in those condition pairs of the rdm where we suspect similarity and 1 where we do not, right?

questions:
i) Since I want to know if the similarity between conditions is driven by high v low I created all candidate models of the 2x2x2 design.
neuro_results_1 = rsatoolbox.inference.eval_fixed(all3_models, rdms, method='corr')

The result indicates that similarity is neither predicted by high-low nor faceVart but only by percetionVimagery... Also do the slight negative parameters implicate that within e.g. high vs low high are more dissimilar to high than to low?
Can I interpret the eval of 0.752 as Spearman r of the explained similarity indicating that they are extremely similar regardless of other conditions?

ii) weighted models
model = rsatoolbox.model.ModelWeighted('Combined RDMs', rdms_models_cat)
theta_corr_regress = rsatoolbox. model.fit_regress(model, rdms, method='corr')
theta = [0.04746518 0.07163716 0.99630075]

rdm_corr = model.predict_rdm(theta_corr_regress)
The average correlation for the correlation parameters is:
0.7544442710828646
rdm_opt = model.predict_rdm(theta)

So if I understand it correctly theta represents the weighting of the different categorical models, right? And since the weights for the first 2 models are basically zero they do not add to the model fit. but isn't the problem that they add up to more than 1?? so can model weights be > 1? Why is that?
Also, the last picture in which I use these theta weights to create the best theoretical model --> this is the best the overall model can explain right and since this is r=0.75 the similarity is really high for within condition perception and imagery? other explanations add virtually nothing. is that the right interpretation or do I need to rephrase this?

also how can I get an average rdm from the real_data I used so I can display it alongside the best model rdm?

Thanks so much for reading this far!!!
If you can only answer some of my questions, I am already really happy! THANKS :)

Have a great day,
Max

[nkriegeskorte]

Dear Max, Before addressing more technical questions, the key thing we need to understand is your representational hypotheses. It seems that you have 8 (=2^3) conditions. So there is one response pattern for each combination of the three factors, such as "perceived faces or high familiarity" or "imagined art of low familiarity". Please confirm if I understand this correctly. My first question would then be what each of the 8 patterns corresponds to in terms of the subject's experience. Is it an average across many different images (e.g. different individual faces)? In that case, you are not studying how different visual content is represented (which would require each image to be a condition). Instead you are studying whether a given brain region has similar average patterns during perception and imagery of familiar faces (say). My first thought is that if you are essentially asking to what extent each of the three factors is reflected in the patterns, you could consider decoding each factor separately. However, it would be good to better understand your hypotheses and to see the RDMs for your brain regions of interest before considering what analysis choices are best. Niko

…

____________________________________________________ Nikolaus Kriegeskorte, PhD Professor of Psychology and Neuroscience Affiliated member, Department of Electrical Engineering Director of Cognitive Imaging, Zuckerman Mind Brain Behavior Institute Columbia University ***@***.***

On Tue, Mar 12, 2024 at 9:08 AM max-kat ***@***.***> wrote: Dear RSA community, I am new to RSA and have some questions regarding how to do and also interpret certain aspects of a 'typical' RSA analysis and would appreciate it if you could guide me in the right direction :) Study idea: We have a 2 (perception v imagery) x 2 (art v faces) x 2 (high v low experience) and we want to know i) if high experiences across perception and imagery are similar and ii) how the data is best described using a weighted model 1. I have the data (unstandardized beta-maps) of 30 participants and their 8 conditions (unfortunately noise-normalization is not possible as not all participants have more than 1 run due to specifics of the design) 2. Reading https://www.diedrichsenlab.org/pubs/Walther_Neuroimage_2016.pdf I think we should use correlation as we are interested in the shape and not the strength of activations. Or is there something I am missing here to consider? rdms = rsr.calc_rdm(data_all, descriptor='conds', method = 'correlation') 3. I created 3 categorical model rdms (perception/face/high). As far as I understand: a dissimilarity of 0 indicates perfect similarity --> so the models should have 0 in those condition pairs of the rdm where we suspect similarity and 1 where we do not, right? questions: i) Since I want to know if the similarity between conditions is driven by high v low I created all candidate models of the 2x2x2 design. neuro_results_1 = rsatoolbox.inference.eval_fixed(all3_models, rdms, method='corr') image.png (view on web) <https://github.com/rsagroup/rsatoolbox/assets/122026208/5a00ebee-b906-4c02-8bb2-1e7ebb4545a7> image.png (view on web) <https://github.com/rsagroup/rsatoolbox/assets/122026208/b673645b-5876-4e42-9b55-340463081ff1> The result indicates that similarity is neither predicted by high-low nor faceVart but only by percetionVimagery... Can I interpret the eval of 0.752 as Spearman r of the explained similarity indicating that they are extremely similar regardless of other conditions? ii) weighted models model = rsatoolbox.model.ModelWeighted('Combined RDMs', rdms_models_cat) theta_corr_regress = rsatoolbox. model.fit_regress(model, rdms, method='corr') theta = [0.04746518 0.07163716 0.99630075] rdm_corr = model.predict_rdm(theta_corr_regress) The average correlation for the correlation parameters is: 0.7544442710828646 rdm_opt = model.predict_rdm(theta) image.png (view on web) <https://github.com/rsagroup/rsatoolbox/assets/122026208/85c7909a-e7bb-40f7-90f1-732543eeaccc> So if I understand it correctly theta represents the weighting of the different categorical models, right? And since the weights for the first 2 models are basically zero they do not add to the model fit. but isn't the problem that they add up to more than 1?? so can model weights be > 1? Why is that? Also, the last picture in which I use these theta weights to create the best theoretical model --> this is the best the overall model can explain right and since this is r=0.75 the similarity is really high for within condition perception and imagery? other explanations add virtually nothing. is that the right interpretation or do I need to rephrase this? also how can I get an average rdm from the real_data I used so I can display it alongside the best model rdm? Thanks so much for reading this far!!! If you can only answer some of my questions, I am already really happy! THANKS :) Have a great day, Max — Reply to this email directly, view it on GitHub <#390>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALWMWYEVPXVDNFAYXBK3KOTYX344DAVCNFSM6AAAAABESGRQW6VHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZWGM3DEOBQHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[max-kat]

Thank you for the quick reply!

Yes, so participants were presented pictures of faces and artworks and once saw them (perception) and once had to imagine them (imagery). After each presentation (actually perceived or just imagined) they rated their experiences on 3 different scales (beauty, moving and pleasure). I created a k=2 mean clustering to get high and low experiences for each of the 2 modalities (perception v imagery) and 2 types (face v artwork).
Doing this allowed me to create 8 different time regressors to extract beta estimates from the fMRI data using FSL.

Due to the subjective nature of the evoked experiences some beta estimates depict the average of 10+ single trials while for other participants one beta estimate might only have 2 or 3 trials cause they consistently did not have e.g. Low Experience ART IMAGERY.

My main research question is whether neuronal responses between high-moving experiences is similar across modality and type.
Does this answer your question?

[max-kat]

just to clarify: as input I have 8 beta maps for the 8 conditions for each participant. Does that even make sense in your oppinion?

(I did this cause many participants did not have all conditions present in all runs. Also some only had 1 run.)

[max-kat]

Thinking about it a bit more:

Does it in general make sense to perform an RSA not on the individual beta map regressors (8 in my case) but on their contrasts, e.g. high minus low of the given modality and type? I do think this kind of contrasting happens often in univariate analyses to 'distill' which areas are purely activated in the 'more(higher)' condition. Wouldn't it then also make sense to run an RSA on these contrasts ? Thinking about it, I would say this gets rid of all the variance due to perception/imagery & face/art cause you are subtracting beta maps of the same types. Which at least in univariate analyses would give you the pure high v low contrast. Could one similarly do this with RSA and then ask are the high and low experiences across all these modalities and types similar or not?

I have the feeling this however cheats the whole purpose of model weighted RSA a bit cause you artifically get rid of all the variance due to other sources and then try to explain the rest, whereas in the model weighted RSA you try to account for this variance by incorporating different models which should explain the observed data as a whole... or am I mistaken?

[JasperVanDenBosch]

@max-kat how many stimuli do you have in each of the conditions? I think there may be a miscommunication here wrt conditions vs stimuli. Typically you'd have a regressor for each stimulus (or all repetitions of that stimulus in the run)

[max-kat]

That varies for each participant.
To give you an example:
One participant had 4 runs and was shown 10 stimuli (5 faces and 5 artworks) in each condition (imagery and perception) within each run.
They also rated the experience the pictures elicited in them. I used these ratings to categorize them into high and low (with a k-means clustering over all participants)
Thus I have all in all 8 conditions: 2 experiences (high and low) X 2 modalities ( perception & imagery) X 2 types (face and artwork).

This person however, did not rate a single image as HIGH in the perception condition within the first 3 runs. They rated 2 images as HIGH in the 4th run. This means I could at most optain 2 single trial beta maps for this condition for this person. This also means that the person had 38 instances of LOW perception (inverse condition) across the 4 runs. This means the data is heavily unbalanced.

Since this weird distribution of data is due to the subjective nature of elicited experiences, I opted to not use single trial beta maps as input for the RSA as I couldn't use multivariate noise normalization anyway for most particiapnts. Thus I used FSL FEAT to give me LSU beta maps (copes) for the 8 conditions for each participant averaged together over all 4 runs.
Now I use these 8 condition copes as input for the rsa.

Does this make it more or less clear?