index.Rmd

---
title: "Groundhog"
author:
  - name: "Patrick Hajjar"
    affiliation: "Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)"
  - name: "Maximilian Held"
    affiliation: "Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)"
  - name: "Ngoc Tuyet Nhung"
    affiliation: "Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)"
  - name: "Eleni Sarri"
    affiliation: "Friedrich-Alexander Universität Erlangen-Nürnberg (FAU)"
  - name: "Valeria Scollo"
    affiliation: "L'Università Di Torino On Line"
date: "`r format(Sys.time(), '%d %B, %Y')`"
bibliography: library.bib
editor_options: 
  chunk_output_type: console
---

<div class="jumbotron" style="color:white; background: linear-gradient( rgba(0, 0, 0, 0), rgba(0, 0, 0, 0.7) ), url(img/murray.jpg) no-repeat center center fixed; -webkit-background-size: cover; -moz-background-size: cover; -o-background-size: cover; background-size: cover;">
  <h2>Imagine you are in a time loop: What would you do?</h1>
  <p>A Q study on a thought experiment.</p>
  <p>
    <span class="label label-info">
      #GroundhogDay #rstats #QMethodology
    </span>
  </p>
  <p><small><sub>
    Image Credit: Groundhog Day (Columbia Pictures 1993), via [IMDB](https://www.imdb.com/title/tt0107048/)
  </sub></small></p>
</div>

```{r, child="README.md"}
```

```{r setup, include = FALSE, cache = FALSE}
library(readr)
library(magrittr)
library(fs)
library(purrr)
library(pensieve)
library(qmethod)
knitr::opts_chunk$set(
  cache = TRUE, 
  autodep = TRUE, 
  message = FALSE, 
  echo = FALSE,
  out.width = "100%")
```


---

## Introduction

*Valeria Scollo*

Q-Methodology represents an innovative research method used  in social sciences in order to study people’s point of view, that is to say, their subjectivity. It was introduced by psychologist William Stephenson (1902 – 1989)  and, even though the method is described mainly as qualitative, it offers a combination of both qualitative and quantitative aspects, through the analysis of factors emerged during the study.

> Most typically in Q, a person is presented with a set of statements about some topic, and is asked to rank-order them (usually from "agree" to "disagree"), an operation referred to as Q sorting. 
> The statements are matters of opinion only (not fact), and the fact that the Q sorter is ranking the statements from his or her own point of view is what brings subjectivity into the picture.
> [@brown-1993-a]


### Our Story

We are a group of five people: Eleni Sarri, Ngoc Tuyet Nhun, Patrick Hajjar, Valeria Scollo and our teacher and leader Maximilian Held. 

As the title says, this project starts with the question „Imagine you are in a time loop: what would you do?“ and it aims to study individual’s subjectivity accordingly. 

To begin with, our group proposed different topics that could represent the possible theme of the study:
however, some of them involved arguments such as death, catastrophes and, generally speaking, events that could dishearten the participants.
Coming up with a satisfactory research question proved to be quite challenging:
there are many different topics through which it is possible to study people’s point of view and opinions on everyday life matters of our society like, for example, racism, feminism, money (…). 
Since they mostly depends on personal background, culture and values, moral and ethical elements play an important role in the process of choosing the possible subject matter and, being the study of individuals’ point of view the goal of this project, to respect and to pay attention to their sensitiveness becomes a crucial priority for researchers.  
Through brainstorming sessions and debates, we developed a -- maybe unusual -- topic:
the so called “Groundhog” scenario.
In this context, Q-Methodology becomes the best instrument that we have to study and to show  how people would react to an imaginary time loop that no one else is aware of and, to some extent, to understand their perceptions and opinions on a similar note.

There is obviously no right or wrong way to provide individuals’ point of view,

> Only subjective opinions are at issue in Q, and although they are typically unprovable, they can nevertheless be shown to have structure and form, and it is the task of Q-technique to make the form manifest for purposes of observation and study
> [@Brown-1996].



## Methods

*Ngoc Tuyet Nhung*

### Item Generation: Selection and Source of items   

As Stephenson suggests in his publication ‘Q methodology and the projective techniques’, a Q set “may be composed of objects, statements, descriptions and behavior, traits, and the like” [-@Stephenson1952, 223], hence there are many options of what can be used as an item. 
In our study we decided to select items that verbalize actions and therefore *behavior regarding our conceptualized scenario.*
In regard to this definition, we conceived items ourselves based on our own ingenuity with inspiration from the movie “The Groundhog Day”, researches on the internet platform Reddit on topics that seem alike, and asking friends and family.  
Whereas the “Groundhog Day” was our first source of inspiration, as it pictures quiet the similar mind game, Reddit was our main internet source, as it is a universally known and used website to find actions or reaction to certain sequences of events, whether fictional or non-fictional. 
The vast and various community ensures good items in term of *coverage and balance*, moreover it provides suggestion to other similar topics, that helped us forming the definite research question and plot.  
Gathering additive ideas among acquaintances firstly confirmed items, we had already gathered, which were supposed to be broadly representative of the opinion of the general population [@Watts2012, 58] and secondly also had the aim to get a feeling for whether our repository of items covered enough ground within our concept or adding different ideas and propositions would be necessary to generate a balanced set of statements that allows participants of all persuasions to express their feelings equally [@BakerConstructingStatementSets2017, 170f.]. 
  
  
### Demographics and Selection of Participants  

Giving the nature of Q method, representing an inversion of the traditional R methodology, in which the Q set, therefore the items, constitutes the study sample and each participant becomes a variable, the recruitment should always be done carefully regarding the research question and whether the viewpoint of these people matters in relation to the subject [@Watts2012, 70f.]. 
However, in our case study, we were not regimented by our topic, as it was universally appealing and we did not have any concepts of a ”particularly interesting or pivotal point of view” (ibid.).
We had 36 participants, which were selected from 5 different set of acquaintances via opportunity sampling, while we still tried to frame the sampling as assorted as possible. 
Most of the people of the participant group were students from university or school, however from different disciplines.
The age of the participants ranges from 55 to 15, with an average age of 25,2 and a median of 22.5.
Almost the same number of females and males were asked, from which 16 stated to be engaged or in a relationship in the time of participating in the study.


### Experiences in the field  
  
Doing this study as a first encounter with Q Method, there were various things to take into consideration and be mindful of, especially while supervising the procedure of the sorting that only later occurs to one as important and useful, while interpreting the factors, e.g. certain question that should have been asked or things that should have been put an emphasize on while instructing the participants. 
Details, one cannot think of, when not having enough practical experiences.
Nevertheless, elaborated prework, as pre-sorting personal information that might influence the participants’ viewpoints  might ease the path for a substantial interpretation [@Watts2012, 75f.].
During the pre-sorts of the items into the three piles (*“Items you disagree with”*, *“Items you do not understand or are ambivalent about”*, *“Items you agree with”*), the neutral pile was reportedly the biggest, with the remainder equally split, which might suggest, that our items were balanced.
Nonetheless, as some respondents noted that some items were similar, especially on the sex related questions, it should be taken into consideration, whether the balance of the items, could be dependent on the age of the participants, because especially older and younger participants, noted that there were too many sex-related questions.
Even if that was the case, it is still not ensured, whether there were in fact top many questions regarding sexuality or it was more of an act of social desirability for participant to comment on that, as it might defuse the awkwardness of sorting these items under observation. 
Even though many people already complained about the number of items (our number of 59 items were in the average of the recommended number by Stephenson), most of the participants were, nonetheless, highly interested in the method itself, because by ranking the items and actually placing the cards, they were actively more involved, than for example in an one-to-one interview. 
This can be explained by what Professor O.J. Harvey from the University of Colorado describes as “one of psychology’s most basic and well established principles” [-@HarveyBeliefsknowledgemeaning1997, 146 f.], which Watts and Stenner describe as our desire to see structure within, and to ascribe meaning to almost every stimuli and event [@Watts2012, 66]. 
Furthermore, the participants were eager to explain their choices, while this can be, again, affiliated to social desirability it also could be ingénue interest on the topic, showing that our study is entitled.  
Regarding the correlation between members of a certain peer groups, even though they seem to agree on many items as expected, they do not necessarily have the same motivation behind it.
One might say: Just because some people placed the same item on the same ranking, it does not mean they have the same motivation in doing so and agree with each other on a moral code.  
In contrast to that, partners in a relationship, however, seem also to have similarities concerning non-sex- or relationship-related items in their sorting, even though relationship status, reportedly, forms the sorting concerning sex-related items such as: cheating, marrying, sexual fantasies, asking someone to have sex. 


```{r distro, include = FALSE}
distro <- qmethod::make.distribution(nstat = 59, max.bin = 7)
names(distro) <- -7:7
# TODO there is a bug here; this should work without specifiyng chessboard, null should default fine
grid <- pensieve::as_psGrid(obj = distro, pattern = "chessboard")
```

```{r, eval=FALSE, include=FALSE}
items <- readr::read_csv(file = "https://docs.google.com/spreadsheets/d/e/2PACX-1vSkT1N44FpPcqAcAp1eChsHI0jlw-F0gqB7w1Sl4ksd6q3vNAoBbkjrx0rjlYc_7hhJiZ0uHvmxKRon/pub?gid=0&single=true&output=csv")
readr::write_rds(x = items, path = "items.rds")
```

```{r items, include = FALSE}
raw_items <- readr::read_rds(path = "items.rds")
items <- matrix(
  data = as.matrix(raw_items[,c("english", "german")]),
  nrow = nrow(raw_items),
  dimnames = list(items = raw_items$shorthandle, languages = c("english", "ngerman"))
)
# write out hashes and df
hash_de <- sapply(X = items[, "ngerman"], FUN = function(x) {
  digest::digest(object = x, algo = "crc32", serialize = FALSE)
})
hash_en <- sapply(X = items[, "english"], FUN = function(x) {
  digest::digest(object = x, algo = "crc32", serialize = FALSE)
})
input_helper_df <- data.frame(
  full_en = items[,"english"],
  full_de = items[,"ngerman"],
  handle = rownames(items),
  hash_en = hash_en,
  hash_de = hash_de
)
readr::write_csv(x = input_helper_df, path = "input_helper.csv")
```

```{r plot-items, eval=FALSE, include=FALSE}
library(qmethod)
qmethod::make.cards(
  q.set = items,
  study.language = "english", 
  output.pdf = TRUE, 
  babel.language = "english", 
  show.handles = FALSE, 
  duplex.double = TRUE, 
  wording.font.size = "\\large",
  file.name = "english")
qmethod::make.cards(
  q.set = items,
  study.language = "ngerman", 
  output.pdf = TRUE, 
  babel.language = "ngerman", 
  show.handles = FALSE, 
  duplex.double = TRUE, 
  wording.font.size = "\\large",
  file.name = "german")
```


### Field Work



Participants were given the following prompt:

> Imagine that, starting now, the next week will repeat itself once.
> In a week, the whole universe will rewind to today, all changes are undone and memories erased.
> You are the only person who knows this, and you alone will remember what happened during this week.
> What would you do?
>
> Rank the following items according to your likelihood for enacting them in the next week.
> Rank all activities highly which you like, even if you could not complete them all in the given time.
>
> Remember that whatever you do in the next seven days would be completely undone at the end of the week; only your memories remain.


Or, in German:

> Stelle dir vor, dass sich die nächste Woche genau ein Mal wiederholt.
> Am Ende der Woche dreht sich die Zeit zurück auf den heutigen Tag. Alle Veränderungen sind revidiert und Erinnerungen gelöscht.
> Du bist die einzige Person, die über diese Wiederholung weiß und nur du kannst dich in Zukunft daran erinnern, was in der Woche passiert ist.
>
> Sortiere die folgenden Aktivitäten danach, wie wahrscheinlich du sie in der Woche durchführen würdest.
> Es ist nicht wichtig, ob du die präferierten Aktivitäten auch alle schaffen könntest.
>
> Denke daran, dass alle Änderungen im Universum am Ende der Woche rückgängig gemacht werden; nur deine Erinnerungen bleiben.


Participants were given the following sorting instructions:

> 1. Choose a pseudonym, so that you can recognize your own datapoints in the results.
    ![](img/pseudo.jpg)
> 1. Read through all the items carefully. 
     Make notes on the items if you like.
> 2. Preliminarily sort the items into three piles:
     1. Items you like.
     2. Items you do not understand or are ambivalent about.
     3. Items you dislike.
     ![](img/presort.jpg)
> 3. Rank-order the items according to how likely you are to do the described activities, from the right (likely to do) to the left (unlikely to do).
      Start with items from the right (likely to do), continue with items from the left (unlikely to do), and finally fill in the middle (ambivalent).
> 4. Consider your completed sort, and revise item positions if necessary.
      ![](img/sort.jpg)
> 5. Add comments to items you feel strongly about, or do not understand.
> 6. Turn all items on their back (only showing a random string), and ask the investigator to take a picture of your completed sort.
    ![](img/codes.jpg)


Participants were reminded:

> - There is no correct or consistent way to complete your Q sort; whatever you feel expresses your subjective view on the matter is correct. 
>   This is not a test - There is no "right answer".
> - Only the horizontal dimension (from left to right) matters; the vertical axis does not matter.
  Items on top of one another merely express ties.
> If you are ambivalent about an item or do not understand it, place it in the middle between the two extremes.
  This is valuable information, too.
> - You must place all items.
> - You must adhere to the distribution.
> - Only *relative* positions matter.
    For example, you may feel positively about *more* than half of the items, and have only very few neutral or negative items.


Important notes for **the investigators**:

- Remember to sample a *diverse* group of people who are likely to display *different* viewpoints on the subject matter.
- We are not going to gather *any* personal data.
  Make sure not to pollute the gathered data with any information that might deanonymize your participants.
  For example *do not* record their email adress on the cards, and make sure that their pseudonyms are properly anonymous.
  If you want to email the results to the participants, store their contact information *separately* and securely, with no reference to the study.
- **Make sure to test the photo you're taking at the end to record the sort**: 
    - Are the item codes legible? 
    - Is the lighting ok?
    - Is the resolution high enough?
    - Is the pseudonym card in the picture?
    - Save several copies of these images.
    - (If possible, delete date and location metadata from these images).
- There may be a tradeoff in our study between a) getting deep, qualitative insights from observing and discussing the sorting behavior and b) avoiding social desirability biases.
  Participants may not feel free to sort some items (e.g. anti-social and sexual), when they are closely observed, or know that the investigator (= you), will "know" their Q sort.
  (For this reason, the item backs are marked with arbitrary codes for identification, so that participants never have to "show" their sorts to the investigators.)
  It's not clear what we should do here, because on the other hand, we probably cannot produce proper anonymity in in-person sorts, and we may also want to closely observe and engage the participants.
- Take the time to carefully discuss the sorting procedure with participants after they are done.
  If you want to (see above caveat), also sit next to them while they sort and prompt them to explain their choices.
  **Take notes!**
- Remind participants to add feedback or any thoughts and comments to the items.
  They can write them on the cards.
  **Provide participants with pens or pencils.**
- When you are preparing the cards, make sure that every participant has a complete set (and no duplicates). 
  This can be quite difficult and requires *very* careful preparation (envelopes are useful).
- Explain the procedure very carefully to the participants, and **show them pictures of the completed sorts**.
  Otherwise, it can be quite hard to understand the methodology.
  In particular, remind them that only the horizontal axis matters, what the extremes are, and that they must adhere to the forced distribution.
- This procedure takes time and some quiet patience (> 60 minutes), as well as some space (e.g. a free table).
  Plan accordingly.


## Analysis

*Eleni Sarri*

### Data

- Max gathered **2** sorts. [^ownsort]
- Patrick gathered **8** sorts. [^ownsort]
- Eleni gathered **8** sorts. [^ownsort]
- Valeria gathered **9** sorts.
- Maria gathered **6** sorts.

[^ownsort]: This includes one sort completed by the author himself.



```{r data-import, include=FALSE, eval=FALSE}
library(googlesheets)
# this is the sheet
groundhog_entry <- gs_title(x = "groundhog-entry", verbose = TRUE)
gs_read(ss = groundhog_entry, ws = "Item Feedback", range = "C1:R60", col_names = TRUE) %>% 
  write_csv(path = "rawdat/item_feedback.csv", append = FALSE, col_names = TRUE)
all_ws <- gs_ws_ls(groundhog_entry)
parts <- all_ws[!(all_ws %in% c("input_helper", "input-template", "Item Feedback"))]
walk(.x = parts, .f = function(x) {
  gs_read(ss = groundhog_entry, ws = x, range = c("A3:O9"), col_names = as.character(-7:7), na = c("#N/A"), col_types = cols(.default = "c")) %>% 
    write_csv(path = fs::path("rawdat", "sorts", x, ext = "csv"), append = FALSE, col_names = TRUE)
})
```

```{r data-cleaning, include = FALSE}
lookup <- readr::read_csv(file = "input_helper.csv")
rootsorts <- fs::path("rawdat", "sorts")
filenames <- fs::dir_ls(rootsorts, recursive = FALSE, all = FALSE, type = "file", glob = "*.csv")
persons <- fs::path_file(filenames)
persons <- fs::path_ext_remove(persons)
rawdat <- purrr::map(.x = persons, .f = function(x) {
  path <- fs::path(rootsorts, x, ext = "csv")
  thissort <- readr::read_csv(file = path, col_names = TRUE)
  thissort <- as.matrix(thissort)
  thissort <- apply(X = thissort, MARGIN = c(1,2), FUN = function(x) {
    if (is.na(x)) {
      return(NA)
    } else {
      keyloc <- which(lookup == "12123", arr.ind = TRUE)
      if (length(keyloc) == 0) {
        keyloc <- which(
          x = stringr::str_detect(string = as.matrix(lookup), pattern = x),
          arr.ind = TRUE
        )
        keyloc <- arrayInd(ind = keyloc, .dim = dim(x = as.matrix(lookup)))
      }
      lookup$handle[keyloc[1,1]]
    }
  })
  # TODO hackfix
  attr(thissort, "pattern") <- "chessboard"
  class(thissort) <- c("matrix")
  thissort <- pensieve::import_psSort(x = thissort, grid = grid, lookup = lookup)
  thissort <- tidyr::gather(tibble::as_tibble(thissort), na.rm = TRUE)
  return(thissort)
})
names(rawdat) <- make.names(persons)
library(dplyr)
qdat <- purrr::reduce(.x = rawdat, .f = left_join, by = "value")
item_order <- qdat$value
qdat$value <- NULL
qdat <- as.matrix(qdat)
rownames(qdat) <- item_order
colnames(qdat) <- make.names(persons)
qdat[,] <- as.integer(qdat)
storage.mode(x = qdat) <- "integer"
```

In this section, we delve into the statistical procedure of our Q-study.
Even though Q-Methodology resembles other quantitative research methods it's actually quite different.

> This rationale exploits  Q methodology?s rather original ability to capture and tell the whole story.
> The same argument can also be used to separate Q methodology from almost all other methods with a first-person and/or qualitative focus.
> Where most methods of these types concentrate on the dissection of a viewpoint or subject matter into its pertinent sub-themes or issues, Q methodology allows the whole -- and the relationships between themes -- to be seen and appreciated.
> In short, the holism of Q methodology is one of its most powerful selling points. 
> [@Watts2012,pp. 177]. 

Three statistical procedures are used in a Q-study:
Correlation, factor analysis and the computation of the factor scores. 
Matrices are the key to a Q-statistical analysis.
So the first step is to form a table with the gathered Q-sortings of the 36 participants and the 59 different Items.

$M = Items x People$.
R59 x 36 


### Correlations

One of the first matrices in Q-Methodolgy shows where each item was placed by each participant in their Q-sortings.
It is a rank of table containing the Q-sorters and the Q-items.
In Q-Methodology the extremes hold the most importance, that's why we have fewer cards in the extremes and generally, a pyramid-shaped ranking is preferred. 

People who have worked on a Q-research know that most of the matrices in a Q-study are correlation matrices.
Q-Methodology is a method who focuses mostly on the correlation and the factoring of people.
So the consequent step is to transform the initial data matrix into a new matrix.

$M = People x People$.  
Meaning a tabular form of $R59 x 59$. 

In Q-methodology people are correlated with each other instead of variables.
The individual sortings are correlated to reveal the similarities, as well as the differences, of the viewpoints on the studied topic.
For the calculation of the correlation, we used Pearson's correlation coefficient formula.

(@Pearson) $$\rho_{X,Y}=\frac{\operatorname{E}[(X-\mu_X)(Y-\mu_Y)]}{\sigma_X\sigma_Y}$$

With the tabular form, its easy to quickly have an overview on the relationship between the participants and their sortings.
The correlation between any two traits can be determined according to the expression rA,B=(??zAzB)/n, where n is the number of persons in the sample.
Each participant?s sorting is compared and calculated with each other?s.
The individual sortings are correlated to reveal the similarities between the viewpoints.

> The higher the correlation in a positive direction, the more similar the two Qsorts;
> the higher the correlation in a negative direction, the more the relationship is an inverse one.
> [@Brown1980, pp. 267]

Focusing on the lower triangle from the correlation matrix the highest correlation in the groundhog study is a $r=0.8$.
Appearing just once between the participants "Bambi" and "Leverrier".
A perfect positive correlation is +1.00. 
In this case, these two participants indicate a high level of agreement.
As seen in the matrix most of the participants don?t really correlate with each other. 
Negative correlations are a rarity in this Q-study, with an $r= -0.3$.
Pearson's r coefficient, being the highest negative number, meaning the Q-sorters didn't disagree on a great level with each other.
The conclusion from the correlation analysis indicates there is no significant correlation between our Q-sorters and their sorting in a groundhog situation.

```{r correlations, fig.height=11, fig.width=11, fig.cap="Pearsons Correlation Coefficient"}
cors <- cor(x = qdat, method = "pearson")
q.corrplot(corr.matrix = cors)
```

## Factor analysis

Following the correlation analysis comes the third step in a Q-study, and probably the key point in Q-methodology. 
The factor analysis.

> But it is rarely the case that the correlation matrix is of much interest since attention is usually on the factors to which the correlations lead:
> the correlation matrix is simply a necessary way station and a condition through which the data must pass on the way to revealing their factor structure.
> What this involves is the subject of the next section.
> [@brown-1993-a, pp. 110)

Significant clusters of correlations exist, which then can be factorized and described as common viewpoints, and individuals can be put into a particular factor.

> Factor analysis, in general, is a method for classifying variables.
> [@Brown1980, pp.208]

In this case it classifies people. 

With the factor analysis, the sortings of each participant are compared with each other and put in a group together, forming a factor, which represents them.
An ideal type.

> In Q-methodology, subjects of personality study are presented with a sample of statements bearing on their personalities;
> those who describe themselves in similar ways load on the same factor;
> classes of variates (factors) are initially presumed to reflect different types of personality.
> [@Brown1980, pp. 96]

The people in a factor are highly intercorrelated with each other, but they have no correlation with people in other factors.

The objective of a factor analysis is to identify a group of respondents who loaded high on the same factor, showcasing that they rank-ordered the items in a similar way. 
To do this the correlation matrix of all Q-sorts is computed. 
Within this matrix, groups of respondents are identified with mutually high correlation coefficients.
Finally, for each identified group, a composite rank-ordering of the statements is computed, which is called a factor, based on the distribution of the statements by the individual respondents defining that factor and their correlation coefficient with the factor as weight. 
This idealized Q-sort of a factor represents the way in which a person loading 100% on that factor would have ranked the 59 statements.

The uniqueness of Q-methodology is that the "scientist", thus we, has the power to control the results.
Meaning, how many factors to retain or which rotation to use -- quartimax rotation, varimax procedure or maybe a rotation by hand. 
It leaves an openness and indeterminacy to consider which option is the most appropriate and informative. [@Watts2012, pp. 80]


### Factor Retention

Using Principal component analysis through the statistical program software as a factor extraction method -- the analysis reduces the data to a few summarizing factors -- plus the scree plot, as well as the parallel analysis, suggested 3 factors to be extracted.

It should be mentioned that "just because" a software did most of the work in the analytical part does not mean that we were unimportant spectators/bystanders.
We, the researchers, had to make the important decisions in this analytical procedure.

In our case, we were uncertain how many factors to extract. 
We contemplated for a long duration how many factors we wanted to be extracted and which rotation was the most practical, if the third factor would be essential in our study or if we should just continue with two factors. 
The three factors correlated weakly with each other.
The first two factors had a great Eigenvalue. 
The Eigenvalue of a factor shows how much of the variance it can explain.
A factors Eigenvalue is calculated by summing the squared loadings of all the Q-sorts of that factor.
Factors need to have an Eigenvalue greater than 1.00.
Meaning that a factor can explain their own value 100%.
The third factor had a greater Eigenvalue than the average, but it was weak. 
We discussed the advantage and disadvantages of each option, in the end, a three factor solution was deemed adequate.
The "adequacy" of course differs from study to study, in our case, we chose with the criterion that three factors would be better and would make the factor interpretation more interesting and diverse.
Factor 1 currently accounts for 24% of the common variance present in the study, factor 2 12% and factor 3 7%.
A high factor EV and variance is considered good, anything in the region 35-40% and above is satisfactory.
The three factors together explain 43% of the variance.
So it is deemed adequate (Table "factor characteristics").
55% of the variance is lost.


```{r paran, include=FALSE}
qparan <- qmethod::q.nfactors(dataset = qdat, iterations = 10000, cutoff = 8, cor.method = "spearman", siglevel = 0.5, quietly = TRUE)
```

```{r screeplot, fig.cap="Scree Plot with Parallel Analysis", fig.width=9}
qparan$screeplot
```

```{r eigenvals}
knitr::kable(x = qparan$eigenvalues, caption = "Eigenvalues from Data and Simulations")
```


### Factor Extraction and Quartimax Rotation

Next step is the rotation.
We used the quartimax rotation. 
Varimax rotation didn't give us Ideal types, but rather variables. 
The varimax rotation was the most mathematically informative solution, whereas the quartimax rotation offered more of a "groupish" character.
Both solutions were examined in subsequent analyses before deciding to use the quartimax rotation for the final solution. 

> Most Q- methodologists don't think that the best mathematical solution is necessarily also the most meaningful or most informative.
> [@Watts2012,pp. 99) 

Plus the quartimax rotated factors maximized the amount of variance explained by the extracted factors. 
We really wanted our participants to be only "one color", to load high on one factor mostly, to have a high factor score on a single factor. 
Therefore the quartimax rotation was our best solution.

```{r extraction, include = FALSE}
quartimax <- qmethod::qmethod(dataset = qdat, nfactors = 3, rotation = "quartimax", forced = TRUE, cor.method = "spearman", reorder = FALSE, threshold = "none", allow.confounded = TRUE)
```

```{r chars}
knitr::kable(quartimax$f_char$characteristics, caption = "Factor Characteristics")
```

```{r factor-cors}
knitr::kable(quartimax$f_char$cor_zsc, caption = "Interfactor Correlations")
```

```{r loadings}
ploas <- pensieve::QLoas(loas = as.matrix(quartimax$loa))
plot(x = ploas, by = "people", use_js = FALSE)
```

```{r r2}
plot(x = ploas, by = "people", use_js = FALSE, r2 = TRUE)
```

In the next section, the three extracted factors are interpreted.


## Interpretation

*Patrick Hajjar* (with contributions from all other authors)

### Factor 1: Lawful Good 

```{r f1, warning = FALSE, fig.width=11, fig.cap="Binned Scores for Factor 1", eval=FALSE}
q.scoreplot.ord(results = quartimax, factor = 1, incl.qdc = FALSE, quietly = TRUE)
```

In this segment, Factor 1, which we dubbed **Lawful Good**, will be interpreted.
Factor 1 has the highest eigenvalue and thus represents more participants than Factor 2 and 3 combined.
In accord with @Watts2012, the extreme loadings will be looked at first.
The high standard deviation, regarding rank +7 and -7, catches the eye.
Participants who exemplify Factor 1 could not agree on the two poles.
The highest loading could be found for "lottery" at rank +7 and "killing" at rank -7.

These two poles alone, don't really allow any sort of thorough interpretation,
but when one is looking at the whole sort, a picture starts to emerge:
Factor 1 placed items of monetary gain, withouth engaging in any sort of active/violent crimes, and of consequence to the real timeline, very high.
They placed the items "lottery", "insider trading" and "expensive travel" all on pole positions within the sort.
Only the item "rob bank" is placed on the negative side of the ranking.
This supports the interpretation that Factor 1 is rather moral and legal abiding. 

They are focused on self-improvement like "me time", "improve myself" but at the same time want to experience new things like "restaurants" and "sexual fantasies"

Another interesting observation could be made about the items "true opinion", "eating", "forbidden places" and "confess".
All these items have in common, that they are a mild sort of rebellion against social conventions.
The key factor is the word "mild".
If they truly wanted to forego social or moral conventions they would place items like "destruction" or "burn state" high, but they focused on interpersonal acts like "true opinion" or "confess".
This assumption is especially confirmed by participants through statements in regard to the item "true opinion".
A few participants commented, when prompted why they placed it so high, that they wouldnt tell people their true opinions about them in the "real timeline" not out of fear for repurcussions but because they wouldn't want to hurt peoples feelings.

When looked at the negative-ranking site of the sort another interesting pattern emerges.
All of the items which have any sort of connection to harming someone else, or oneself, are placed low.
Factor 1 is not curious about the afterlife, or anything death related.
Factor in general is more focused on themselves then on others.
They don't want to indulge in any form of violence, be it vandalism ("grand theft auto", "burn state", "graffiti") or physical.
They also all placed the item "kidnapping" very low and if prompted participants were very aggravated about the wording of the item.

Even though they placed "insider trading" high, Factor 1 in general is averse to crime.
It is to be presumed that the general populace is not aware that mere insider trading is a crime by law.
Factor 1 placed it high because of the possibility to transfer the monetary gain outside the timeloop and to change the real timeline.

At the same time they placed the item "sexual fantasies" and "ask sex" high, while placing "cheating" low.
This could indicate that they are either single, sexually frustrated in regards to their relationship or that they placed "cheating" low because each participant knew his/her instructor and a form of social censoring took place.

The standard deviation of item placement is lowest in the neutral ranking spectrum.
This indicates a very similar sorting by all the Factor 1 assigned participants.
In accord with the above mentioned instructions, participants were told that the items place in the center are items they don't care about or they don't understand. 

Factor 1 placed the item "pretendsoelse" universally on the zero ranking.
This could indicate that Factor 1 feels generally good about themselves.
As mentioned before Factor 1 is focused on self-improvement, but to improve oneself, one first has to know who they are and what they are able to improve.

Factor 1 is rather critical and conservative.
This is supported by the placement of "fortune teller", "embarrassing" and "karaoke". 
"Karaoke" and "embarrassing" could both be placed on the same rank, because the notion of karaoke is in Germany a rather excentric experience that is connotated with a high degree of embarrassment. 
"extreme_sports" could be placed on rank 0 because the thrill of extreme sports results through putting oneself in danger and the possibility of death.
As mentioned before, Factor 1 is not very fond of the concept of death.
The higher placement of "extreme_sports" in regard to the other death-related items is probably because with extreme sports death is only a possibility, while commiting suicide is guaranteed to result in death.

The item "quitq" was generally met with confusion and thus placed on the -1 rank.

Factor 1 is a generally moral and law abiding person, that is deeply rooted in its social environment.
They want to rebel against the estabslishment, but only in a social-sanctioned frame.
They focus on financial stability following the groundhog event.


### Factor 2: True neutral

```{r f2, warning = FALSE, fig.width=11, fig.cap="Binned Scores for Factor 2", eval=FALSE}
q.scoreplot.ord(results = quartimax, factor = 2, incl.qdc = FALSE, quietly = TRUE)
```

In the following segment, Factor 2 will be interpretated.
As with Factor 1, the interpretation will be based on the mentioned examples by Watts and Stenner.

At first sight, the standard deviation across the whole sort is significantly lower for Factor 2 then for Factor 1.
Factor 2 and also Factor 3 had fewer participants whose Q-Sort exemplified the specific factors and thus a lower deviation is to be expected.

First I am going to take a look at the two poles.
Factor 2 also placed "lottery" and "insidertrading" as their highest positive rankings but placed "theusual" and "scared" on their respective negative rankings. 

At first look, Factor may seem averse to morals and social conventions but after taking a closer this presumption is proved wrong.

Even though they are interested in monetary gain, which would affect the real timeline and thus their real life, Factor 2 is also highly curious.
When looked at single items their behavior seems irrational and destructive, but when looking at the greater picture there also seems to be a pattern for Factor 2.

They ranked "killing", "drugs", "afterlife", "fakedeath" and "extremesports" all on the same rank (+3).
When looking at rank 2, for example "forbiddenplaces", there seems to emerge a yearning for new experiences.
Factor 2 is extremely curious.
They want to experience otherwise unexperiencable or rather socially reprehensible things. 
They also aren't as focused on improving themselves as Factor 1. 

Factor 2 wants to found a religion, rather than join an existing one.
The interpretation that Factor 2 is not bent on pure disruptive behavior is based on the items "makehappy" (+1) and "makebadday" (+2). 
When directly compared, these items seem paradoxical to each other, but when looked at with a filter of curiosity, they work together.

Factor 2 wants to kill because they want to know what it is like.
They want to kill themselves, so they can experience what the afterlife is like. 
It isn't about the violent act itself, but rather the knowledge hidden behind it.
This assumption is also supported by "accident" at rank 2.

The negative rankings of the sort give even more ground to this hypothesis.

They ranked items like "party", "sleep" and "marry" negative, because they are mundane.
These are all achievable things and one doesn't need a timeloop to experience them.
They seek the thrill.

This whole sort is driven by the aforemented thrill.
The adrenaline rush that is achieved by doing activites that are looked down upon by society. 

The item "actnormally" at rank -5 supports this narative. 

The neutral part of the ranking is more diversed then the two respective poles. 

Just like Factor 1, Factor 2 also placed "quitq" on the neutral spectrum.
This can also be a sign that the item is misphrased or rather extremely out of the blue and thus incomprehensible. 

Another interesting correlation can be found between "destruction" (0) and "burnstate" (+1). 
Factor 2 is indifferent towards pure destruction for the sake of destroying things, but if there is an anarchical purpose behind it they rank it higher. 

Just like with Factor 1, they placed the items "sexualfantasies" and "asksex" positive but "cheating" on the negativ side. 
This can also be due to them being single or rather because of social censoring due to knowing the instructor. 

Factor 2 seems to see the timeloop as chance rather than a threat and wants to make the most out of it. 
They pursue knowledge for the sake of knowledge and act under the mantra of 
"the end justifies the means"
They want to break free but in the certainty that everything will be reversed without them having to face the consequences of their actions. 


### Factor 3: Chaotic Neutral

```{r f3, warning = FALSE, fig.width=11, fig.cap="Binned Scores for Factor 3", eval=FALSE}
q.scoreplot.ord(results = quartimax, factor = 3, incl.qdc = FALSE, quietly = TRUE)
```

Where as Factor 1 and 2 seemed to hide a pattern, Factor 3 is just pure chaos. 
Like Factor 2, the standard deviation is extremely low, which is also explained by a lower number of participants that exemplified the specific factors representing Factor 3. 
It is even lower than Factor 2.

On the respective poles they placed "cheating" (+7) and "blinddeaf" (-7)

They placed all the items with a sexual notion relatively high.
This implies there was no real social census, like with Factor 1 and 2 and they are either single or probably not in a sexually fulfilling relationship.

Another interesting observation could be made about their aversion towards violence.
They would commit non-violent crimes (+2) but abhor killing (-4) or destruction (-5). 
They are also not interested in vandalism of any sort which is supported by "burnstate" (-2) and graffitti (-2).

There is a bit of a paradox as they placed "avoidpeople" at rank +3 but at the same time placed "trueopinion" (+6) and confess (+3) quite high. 

Factor 3 is not as curious as Factor 2 and not as moral-abiding as Factor 1. 

Their negative rankings imply that they would not indulge in criminal activities that could physically harm another person and are rather focused on random arbitrary acts. 

In regard to the items placed on the neutral spectrum of the ranking, they are confident in their identity, they are indifferent about joining a cult (0) or pretending to be someone else (0). 

Just like the other two Factors, they placed "quitq" at a neutral ranking (0), which can also be ascribed to the nature of the item itself.

Factor 3 would rather make someones day horrible (+1) then wonderful (-4) and is also not interested in preventing a possible accident,
which is ironic because they placed "unprotectedsex" at +1.

Another interesting notion, in comparison with Factor 1 and Factor 2, is the missing focus on monetary gain outside of the time loop.
Factor 3 placed "lottery" at -3 and "insidertrading" at +2.
At the same time Factor 3 doesn't want to pay their bills (+3). 

If one was to devise a pattern for Factor 3 it would be highly focused on the respective persons whim.
Factor 3 is not as focused on monetary gain, but rather on sexual freedom.
They approve of random actions (+5), highly approve of sleeping (+5) and at the same time don't really know what they want.
Factor 3 embodies impulsiveness.

## Conclusion

### Valeria Scollo

Q-methodology represents an innovative research approach to study attitudes of the stakeholders, however it is still not widely used in all disciplines. 

It obviously presents some advantages and disadvantages. 
For example, one of the main limitations is that the Q sorting is quite time-consuming and it needs to be carefully explained to participants, since they are not familiar with it and sometimes they could not exactly understand how to proceed: 
validity can be compromised by the misinterpretation of some items. 

Furthermore, Q methodology is limited only to small samples, therefore results can't be generalised to the rest of the population. 
At the same time, anybody can take part to a project on Q Methodology: 
although some participants may not have a specific idea about the topic, they can still share their perspective just from reading the statements and sorting them.

Compared to other approaches, Q-Method could be considered more accurate: it proposes a question and its respective answers, which are "already decided" by researchers. 
Whereas asking the same question without any given instruction -for example, during an interview- could let the stakeholders swerve from the question line,  Q Methodology leads people toward different paths, which are, however, limited according to the cards and it permits to study subjects' specific point of view accordingly, without deviating from the original goal. 

Finally, lots of individuals that took part to the project defined this new approach as "fun" and "amusing": 
overall, Q-Methodology gains its interesting role in the sociology research field, being not only an innovative and functional new way to study subjectivity for researcher, but also an enjoyable method for its public.


### Patrick Hajjar

When I enlisted for this seminar I was unabled to imagine what Q-Methodology really meant.
I put it aside as a standard qualitative methodology and at first couldn't quite grasp what it is about.
This conclusion was mainly based on the missing knowledge towards the statistical notions working in the background of a q-sort.
In hindsight, I do have to say that my first impression was wrong.

The approach of confrontating participants with cards instead of multiple pieces of paper with written question was really different and interesting.
All my participants were sceptical beforehand but after engaging in a q-sort started to really get into it.
They may have not talked as much as i would have liked, but the statements they gave were insightful.
People didn't just place the items so they could get it over with,
but were really thoughtful about the respective rankings and often times switched the cards around. 

I really liked the engagement of instructor and participant and when looking at the collected data,
I do have to agree with Brown on Stephensons Statement "A methodology is not merely a technique but a profound way of approaching nature".

The Factors could have been a little more diverse for my taste and specifically Factor 3 isn't genuinely representative.
Especially in comparison to the other two factors.

All in all, it was a thought-provoking experience and distinctive way to approach subjective behavior in a methodological way.


### Eleni Sarri

If I had to describe Q-Methodology in one word I would call it a roller coaster ride. 
True that's not one word, but its impossible to characterize it with a single word.
At least concerning me.
My experience with Q-Method started first with a quick research on the internet, which -- I thought -- gave me a slight taste of what I had to expect from our seminar.
A qualitative research method to study peoples "subjectivity" on different matters used in psychology and political sciences. 
Short and simple.
It woke my interest further to explore this method and with a good feeling I registered for the seminar.

On the first day of the class, I started having my doubts.
A feeling similar to panic started spreading. 
I was bombarded with so much new information, with so many new things I had to learn and comprehend first, which made me doubt my initial decision of attending the class.
Fortunately, my panic started to slowly dissolve after being given the opportunity to complete a Q-sort myself.
I could see the whole picture of what Q-Method is.  
After starting our own Q-study and working together on finding a topic, as well as writing the statements made me realize that Q-Method isn't as different as other qualitative methods. 
The difference lies in the Q-sorting.
Probably the most thrilling and intriguing part, I dare say for everyone.
Observing the chosen people complete the Q-sort is an entertaining and informative experience.
Learning the reasons and listening to their arguments on why they placed the Q-items in that specific order, as well as seeing the similarities and differences of what the participants prioritize in a groundhog situation, including the interpretation of the extracted factors made it a unique experience. 

Subsequently, the fun part ended and the dreadful part began. 
The roller coaster train came to the final and steepest drop. 
The statistical analysis.
I and statistics have a hate-love relationship. 
I put time and effort into trying to understand it, but in the end, I'm left confused and frustrated.
On the plus side, we didn't have to manually calculate anything thanks to the Q-software. 
However, I chose to write the statistical analysis segment. 
The truth is with this choice I wanted to challenge myself and "face my fears".
Interestingly enough, I enjoyed writing it and I can actually say that I learned many things from all this process.
I just hope that in the end, it won't backfire on me.

That was my "emotional" roller coaster with  Q-methodology.
I experienced its ups and downs, the anticipating parts and the dreadful moments.  
In the end, all this process gave me so much enjoyment and satisfaction that I definitely want to experience it again, but be better prepared and less fearful.  


## References