This folder contains the code for the evaluation of our user study.
result_cleaning.py takes the anonymized raw results from our survey tool (finishedsurvey_anonymized.csv) and cleans them. In particular, this includes:
- removing artifact data generated by the survey tool.
- checking whether the answers in each task are correct.
- combining the answers for the three sub-tasks for each task.
- filtering participants that did not use any counterfactuals during the first task.
- combining the Explanation Satisfaction answers for each task.
These results are saved in cleaned_data.csv. Furthermore, result_cleaning.py creates interesting_values.csv which only contains the most interesting and aggregated results.
Finally, it creates three dataframes that only contain the main values (corresponding to our main hypotheses) for each pair of conditions (mv_control_olson.csv, mv_control_starGAN.csv, and mv_olson_starGAN.csv). These were used to easily conduct Mann-Whitney-U-Tests with the open source software Jamovi (Version 2.2.5.0). The corresponding Jamovi files files are control-olson.omv, control-starGAN.omv, and olson-starGAN.omv.
demographics.py calculates and prints demographics like the mean age and the percentage of female participants.
main_plots.py generates the main plots for our paper (Main task scores and explanation satisfaction) and saves them in the folder figures.
demographics.py generates plots for the participants demographics and saves them in the folder figures/demographic.
explorative_plots.py generates additional plots we looked at and saves them in figures/explorative.