Chapter 5 MiniProject TWO (Task-switching)
In the second mini-project, you will read, summarize and discuss the paper by Stoet et al. (2013). Then, you will attempt to replicate their results in the lab, by conducting an experiment and analyzing the data.
5.1 What’s in store
Students read paper and write QALMRI (15-20)
Group discussion about paper (15-20)
Students download the task-switching program available from the website and individually complete the task. Individual students then enter their data in the master spreadsheet
Discussion of how to analyze the data. Major analysis goals are:
Was there a mixing cost? Compare pure lists to mixed lists
Was there a switching cost? Compare switch vs. repeat trials in pure lists
Did these effects depend on gender? Did the women have a smaller mixing cost than the men? Did the women have a smaller switching cost than the men?
Students break into groups to analyze the data.
5.2 Critical Concepts
5.2.1 Multiple Dependent and Independent Variables
In lab 5 we begin looking at more complex designs that involve multiple dependent or and independent variables. For example, we can look at the dependent variable of reaction time or error rate, which are our two difference measured variables. There are also three independent variables, including gender (male vs female), block (pure list vs. mixed list), and trial sequence (repeat trial vs. switch trial).
5.2.2 Main effects and interactions
Throughout this course we will be talking a lot about main effects and interactions. Feel free to jump ahead in the textbook and learn more about them.
Main effects refer to the influence of single independent variables. And, each independent variable always has one main effect. So, we could look at the main effect of gender, block, or trial sequence. The main effect is like conducting a one-way ANOVA on the variable of interest, collapsing over the other variables. For example, a main effect of trial sequence usually shows slower mean reaction times in the switch condition compared to the repeat condition. This is called the task-switching cost, or also the task-switching effect.
Interactions occur when the effect of one independent variable depends on the levels of another independent variable. For example, if women actually multi-task better than men, then we should expect an interaction between gender and task-switching. Specifically, women should be better at task-switching than men. This would mean that women should have smaller task-switching costs (meaning they are unaffected by switching) compared to men (who should have bigger task-switching costs if they are more affected by switching).
If women have smaller task switching costs than men, then the effect of task-switching independent variable (repeat vs. switch) will depend on the level of the gender variable (women vs. men), which is the definition of an interaction.
5.2.3 Manipulating an Effect
Things get complicated when we add more dependent and independent variables, and it can be easy to lose track of what is going on in an experiment.
Many experiments are conducted for the purpose of better understanding some phenomena. For example, researchers observe phenomena like the task-switching effect, they create theories to explain the phenomena, then they test their theories by running more experiments.
A primary question is often to identify factors that manipulate, influence, or somehow change the phenomena of interest. We know that switching between tasks can hurt performance, and we measure this cost by calculating a switch cost, which is the difference in performance between switching conditions and repeating conditions. What kinds of factors would make this cost ? Perhaps extended practice, time of day, gender, personality, motivation, drug interventions, details of the task, and many other factors could make the normal cost get smaller. Many of these factors might also make the effect get larger. And, knowledge of what makes switching costs smaller or larger can be used to test theories, which need to explain how and why those factors cause switching costs to change.
The Stoet et al. (2013) took this approach to determine if switch costs are smaller for men than women. Notice we have been talking about switch costs, which is the phenonemon and effect of interest. We have not been talking too much about the dependent variable of reaction times that we use to compute the switch costs. Remember that switch costs are computed as RT switch - RT repeat.
5.2.4 Reducing a two-factor design to a single-factor design using difference scores
We can use the term switch cost as a convenient term to describe the effect of the switching variable on reaction times. We can also think of the switch cost as the dependent variable of interest, rather than the base reaction time scores. Thinking of the switch cost this way can change how you conceptualize the design.
For example, how many independent variables are in a design testing whether women have smaller switch costs than men? The answer could be two or one, depending on how you think of the design.
If we go with two, then the design involves the independent variables gender (women vs. men) and trial sequence (switch vs. repeat), and the dependent variable of reaction time. This design would have two main effects (effect of gender, and effect of switching), and one interaction (gender x switching). We would be mainly interested in the interaction. For example, women may have a smaller difference in mean reaction time between the switch and repeat conditions than men.
We can reduce the design to a simple single-factor design. To do so, we create a new dependent variable called switch costs. For example, for each subject we compute the mean reaction in the switch and repeat condition. Then we calculate the switch costs (difference between switch and repeat) for each subject. Now, each subject has one switch cost, rather than two reaction times. Now, the design would have gender as the single independent variable, and switch costs as the dependent variable. To determine whether women have smaller costs than men would you could run a t-test comparing the switch-costs between conditions.
In addition to this material below, you can also look at the examples of conducting factorial ANOVAs in software using the lab manual from PSYC 3400 Introductory Statistics, and refresh your understanding of factorial ANOVAs by reading the Factorial ANOVA chapter from the PSYC 3400 textbook.
5.3 Data-analysis tips
To give you an idea about the analyses you will be performing, we will again create simulated data to mimic aspects of the experiment, and then go through the steps of performing the analysis.
We will simulate data for the switching cost. Specifically, we will imagine that women have a smaller switching cost than men. The code below generates sample data for 10 women and 10 men, who each have mean reactions in the repeat and switch conditions. For women, the mean reaction times were 580 ms for repeat and 600 ms for switch sequences, for a total expected switch cost of 20 ms. For men, the mean reaction times were 580 ms for repeat and 650 ms for switch sequences, for a total expected switch cost of 70 ms.
5.3.1 Simulating the data
women_switch <-round(rnorm(10,600,20))
women_repeat <-round(rnorm(10,580,20))
men_switch <-round(rnorm(10,650,20))
men_repeat <-round(rnorm(10,580,20))
all_data<-data.frame(Subject=c(rep(seq(1,10,1),2),
rep(seq(11,20,1),2)),
Gender=rep(c("Female","Male"),each=20),
Sequence=rep(rep(c("switch","repeat"),each=10),2),
RT=c(women_switch,women_repeat,
men_switch,men_repeat))
knitr::kable(all_data)| Subject | Gender | Sequence | RT |
|---|---|---|---|
| 1 | Female | switch | 587 |
| 2 | Female | switch | 582 |
| 3 | Female | switch | 574 |
| 4 | Female | switch | 601 |
| 5 | Female | switch | 626 |
| 6 | Female | switch | 576 |
| 7 | Female | switch | 587 |
| 8 | Female | switch | 592 |
| 9 | Female | switch | 596 |
| 10 | Female | switch | 577 |
| 1 | Female | repeat | 596 |
| 2 | Female | repeat | 574 |
| 3 | Female | repeat | 531 |
| 4 | Female | repeat | 596 |
| 5 | Female | repeat | 542 |
| 6 | Female | repeat | 604 |
| 7 | Female | repeat | 569 |
| 8 | Female | repeat | 565 |
| 9 | Female | repeat | 602 |
| 10 | Female | repeat | 556 |
| 11 | Male | switch | 639 |
| 12 | Male | switch | 689 |
| 13 | Male | switch | 635 |
| 14 | Male | switch | 655 |
| 15 | Male | switch | 681 |
| 16 | Male | switch | 654 |
| 17 | Male | switch | 674 |
| 18 | Male | switch | 658 |
| 19 | Male | switch | 630 |
| 20 | Male | switch | 645 |
| 11 | Male | repeat | 553 |
| 12 | Male | repeat | 564 |
| 13 | Male | repeat | 573 |
| 14 | Male | repeat | 567 |
| 15 | Male | repeat | 561 |
| 16 | Male | repeat | 580 |
| 17 | Male | repeat | 581 |
| 18 | Male | repeat | 563 |
| 19 | Male | repeat | 619 |
| 20 | Male | repeat | 544 |
5.3.2 Plotting the data
We can plot the data at least two ways. See the bar and line graphs below. Note that the x-axis changes between graphs.
library(ggplot2)
library(plyr)
sde<-function(x){sd(x)/length(x)}
plot_means<-ddply(all_data,.(Gender,Sequence),summarise,
MeanRT=mean(RT),
SE=sde(RT))
limits <- aes(ymax = MeanRT + SE, ymin = MeanRT - SE)
ggplot(plot_means,aes(x=Gender, y=MeanRT, group=Sequence,fill=Sequence))+
geom_bar(position="dodge",stat="identity")+
geom_errorbar(limits, width=.3,position=position_dodge(.9))+
theme_classic(base_size=12)+
coord_cartesian(ylim=c(500,700))+
ylab("Mean RT")+
xlab("Trial sequence")
ggplot(plot_means,aes(x=Sequence, y=MeanRT, group=Gender,shape=Gender))+
geom_line()+
geom_point()+
geom_errorbar(limits, width=.2)+
theme_classic(base_size=12)+
ylab("Mean RT")+
xlab("Trial sequence")
The graphs show that the switch costs (difference between repeat and switch trials) is smaller for women and men. Which is good, because we are simulating the data with this outcome in mind.
5.3.3 Running the ANOVA
The next step is to conduct an ANOVA. This design has two factors or independent variables, gender and trial sequence. The gender variable is between-subjects, and the trial sequence variable is within subjects. So, we will run a 2 (Gender: Female vs. Male) x 2 (trial sequence: Repeat vs. Switch) mixed design ANOVA with Gender as the betwen-subjects factor, and trial sequence as the within-subjects factor.
library(broom)
library(xtable)
all_data$Subject<-as.factor(all_data$Subject)
aov.out<-aov(RT~Gender*Sequence+Error(Subject/Sequence),all_data)
aov_summary<-summary(aov.out)
knitr::kable(xtable(aov_summary))| Df | Sum Sq | Mean Sq | F value | Pr(>F) | |
|---|---|---|---|---|---|
| Gender | 1 | 9985.6 | 9985.6000 | 28.08757 | 0.0000488 |
| Residuals | 18 | 6399.3 | 355.5167 | NA | NA |
| Sequence | 1 | 25908.1 | 25908.1000 | 51.49976 | 0.0000011 |
| Gender:Sequence | 1 | 11971.6 | 11971.6000 | 23.79698 | 0.0001209 |
| Residuals | 18 | 9055.3 | 503.0722 | NA | NA |
mt<-model.tables(aov.out,"means")
mt## Tables of means
## Grand mean
##
## 597.45
##
## Gender
## Gender
## Female Male
## 581.6 613.2
##
## Sequence
## Sequence
## repeat switch
## 572.0 622.9
##
## Gender:Sequence
## Sequence
## Gender repeat switch
## Female 573.5 589.8
## Male 570.5 656.0The above shows the ANOVA table and the means for the main effects and interaction. We also conduct t.test comparisons to look at the switch costs separately for men and women.
FemaleT<-t.test(RT~Sequence,all_data[all_data$Gender=="Female",],paired=TRUE,var.equal=TRUE)
MaleT<-t.test(RT~Sequence,all_data[all_data$Gender=="Male",],paired=TRUE,var.equal=TRUE)
FemaleT##
## Paired t-test
##
## data: RT by Sequence
## t = -1.6548, df = 9, p-value = 0.1323
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -38.582344 5.982344
## sample estimates:
## mean of the differences
## -16.3MaleT##
## Paired t-test
##
## data: RT by Sequence
## t = -8.3757, df = 9, p-value = 1.531e-05
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -108.59235 -62.40765
## sample estimates:
## mean of the differences
## -85.55.4 Writing up the results
The next step is to interpret the results and write them up. Here is an example write-up.
The mean reaction times for each subject in trial sequence condition were submitted to a 2 (Gender: Female vs. Male) x 2 (trial sequence: Repeat vs. Switch) mixed design ANOVA with Gender as the betwen-subjects factor, and trial sequence as the within-subjects factor. Mean reaction times in each condition collapsed across subjects are displayed in Figure 1.
The main effect of gender was significant, F(1, 18) = 28.09, MSE = 355.52, p < 0. Women (582 ms) had faster mean reaction times than men (613 ms).
The main effect of trials sequence was significant, F(1, 18) = 51.5, MSE = 503.07, p < 0. Repeat trials (572) had faster mean reaction times than switch trials (623).
Most important was the significant two-way interaction between gender and trial sequence, F(1, 18) = 23.8, MSE = 503.07, p < 0. We interpreted the interaction further by conducting the following comparisons. Women showed a significant switch cost, t(9) = -1.65, p = 0.132, with faster mean reaction times for repeat (574) than switch (590) trials. Men also showed a significant switch cost, t(9) = -8.38, p = 0, with faster mean reaction times for repeat (570) than switch trials (656). The presence of an interaction indicates that the size of the switch cost for women was significantly smaller than the size of the switch cost for men.
References
Stoet, Gijsbert, Daryl B. O’Connor, Mark Conner, and Keith R. Laws. 2013. “Are Women Better Than Men at Multi-Tasking?” BMC Psychology 1 (1): 18.