Chapter 5 MiniProject TWO (Task-switching)

In the second mini-project, you will read, summarize and discuss the paper by Stoet et al. (2013). Then, you will attempt to replicate their results in the lab, by conducting an experiment and analyzing the data.

5.1 What’s in store

  1. Students read paper and write QALMRI (15-20)

  2. Group discussion about paper (15-20)

  3. Students download the task-switching program available from the website and individually complete the task. Individual students then enter their data in the master spreadsheet

  4. Discussion of how to analyze the data. Major analysis goals are:

    1. Was there a mixing cost? Compare pure lists to mixed lists

    2. Was there a switching cost? Compare switch vs. repeat trials in pure lists

    3. Did these effects depend on gender? Did the women have a smaller mixing cost than the men? Did the women have a smaller switching cost than the men?

  5. Students break into groups to analyze the data.

5.2 Critical Concepts

5.2.1 Multiple Dependent and Independent Variables

In lab 5 we begin looking at more complex designs that involve multiple dependent or and independent variables. For example, we can look at the dependent variable of reaction time or error rate, which are our two difference measured variables. There are also three independent variables, including gender (male vs female), block (pure list vs. mixed list), and trial sequence (repeat trial vs. switch trial).

5.2.2 Main effects and interactions

Throughout this course we will be talking a lot about main effects and interactions. Feel free to jump ahead in the textbook and learn more about them.

Main effects refer to the influence of single independent variables. And, each independent variable always has one main effect. So, we could look at the main effect of gender, block, or trial sequence. The main effect is like conducting a one-way ANOVA on the variable of interest, collapsing over the other variables. For example, a main effect of trial sequence usually shows slower mean reaction times in the switch condition compared to the repeat condition. This is called the task-switching cost, or also the task-switching effect.

Interactions occur when the effect of one independent variable depends on the levels of another independent variable. For example, if women actually multi-task better than men, then we should expect an interaction between gender and task-switching. Specifically, women should be better at task-switching than men. This would mean that women should have smaller task-switching costs (meaning they are unaffected by switching) compared to men (who should have bigger task-switching costs if they are more affected by switching).

If women have smaller task switching costs than men, then the effect of task-switching independent variable (repeat vs. switch) will depend on the level of the gender variable (women vs. men), which is the definition of an interaction.

5.2.3 Manipulating an Effect

Things get complicated when we add more dependent and independent variables, and it can be easy to lose track of what is going on in an experiment.

Many experiments are conducted for the purpose of better understanding some phenomena. For example, researchers observe phenomena like the task-switching effect, they create theories to explain the phenomena, then they test their theories by running more experiments.

A primary question is often to identify factors that manipulate, influence, or somehow change the phenomena of interest. We know that switching between tasks can hurt performance, and we measure this cost by calculating a switch cost, which is the difference in performance between switching conditions and repeating conditions. What kinds of factors would make this cost ? Perhaps extended practice, time of day, gender, personality, motivation, drug interventions, details of the task, and many other factors could make the normal cost get smaller. Many of these factors might also make the effect get larger. And, knowledge of what makes switching costs smaller or larger can be used to test theories, which need to explain how and why those factors cause switching costs to change.

The Stoet et al. (2013) took this approach to determine if switch costs are smaller for men than women. Notice we have been talking about switch costs, which is the phenonemon and effect of interest. We have not been talking too much about the dependent variable of reaction times that we use to compute the switch costs. Remember that switch costs are computed as RT switch - RT repeat.

5.2.4 Reducing a two-factor design to a single-factor design using difference scores

We can use the term switch cost as a convenient term to describe the effect of the switching variable on reaction times. We can also think of the switch cost as the dependent variable of interest, rather than the base reaction time scores. Thinking of the switch cost this way can change how you conceptualize the design.

For example, how many independent variables are in a design testing whether women have smaller switch costs than men? The answer could be two or one, depending on how you think of the design.

If we go with two, then the design involves the independent variables gender (women vs. men) and trial sequence (switch vs. repeat), and the dependent variable of reaction time. This design would have two main effects (effect of gender, and effect of switching), and one interaction (gender x switching). We would be mainly interested in the interaction. For example, women may have a smaller difference in mean reaction time between the switch and repeat conditions than men.

We can reduce the design to a simple single-factor design. To do so, we create a new dependent variable called switch costs. For example, for each subject we compute the mean reaction in the switch and repeat condition. Then we calculate the switch costs (difference between switch and repeat) for each subject. Now, each subject has one switch cost, rather than two reaction times. Now, the design would have gender as the single independent variable, and switch costs as the dependent variable. To determine whether women have smaller costs than men would you could run a t-test comparing the switch-costs between conditions.

In addition to this material below, you can also look at the examples of conducting factorial ANOVAs in software using the lab manual from PSYC 3400 Introductory Statistics, and refresh your understanding of factorial ANOVAs by reading the Factorial ANOVA chapter from the PSYC 3400 textbook.

5.3 Data-analysis tips

To give you an idea about the analyses you will be performing, we will again create simulated data to mimic aspects of the experiment, and then go through the steps of performing the analysis.

We will simulate data for the switching cost. Specifically, we will imagine that women have a smaller switching cost than men. The code below generates sample data for 10 women and 10 men, who each have mean reactions in the repeat and switch conditions. For women, the mean reaction times were 580 ms for repeat and 600 ms for switch sequences, for a total expected switch cost of 20 ms. For men, the mean reaction times were 580 ms for repeat and 650 ms for switch sequences, for a total expected switch cost of 70 ms.

5.3.1 Simulating the data

women_switch <-round(rnorm(10,600,20))
women_repeat <-round(rnorm(10,580,20))
men_switch <-round(rnorm(10,650,20))
men_repeat <-round(rnorm(10,580,20))
all_data<-data.frame(Subject=c(rep(seq(1,10,1),2),
                               rep(seq(11,20,1),2)),
                     Gender=rep(c("Female","Male"),each=20),
                     Sequence=rep(rep(c("switch","repeat"),each=10),2),
                     RT=c(women_switch,women_repeat,
                          men_switch,men_repeat))

knitr::kable(all_data)
Subject Gender Sequence RT
1 Female switch 587
2 Female switch 582
3 Female switch 574
4 Female switch 601
5 Female switch 626
6 Female switch 576
7 Female switch 587
8 Female switch 592
9 Female switch 596
10 Female switch 577
1 Female repeat 596
2 Female repeat 574
3 Female repeat 531
4 Female repeat 596
5 Female repeat 542
6 Female repeat 604
7 Female repeat 569
8 Female repeat 565
9 Female repeat 602
10 Female repeat 556
11 Male switch 639
12 Male switch 689
13 Male switch 635
14 Male switch 655
15 Male switch 681
16 Male switch 654
17 Male switch 674
18 Male switch 658
19 Male switch 630
20 Male switch 645
11 Male repeat 553
12 Male repeat 564
13 Male repeat 573
14 Male repeat 567
15 Male repeat 561
16 Male repeat 580
17 Male repeat 581
18 Male repeat 563
19 Male repeat 619
20 Male repeat 544

5.3.2 Plotting the data

We can plot the data at least two ways. See the bar and line graphs below. Note that the x-axis changes between graphs.

library(ggplot2)
library(plyr)
sde<-function(x){sd(x)/length(x)}
plot_means<-ddply(all_data,.(Gender,Sequence),summarise,
                       MeanRT=mean(RT),
                       SE=sde(RT))

limits <- aes(ymax = MeanRT + SE, ymin = MeanRT - SE)

ggplot(plot_means,aes(x=Gender, y=MeanRT, group=Sequence,fill=Sequence))+
  geom_bar(position="dodge",stat="identity")+
  geom_errorbar(limits, width=.3,position=position_dodge(.9))+
  theme_classic(base_size=12)+
  coord_cartesian(ylim=c(500,700))+
  ylab("Mean RT")+
  xlab("Trial sequence")

ggplot(plot_means,aes(x=Sequence, y=MeanRT, group=Gender,shape=Gender))+
  geom_line()+
  geom_point()+
  geom_errorbar(limits, width=.2)+
  theme_classic(base_size=12)+
  ylab("Mean RT")+
  xlab("Trial sequence")

The graphs show that the switch costs (difference between repeat and switch trials) is smaller for women and men. Which is good, because we are simulating the data with this outcome in mind.

5.3.3 Running the ANOVA

The next step is to conduct an ANOVA. This design has two factors or independent variables, gender and trial sequence. The gender variable is between-subjects, and the trial sequence variable is within subjects. So, we will run a 2 (Gender: Female vs. Male) x 2 (trial sequence: Repeat vs. Switch) mixed design ANOVA with Gender as the betwen-subjects factor, and trial sequence as the within-subjects factor.

library(broom)
library(xtable)
all_data$Subject<-as.factor(all_data$Subject)
aov.out<-aov(RT~Gender*Sequence+Error(Subject/Sequence),all_data)
aov_summary<-summary(aov.out)
knitr::kable(xtable(aov_summary))
Df Sum Sq Mean Sq F value Pr(>F)
Gender 1 9985.6 9985.6000 28.08757 0.0000488
Residuals 18 6399.3 355.5167 NA NA
Sequence 1 25908.1 25908.1000 51.49976 0.0000011
Gender:Sequence 1 11971.6 11971.6000 23.79698 0.0001209
Residuals 18 9055.3 503.0722 NA NA
mt<-model.tables(aov.out,"means")
mt
## Tables of means
## Grand mean
##        
## 597.45 
## 
##  Gender 
## Gender
## Female   Male 
##  581.6  613.2 
## 
##  Sequence 
## Sequence
## repeat switch 
##  572.0  622.9 
## 
##  Gender:Sequence 
##         Sequence
## Gender   repeat switch
##   Female 573.5  589.8 
##   Male   570.5  656.0

The above shows the ANOVA table and the means for the main effects and interaction. We also conduct t.test comparisons to look at the switch costs separately for men and women.

FemaleT<-t.test(RT~Sequence,all_data[all_data$Gender=="Female",],paired=TRUE,var.equal=TRUE)
MaleT<-t.test(RT~Sequence,all_data[all_data$Gender=="Male",],paired=TRUE,var.equal=TRUE)
FemaleT
## 
##  Paired t-test
## 
## data:  RT by Sequence
## t = -1.6548, df = 9, p-value = 0.1323
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -38.582344   5.982344
## sample estimates:
## mean of the differences 
##                   -16.3
MaleT
## 
##  Paired t-test
## 
## data:  RT by Sequence
## t = -8.3757, df = 9, p-value = 1.531e-05
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -108.59235  -62.40765
## sample estimates:
## mean of the differences 
##                   -85.5

5.4 Writing up the results

The next step is to interpret the results and write them up. Here is an example write-up.

The mean reaction times for each subject in trial sequence condition were submitted to a 2 (Gender: Female vs. Male) x 2 (trial sequence: Repeat vs. Switch) mixed design ANOVA with Gender as the betwen-subjects factor, and trial sequence as the within-subjects factor. Mean reaction times in each condition collapsed across subjects are displayed in Figure 1.

The main effect of gender was significant, F(1, 18) = 28.09, MSE = 355.52, p < 0. Women (582 ms) had faster mean reaction times than men (613 ms).

The main effect of trials sequence was significant, F(1, 18) = 51.5, MSE = 503.07, p < 0. Repeat trials (572) had faster mean reaction times than switch trials (623).

Most important was the significant two-way interaction between gender and trial sequence, F(1, 18) = 23.8, MSE = 503.07, p < 0. We interpreted the interaction further by conducting the following comparisons. Women showed a significant switch cost, t(9) = -1.65, p = 0.132, with faster mean reaction times for repeat (574) than switch (590) trials. Men also showed a significant switch cost, t(9) = -8.38, p = 0, with faster mean reaction times for repeat (570) than switch trials (656). The presence of an interaction indicates that the size of the switch cost for women was significantly smaller than the size of the switch cost for men.

References

Stoet, Gijsbert, Daryl B. O’Connor, Mark Conner, and Keith R. Laws. 2013. “Are Women Better Than Men at Multi-Tasking?” BMC Psychology 1 (1): 18.