Quarto as a tool for reproducible research projects in psychology

Some initial reactions to using quarto as a platform for creating and communicating reproducible research projects in psychology.

Last knit: 2023-08-04

For a good few years I have been adopting reproducible workflows for my cognitive psychology research projects. Vuorre and Crump (2021) discussed an R-based approach to creating and sharing research assets in a reproducible manner, and we called it vertical. Since then I have used that approach for most new projects in the lab. However, quarto is the NKOTB, and I’ve been itching to try it for managing assets in a psych research project. And, in the last few days I got the chance to start a project from scratch using quarto, actually several projects. I’m going to digest some of my experiences and lay breadcrumbs for my future self about what I like and am learning so far.

Note

Note: this post is mainly inspired by the fact that I just converted my lab website where this blog is hosted from a pkgdown website to a quarto website. So, I really feel like I need to make a new post to justify the time it took to make the conversion. TLDR: I’m not totally sure what the purpose of the post is yet.

The vertical approach

Here’s an overview of the vertical approach, which uses the R-package template as a starting point for organizing files. This has the benefit of being able to quickly create websites using {pkgdown}. It’s an R-centered approach, which is great for me because I use R for everything. So, if my project includes custom R functions, I can immediately drop them into the R-package template and create an R package for the project. This process can be useful for documenting data from the project too. Additional assets such as experiment script code, posters, manuscripts, etc., are given their own folders. We go into much more depth about vertical in the paper and on the vertical webpage.

Using vertical for my own projects I gravitated toward following pkgdown guidelines for almost everything. For example, many of the assets I was creating were R Markdown documents. I put those in the vignettes folder, which allows pkgdown to render them automatically. Pkgdown is intended to create websites for R packages and not websites in general, so customizing the website to the needs of particular project can be challenging.

What’s most important to me is adopting an organized and convenient method for creating and sharing assets in a reproducible way. This can be done in many languages, and doesn’t need to be R centric. The same goals can also be accomplished without twisting things like pkgdown into positions they weren’t intended for. Enter quarto.

Trying quarto on for size

Quarto is new, still working out some kinks, but in my experience so far, it’s a mean reproducible document creation machine. And, I heart quarto.

Over the summer I dabbled with the new quarto book format, and converted my undergraduate statistics textbook from a bs4_book to quarto book. That process was relatively painless, and I was encouraged to try more quarto.

I often mentor students in the process of learning R for various purposes, and I like to show students how they can make websites and blogs with R Studio, and then use them as a venue to learn R and showcase their work. Blogs and websites were pretty easy before, and they are still very easy with quarto. I had finagled a blog system based on pkgdown rendering vignettes, and after seeing the quarto version I knew I’d make the switch at some point.

Next semester I’m teaching two courses that will both involve students creating blogs and websites. I was leaning toward using quarto, and now even more so. I have been using a pkgdown approach for building course websites (e.g., like this one for introductory cognition), but after seeing some examples of quarto websites for a course (e.g., like data-science in a box), I decided I needed to try that too (e.g., this emerging course website).

This week, just before I decided to redo my lab website in quarto, I also tried using quarto for a research project instead of vertical, and that’s what really sold me.

Quarto for new research

I am not sharing this website yet because the project is still in very early stages of development. However, it has been really fun and productive to use quarto from the very beginning of the project.

At this stage, graduate student Drew Shives and I know our research direction, but we are still doing a lot of reading, combined with stimulus creation, and experiment-idea generation. It’s a lot of creative work that can get messy very fast in terms of keeping track of notes, code, and ideas. This is where I have found using quarto as a personal research organization tool very useful.

Here’s a quick look at the setup, and why I like it. First, I used R Studio to create a new quarto website project. Then, I modified _quarto.yml as follows:

project:
  type: website
  output-dir: ../docs/quarto

website:
  title: "DRMsource"
  navbar:
    left:
      - href: index.qmd
        text: Home
      - about.qmd
      - href: ../index.html
        text: R package
  sidebar:
    - title: "Home"
      style: "floating"
      collapse-level: 3
      align: left
      contents: auto

format:
  html:
    theme: minty
    css: styles.css
    toc: true

editor: visual

In quarto 1.2 (pre-release build), there is a really nice feature for auto-generating navigation in the side bar. For example, in the above yaml, under sidebar (which controls the left-side navigation on the webpage), I set the contents option to auto. As a result, the name of any folder in the project will be converted to a section header, and the title of each .qmd inside a folder will be added to navigation list under each section. Here is the current directory structure of the project:

Right now I’m creating things in three different folders: Aims, Design, and Reading. Any .qmd file inside a folder automatically get’s rendered to the website, and listed under sections. I have a .qmd for developing project motivation in the Aims folder, a bunch of .qmds for stimulus generation in the Design folder, and a couple .qmds to keep track of readings in a readings folder.

This project organization structure may or may not last very long, depending on what happens as we work through the project. Fortunately, it is a snap to change the structure. Just rename the folders, or switch files between folders, and the website automatically updates itself. That’s great.

Less to do with quarto, and more to do with Rstudio’s new and improved visual editor, I’m super excited about how easy it is to add content in rmd or qmd files. Like it I want to add a screen shot, I take one and drag/drop it onto the text in the visual editor, and it automatically gets added to an images folder. That’s fire. And, zotero integration for easy citing while writing, is more fire.

So, hurrah for quarto! And, it’s about to be another Friday night, which means I need to stop writing and hit the render button.

References

Vuorre, Matti, and Matthew J. C. Crump. 2021. “Sharing and Organizing Research Products as R Packages.” Behavior Research Methods 53: 792–802. https://doi.org/https://doi.org/10.3758/s13428-020-01436-x.