Author: Dave Kleinschmidt

Going full Bayesian with mixed effects regression models

Posted on Updated on

Thanks to some recently developed tools, it’s becoming very convenient to do full Bayesian inference for generalized linear mixed-effects models. First, Andrew Gelman et al. have developed Stan, a general-purpose sampler (like BUGS/JAGS) with a nice R interface which samples from models with correlated parameters much more efficiently than BUGS/JAGS. Second, Richard McElreath has written glmer2stan, an R package that essentially provides a drop-in replacement for the lmer command that runs Stan on a generalized linear mixed-effects model specified with a lme4-style model formula.

This means that, in many cases, you simply simply replace calls to (g)lmer() with calls to glmer2stan():

library(glmer2stan)
library(lme4)
lmer.fit <- glmer(accuracy ~ (1|item) + (1+condition|subject) + condition, 
                  data=data, family='binomial')
summary(lmer.fit)
library(glmer2stan)
library(rstan)
stan.fit <- glmer2stan(accuracy ~ (1|item) + (1+condition|subject) + condition, 
                       data=data, family='binomial')
stanmer(stan.fit)

There’s the added benefit that you get a sample from the full, joint posterior distribution of the model parameters

Read on for more about the advantage of this approach and how to use it.

Using plyr to get intimate with your data

Posted on Updated on

I gave a short tutorial [pdf slides] at the LSA summer institute on one of my favorite R packages: plyr (another brilliant Hadley Wickham creation). This package provides a set of very nice and semantically clean functions for exploring and manipulating data. The basic process that these functions carry out is to split data up in some way, do something to each piece, and then combine the results from each piece back together again.

One of the most common tasks that I use this for is to do some analysis to data from each subject in an experiment, and collect the results in a data frame. For instance, to calculate the mean and variance of each subject’s reaction time, you could use:

ddply(my.data, "subject.number", function(d) {
  return(data.frame(mean.RT=mean(d$RT), var.RT=mean(d$RT)))
})

Plyr also provides a whole host of convenience functions. For instance, you could accomplish the same thing using a one-liner:

ddply(my.data, "subject.number", summarise, mean.RT=mean(RT), var.RT=var(RT))

There are lots more examples (as well as more background on functional programming in general and the other use cases for plyr) in the slides [pdf] (knitr source is here, too).

Running phonetic (adaptation) experiments online

Posted on Updated on

I’ve developed some JavaScript code that somewhat simplifies running experiments online (over, e.g., Amazon’s Mechanical Turk). There’s a working demo, and you can download or fork the source code to tinker with yourself. The code for the core functionality which controls stimulus display, response collection, etc. is also available in its own repository if you just want to build around that.

If you notice a bug, or have a feature request, open an issue on the issue tracker (preferred), or comment here with questions and ideas. And, of course, if you want to contribute, please go ahead and submit a pull request. Everything’s written in HTML, CSS, and JavaScript (+JQuery) and aims to be as extensible as possible. Happy hacking!

If you find this code useful for your purposes, please refer others to this page. If you’d like to cite something to acknowledge this code or your own code based on this code, the following is the paper in which we first used this paradigm:

  1. Kleinschmidt, D. F., and Jaeger, T. F. 2012. A continuum of phonetic adaptation: Evaluating an incremental belief-updating model of recalibration and selective adaptation. Proceedings of the 34th Annual Meeting of the Cognitive Science Society (CogSci12), 605-610. Austin, TX: Cognitive Science Society.

A more detailed journal paper is currently under review. If you’re interested, subscribe to this post and get the update when we post the paper here once it’s out (or contact me if you can’t wait).

Knit from the command line

Posted on

Knitr is a great way to combine document markup (Latex, Markdown, HTML, etc.) with R code for data analysis and visualization. It pulls out the chunks of R code, runs them, and re-inserts the results into the document source (usually a .tex file), which can then be compiled as usual. Normally you would call it from an R console (or use something like RStudio), but what if you want to call it from the command line, like latex?  Here’s a little shell script that I use to automate the knitting of .Rnw files (combining R and Latex): knit.sh.

It call knit() inside R, then runs pdflatex on the resulting file. It is very simple to use (you must of course have the knitr package installed in R):
knit.sh awesomefile.Rnw

This would produce awesomefile.pdf (as well as the intermediate file awesomefile.tex, and the extracted R commands, awesomefile.R). You might even rename the script as knit and put it somewhere on your search path (maybe /usr/local/bin/) to be even more fancy.

Collect keyboard responses asynchronously in Javascript

Posted on Updated on

Some of our Mechanical Turk experiments are written in straight-up javascript, which gives you a lot of control and flexibility but at the expense of having to write some pretty basic functionality from scratch.  I recently was in a situation where I wanted to collect separate keyboard responses in different but possibly overlapping time windows: stimuli are coming in fast and on some of them, the subject needs to press the spacebar.  Rather than fix my design so that the response windows would never overlap, I decided to write a function that would collect a one-off keyboard response, asynchronously, meaning that other experiment control code can run behind it. Read the rest of this entry »