2009 Workshop on Ordinary and Multilevel Models (WOMM)

We are pleased to announce a workshop on ordinary and multilevel (a.k.a. mixed) model analyses of psycholinguistic data, to be held in conjunction with the 2009 CUNY conference on sentence processing. The workshop is now over. Thanks to all who attended.

The goal is a discussion of quality standards for the field (how should we fit models? when can we trust them? what should we report in papers?). We hope to gather interested researchers who are beginning to use regression and multilevel methods and “experts” who have already used them for some time. Undoubtedly, it will not be the case that there is only one correct statistical method for each problem, but hopefully the discussion will lead to some common ground and increased awareness of what experienced regression users consider “good”. It will be impossible to arrive at standards for all types of models, but the goal is to work towards some general minimum quality standards.  The workshop is structured to allow for plenty of time for discussion following overview presentations.

The target audience for this workshop are researchers that have already used regression models for their research and would like to have a forum to discussion common issues with the use of these models. The presentations will be given by researchers familiar with both psycholinguistic data and regression analyses.

What we will assume and what we will not cover: We will assume that attendees are familiar with the conceptual idea of linear models and have used at least ordinary linear regression before (preferably, also logistic regression). We will assume that attendees are interested in multilevel/mixed/hierarchical modeling, want to use it for data analysis, and have read some work using them or discussing them. While we may comment in passing on certain advantages of these models, it is not the goal of this workshop to “sell” multilevel models. Several well-written motivations for the use of multilevel models of psycholinguistics research are available.  We recommend Baayen at al., 2008 for a discussion of the trade-offs of multilevel linear models compared to traditional approaches. For arguments for the use of multilevel logit models instead of ANOVAs, see Jaeger, 2008 and for a power comparison of such models and alternative approaches, see Dixon, 2008. The goal of this workshop is merely to provide an introduction to these models and a discussion of some common issues with them.

Contributors

• Harald Baayen
• Dale Barr
• Austin Frank
• Florian Jaeger
• Victor Kuperman
• Roger Levy
• Fermin Moscoso del Prado Martin

We are grateful to Jennifer Arnold, Jim Magnuson, Dan Mirman, John Trueswell, and Shravan Vasishth for help with the preparation of this workshop.

Program (with downloadable slides and videos)

Below the talks are linked to the titles of the lectures. Some talks also were videotaped. Talks were videotaped two ways. One way is with the slides directly from the computer with audio, and the other way filmed from the back of the room with audio. The screen doesn’t show well at all when videotaped from the back of the room and the video is highly compressed to keep the download size reasonable. Feel free to ask us for the full versions for teaching purposes. Also, unfortunately, several recordings seem to have failed, so that we are left with only parts of certain talks and no videos of some talks. This was out of our control.

12:50ish     Bus will arrive at CMB
12:58        Brief intro: Goals of the workshop (Video: Slides Audience)
1:00-1:25pm  Conceptual background for multilevel models (Video: Slides Audience)
             (Roger Levy and Fermin Moscoso del Prado Martin)
1:25-1:35pm  Q&A
1:35-2:15pm  Issues and solutions in fitting, evaluating, and interpreting
             regression models (Video: Slides Audience, Note: Video cuts off due to tape running out at workshop)
             (Florian Jaeger and Victor Kuperman)
2:15-2:50pm  Q&A
2:50-3:05pm  Break

3:05-3:40pm  Advanced issues in multilevel model building and evaluation (Video: Slides Audience)
             (Harald Baayen)
3:40-4:00pm  Q&A with entire panel
4:00-4:15pm  Break
4:15-4:30pm  General discussion and open Q&A with entire panel

4:30-5:10pm  Multilevel model analysis of time-series (eye-tracking)
             data analysis (Video: Slides Audience)
             (Dale Barr and Austin Frank)
5:10-5:30pm  Q&A
5:30-5:35pm  Possible future plans (see below) (Video: Audience,  Note: Video cuts off due to tape running out at workshop)

Oh, and note:  R rocks!

Feedback

We will post a short poll here after the workshop so that you can give us feedback and post ideas, remaining questions, and such for future workshops. We would very much appreciate if you can revisit this page after the workshop (we will send out an email) to give us feedback. If you are a visitor of this page, feel free to leave comments about the usefulness or lack thereof of the information provided on this page.

Financial acknowledgments

We are also grateful for financial support and organizational help from

• UC Davis (Matt Traxler and Tamara Swaab)
• Center of Language Science (University of Rochester)
• Institute for Research in Cognitive Science (University of Pennsylvania)
• Center of Mind and Brain, UC Davis (Andy Yonelinas)
• Jennifer Arnold (UNC)
• Thomas Wasow (Stanford University)

Thanks also to Tamara Swaab (Psychology, UC Davis), Noelle Blalock (CMB), and Andrew Watts (HLP lab, University of Rochester) for help with the organization!

Location and time

When: 03/25/09 (the day before CUNY), 1pm – 5:30pm

Where:

UC Davis

CMB Conference Room

267 Cousteau Place (East Davis)

Transportation

By shuttle: Great news! The CMB has generously provided funds for a conference shuttle from MU to the conference site (CMB). Alexis Norausky is going to pick up people in front of Freeborn Hall at 12:30pm and walk you to the Memorial Union (MU) bus station close by where the bus will leave at 12:35pm. We don’t know the bus number yet, but the bus will say “Not service” or “special”. The bus will arrive at CMB (the conference location) at 12:50pm. The bus seats 40 people and should have room for plenty more standing. But if that sounds too croweded to you, you may consider going by car, cab, or the normal buses (see below).

After the workshop, the bus will be available at the CMB at 5:30 to pick up people returning to the MU, and will wait there until 5:45 before bringing people back to the MU (or it can leave sooner once everyone riding the shuttle gets on). Thanks in advance to Alexis for driving and CMB for sponsoring transportation =).

By car: There is free parking at CMB.

By bus: The A-line will work — it is only a short distance from one of the  stops on 5th street to the CMB, and the bus will go through central Davis, where most people will have their hotel rooms. Come early and save yourself a nice seat ;). We will be there starting at 12:45.

Registration

Registration is now closed. We are at the limit with regard to space.