Out of recent conversations with a whole bunch of folks (e.g. John Trueswell, Jennifer Arnold, Elsi Kaiser, Matt Traxler, Mike Tanenhaus, Jim Magnuson, and more), we came up with the idea to possible hold a workshop on “good practices in ordinary and multilevel regression models” [working title ;)] for researchers working on psycholinguistics/the psychology of language just a day before CUNY 2009 (to be held 03/26-28 at UC Davis), so 03/25 in Davis. This is just a baby of thought at this point, but if you’re interested, I’ve summarized some thoughts below and I’d appreciate your feedback (just leave a comment below and I will receive it).
Regression techniques, including multilevel/mixed models, have received increasing attention in the literature (especially, the special issue in JML that’s about to come out) and more and more researchers in psycholinguistics seem to be gviving them a shot. however, there are not (yet) a standard part of the education in statistics for most people in our field. This leads to the problem that more and more work uses these analyses while at the same time there is a large degree of uncertainty among the users as to what constitutes good practice. This workshop is meant to provide a forum to discuss question like: what do you need to do to check whether you can trust your mode? What information do you need to provide in a paper so that readers can evaluate your model?
What the workshop is meant to provide a forum for:
- the meeting is thought of as an informal meeting with at least 50% discussion, question time for researchers that are beginning to work with these models. It’s not meant as an introduction to these models (i.e. not a tutorial on regression or multilevel models; we would be catering mostly to folks who are in the process of using these models, no? I also don’t imagine this to be an advertisement workshop for these models, but rather a forum for those of us who are interested in them to exchange ideas and what we know about best practices).
- we want to keep this meeting simple (it’s too late to organize a huge workshop), lasting between 3 – 6 hours in the afternoon on the day before CUNY?
- there would be lectures/introductions to the following issues
- common issues in regression modeling
- overly influential cases
- model quality (e.g. residuals for linear models)
- building a model: adding/removing variables (also: interactions)
- some solutions to these problems for common model types
- outlier handling
- removing collinearity (e.g. PCA, residualization)
- stratification (using subsets of data)
- interpreting the model, making sure the model answers the question of interest:
- testing significance (SE-based tests vs. model comparison)
- interpration of model output, e.g. interpreation of coefficients
- (also: coding of variables)
- follow-up tests
- differences between different models (e.g. ordinary vs. multilevel; linear vs. logit) in terms of available measures of fit; test of significance; etc.
- what’s out there right now in terms of model types and what’s soon to come? how do these models relate to other models being using in other disciplines? — Harald suggested to invite Doug Bates (the developer of lmer and a first class statistician working on multilevel models; he has apparently attended other similar conferences where folks were interested in multilevel models), Matt mentioned Shelley Blozis (a quantitative psychologists at UC Davis working with various types of multilevel models). I think that is a GREAT idea, because quite frankly they know loads more about these models than we (at least I) do. I think especially for a look into the future and where the field may be heading, they could be very helpful. However, we probably would need some funding to invite Doug Bates, if we want to do that (on the odd chance that he might have time).
- Any ideas as to how we could get a modest amount of funding, e.g. to invite Doug Bates and/or Shelly Blozis and/or anybody else you would think is a great person for what we have in mind?
- Update 10/18/08: Several people have offered potential institutional support for this event:
- common issues in regression modeling
That’s my five cents for now. Updates to follow. Please feel free to leave comments.