### Pre-CUNY Workshop on “Good practices in ordinary and multilevel regression models”?

Out of recent conversations with a whole bunch of folks (e.g. John Trueswell, Jennifer Arnold, Elsi Kaiser, Matt Traxler, Mike Tanenhaus, Jim Magnuson, and more), we came up with the idea to possible hold a workshop on “good practices in ordinary and multilevel regression models” [working title ;)] for researchers working on psycholinguistics/the psychology of language just a day before CUNY 2009 (to be held 03/26-28 at UC Davis), so 03/25 in Davis. This is just a baby of thought at this point, but if you’re interested, I’ve summarized some thoughts below and I’d appreciate your feedback (just leave a comment below and I will receive it).

**Motivation**

Regression techniques, including multilevel/mixed models, have received increasing attention in the literature (especially, the special issue in JML that’s about to come out) and more and more researchers in psycholinguistics seem to be gviving them a shot. however, there are not (yet) a standard part of the education in statistics for most people in our field. This leads to the problem that more and more work uses these analyses while at the same time there is a large degree of uncertainty among the users as to what constitutes good practice. This workshop is meant to provide a forum to discuss question like: what do you need to do to check whether you can trust your mode? What information do you need to provide in a paper so that readers can evaluate your model?

**What the workshop is meant to provide a forum for:**

- the meeting is thought of as an informal meeting with at least 50% discussion, question time for researchers that are beginning to work with these models.
I also don’t imagine this to be an advertisement workshop for these models, but rather a forum for those of us who are interested in them to exchange ideas and what we know about best practices*It’s not meant as an introduction to these models (i.e. not a tutorial on regression or multilevel models*; we would be catering mostly to folks who are in the process of using these models, no?**).** **we want to keep this meeting simple (it’s too late to organize a huge workshop), lasting between 3 – 6 hours in the afternoon on the day before CUNY?**

- there would be lectures/introductions to the following issues
- common issues in regression modeling
- collinearity
- overfitting
- overly influential cases
- overdispersion?
- model quality (e.g. residuals for linear models)
- building a model: adding/removing variables (also: interactions)

- some solutions to these problems for common model types
- outlier handling
- centering
- removing collinearity (e.g. PCA, residualization)
- stratification (using subsets of data)

- interpreting the model, making sure the model answers the question of interest:
- testing significance (SE-based tests vs. model comparison)
- interpration of model output, e.g. interpreation of coefficients
- (also: coding of variables)
- follow-up tests

- differences between different models (e.g. ordinary vs. multilevel; linear vs. logit) in terms of available measures of fit; test of significance; etc.
*what’s out there right now in terms of model types and what’s soon to come? how do these models relate to other models being using in other disciplines?***—**Harald suggested to invite Doug Bates (the developer of*lmer*and a first class statistician working on multilevel models; he has apparently attended other similar conferences where folks were interested in multilevel models), Matt mentioned Shelley Blozis (a quantitative psychologists at UC Davis working with various types of multilevel models). I think that is a GREAT idea, because quite frankly they know loads more about these models than we (at least I) do. I think especially for a look into the future and where the field may be heading, they could be very helpful. However, we probably would need some funding to invite Doug Bates, if we want to do that (on the odd chance that he might have time).**Any ideas as to how we could get a modest amount of funding, e.g. to invite Doug Bates and/or Shelly Blozis and/or anybody else you would think is a great person for what we have in mind?****Update 10/18/08:**Several people have offered potential institutional support for this event:**Matt Traxler and Tamara Swaab (UC Davis)****Center of Language Science (University of Rochester)****Institute for Research in Cognitive Science (University of Pennsylvania)**

- common issues in regression modeling

That’s my five cents for now. Updates to follow. Please feel free to leave comments.

Florian

October 18, 2008 at 8:22 pm

Great idea Florian. Here are a few specific (nuts-and-bolts) issues that might be good to cover in some way:

1) what kinds of standards should we arrive at for which random effects to include in a model for mixed effects analyses of by-subjects, by-items ANOVA type data of the sort that are classic in psycholinguistics? Should we try to arrive at such standards at all?

2) what is the best way to assess the significance of contribution to a mixed-effects model of several fixed-effect parameters simultaneously? (e.g., main effect of a 3-level factor)

3) should we be moving toward using non-Gaussian linking functions in the analysis of psycholinguistic RT data?

My two cents,

Roger

LikeLike

October 20, 2008 at 4:18 pm

Good ideas, Roger. Dale Barr made the following additional suggestions:

It sounds like a really useful workshop. I hope that we can also put

the following “eyetracking specific” issues on the agenda:

1. getting the “right” standard errors in the face of dependencies that

are the result of the very high sampling rate of eyetrackers

2. comparing across distinct regions of interest

3. baseline issues, i.e., measuring and controlling for effects that are

present prior to the onset of a stimulus word/expression

LikeLike

October 21, 2008 at 10:30 am

Another speaker who was suggested is Reinhold Kliegl, who apparently gives excellent talks and he knows the CUNY audience well. The problem I see is that flying him over from Potsdam … given that this is a spontaneous idea and we didn’t apply for funding =).

LikeLike

November 8, 2008 at 4:57 pm

[…] workshop on regression and multilevel modeling (cntd) Some time ago, I announced that some folks have been thinking about organizing a small workshop on common issues and standards […]

LikeLike