speech adaptation

Speech recognition: Recognizing the familiar, generalizing to the similar, and adapting to the novel

Posted on Updated on

At long last, we have finished a substantial revision of Dave Kleinschmidt‘s opus “Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel“. It’s still under review, but we’re excited about it and wanted to share what we have right now.

Phonetic recalibration via belief updating
Figure 1: Modeling changes in phonetic classification as belief updating. After repeatedly hearing a VOT that is ambiguous between /b/ and /p/ but occurs in a word where it can only be a /b/, listeners change their classification of that sound, calling it a /b/ much more often. We model this as a belief updating process, where listeners track the underlying distribution of cues associated with the /b/ and /p/ categories (or the generative model), and then use their beliefs about those distributions to guide classification behavior later on.

The paper builds on a large body of research in speech perception and adaptation, as well as distributional learning in other domains to develop a normative framework of how we manage to understand each other despite the infamous lack of invariance. At the core of the proposal stands the (old, but often under-appreciated) idea that variability in the speech signal is often structured (i.e., conditioned on other variables in the world) and that an ideal observer should take advantage of that structure. This makes speech perception a problem of inference under uncertainty at multiple different levels Read the rest of this entry »

Running phonetic (adaptation) experiments online

Posted on Updated on

I’ve developed some JavaScript code that somewhat simplifies running experiments online (over, e.g., Amazon’s Mechanical Turk). There’s a working demo, and you can download or fork the source code to tinker with yourself. The code for the core functionality which controls stimulus display, response collection, etc. is also available in its own repository if you just want to build around that.

If you notice a bug, or have a feature request, open an issue on the issue tracker (preferred), or comment here with questions and ideas. And, of course, if you want to contribute, please go ahead and submit a pull request. Everything’s written in HTML, CSS, and JavaScript (+JQuery) and aims to be as extensible as possible. Happy hacking!

If you find this code useful for your purposes, please refer others to this page. If you’d like to cite something to acknowledge this code or your own code based on this code, the following is the paper in which we first used this paradigm:

  1. Kleinschmidt, D. F., and Jaeger, T. F. 2012. A continuum of phonetic adaptation: Evaluating an incremental belief-updating model of recalibration and selective adaptation. Proceedings of the 34th Annual Meeting of the Cognitive Science Society (CogSci12), 605-610. Austin, TX: Cognitive Science Society.

A more detailed journal paper is currently under review. If you’re interested, subscribe to this post and get the update when we post the paper here once it’s out (or contact me if you can’t wait).