Julie E. Boland
University of Michigan
Increasingly, psycholinguists are tracking eye movements during listening. Shifts in visual attention are clearly linked to the conceptual representations that result from language comprehension. However, there has been little evidence that this paradigm taps the transitory linguistic structures that form the foundation of sentence comprehension. Altmann and Kamide's (1999, Kamide, Altmann, & Haywood, 2001) research is the closest approximation. They have shown, for example, that listeners begin looking to a cake upon hearing "The boy ate the..." and that such looks are modulated by the combined constraints of the verb and the sentential subject. These data may suggest that listeners use verb argument structure to direct attention to likely arguments prior to phonological evidence of the arguments. However, the data are also consistent with a view in which the listener's gaze shifts in response to situational schema or non-linguistic plausibility assessments. On this account, listeners would be expected to look at a bed upon hearing "The boy slept..." even though "bed" is not a potential argument of "slept."
To differentiate these accounts, I manipulated both argument status and typicality of the target picture (see example sentences below). Participants viewed a four-photo array depicting the subject, direct object, a filler picture, and either a typical or atypical recipient/instrument/location while listening to sentences containing dative, transitive action, or intransitive verbs. The only task was answering comprehension questions. I analyzed participant gaze during an interval immediately following verb onset until early in the PP containing the target recipient, instrument, or location. Looks to the sentential subject predominated initially, then sharply declined in dative and transitive conditions, but not intransitive conditions. Beginning 500-1000 ms after verb onset (well before the onset of the PP containing the target), recipients attracted more looks than instruments and locations. Thus, I found effects of argument structure on both looks to the sentential subject and looks to the target noun. In contrast, there were no effects of typicality. Looks to plausible recipients ("class") were neither faster nor longer than looks to implausible recipients ("fireman"). Note that outside the context of books and teachers "firemen" are good recipients of an assignment. However, the lack of typicality effects in the instrument and location conditions as well suggests that typicality effects may be task dependent.
These results demonstrate that argument structure, rather than general conceptual knowledge, rapidly directed listener attention to relevant aspects of a visual scene.
Example Stimuli (typical target listed first)
Dative/Recipient: The book was challenging, so the teacher assigned it as homework to the class/firemen.
Action/Instrument: A broken pipe was flooding the laundry room, so the plumber cut it precisely with a saw/an axe Monday evening.
Intransitive/Location: The girl slept for a while on the bed/bus this afternoon.
References
Altmann, G. T. M., & Kamide, Y. (1999). Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition, 73, 247-264.
Kamide, Y., Altmann, G. T. M., & Haywood, S. (2001). Evidence for the time-course of constraint-application during sentence processing in visual contexts. The 14th Annual CUNY Conference on Human Sentence Processing, University of Pennsylvania, Philadelphia.