Abstract
Humans communicate using language, but they also communicate using gesture – spontaneous movements of the hands and body that universally accompany speech. Gestures can be distinguished from other movements, segmented, and assigned meaning based on their forms and functions. Moreover, gestures systematically integrate with language at all levels of linguistic structure, as evidenced in both production and perception. Viewed typologically, gesture is universal, but nevertheless exhibits constrained variation across language communities (as does language itself ). Finally, gesture has rich cognitive dimensions in addition to its communicative dimensions. In overviewing these and other topics, we show that the study of language is incomplete without the study of its communicative partner, gesture.
1. Introduction
The main aim of modern linguistics is to document and analyze the grammatical patterns of human language. In pursuing this aim, linguists often create “grammars” for individual languages. Each of these works characterizes in great detail the words and rules of a particular language – the stuff speakers know when we say they know Seri, Georgian, or English. There are some topics covered in every grammar, and other topics that are rarely, if ever, included. One topic likely to be in the latter category is gesture, the focus of this article. From one perspective, this omission makes sense. After all, gesture is not part of the language proper. (Or is it?) But from another perspective, omitting gesture is puzzling simply because wherever people use language – any language – they use gesture too. Gesture is universal, just as universal as language, and, as we will see, gesture and language go hand in hand. At almost every level of analysis that linguists are interested in – from prosody to discourse structure – research has recently uncovered systematic and sometimes surprising relationships between language and gesture. In this review, we describe what is known about these relationships and about the properties and patterns of gesture itself.
2. Defining, identifying, and classifying gestures
First, it may be helpful to dispel some myths about what gesture is and what it is not. Gesture is not just for Italians (though their gestures do stand out in certain respects, as we discuss below); it’s not what mimes do (that is what is called pantomime); it’s not the same as the signs of sign languages (though there are interesting connections between the two, which we touch on later); and, finally, it’s not generally impolite (though sticking your finger in someone’s face still, in many cases, is). So what, then, is gesture? Kendon (2004:7) defines gesture as “visible action when it is used as an utterance or as part of an utterance.” Such visible actions are diverse and include: points, shrugs, and nods; illustrations of the size, shape, and location of objects; demonstrations of how to perform actions; depictions of abstract ideas and relationships; and many other everyday communicative actions of the body. Our focus here is on gestures produced during the course of spoken language production – co-speech gesture – but there are also interesting cases of hearing individuals using gesture in place of speech because of taboos (e.g., Kendon 1988) or noise (e.g., Meissner and Philpott 1975).
Listeners seem to intuitively distinguish gestures from the stream of other motor actions performed in the course of communication (Kendon 2004), including fidgeting and functional interaction with objects, such as drinking from a glass. Gesture (as we use the term here, but see Ekman and Friesen 1969) also does not include the body language or affective facial expressions or reactions that often reveal a person’s attitude or emotional state, such as moving away from one’s interlocutor, wincing in pain, or laughing. The reason for excluding these movements may be framed in terms of Lyons’ (1977) distinction between informative and communicative signals. Many of our everyday actions function as informative signals to our interlocutors even though they are not necessarily intended to communicate. Moving a glass to our mouth to take a drink, for example, informs the world that we are thirsty. Moving an empty, cupped hand toward our mouth, however, communicates the idea of taking a drink. This does not mean that we are fully aware of all of our gestures or that they all have crystal clear meanings, just that they are part of our general effort to communicate. Rhythmic “beat” gestures (Efron 1972; Ekman and Friesen 1972; McNeill 1992), for example, play an important role in language production, even though the gestural forms themselves may communicate nothing specific beyond emphasis.
Gesture, like speech, can be thought of in terms of units, and it is often useful to segment a gesture from the stream of gestural activity. Some traction on this matter can be gained by considering the “phases” of a gesture, as defined by Kendon (1980): preparation, stroke, and retraction ( for a more recent perspective on gesture identification and coding, including proof of concept from inter-coder reliability levels, see Kita, Van Gijn, and Van der Hulst 1998). The preparation phase is the movement of the hand as it readies itself for the gestural stroke. The stroke phase is the most effortful and most meaningful phase of the gesture. It may then be followed by a retraction phase, where the hand returns to resting position, or it may be followed by the preparation or stroke phase of a subsequent gesture. Thus, just as the syllables of language can be segmented and counted by identifying the syllable nuclei, so too can the gesture stream be segmented using the stroke nuclei of individual gestures and their associated preparation and retraction phases (both of which are optional, as are the onsets and codas of syllables). Gestural phases can also include holds, moments in which the hands remain static in gesture space.
Once identified and segmented, gestures can be classified along a number of dimensions, and these taxonomies are important in understanding the relationship between gesture and speech. One way that gesture can be classified is according to the articulator used to produce the gesture, for example, the hand or the head. The field of gesture research has focused primarily on manual gestures – which appear to be most common and most complex – but gestures produced with the head and face are also commonplace in speech communities around the world. Indeed, one candidate for a gestural universal is the use of the head (e.g., head nod and headshake) to convey affirmation and negation ( Jakobson, 1972; Kendon 2002). The properties and patterns of these and other gestures produced with non-manual articulators are an interesting frontier for future research, but here we follow the field’s focus on the hands.
A second way to classify gestures is according to their function in communication. Here, the main divide lies between gestures that are interactive – that is, gestures that manage the communicative dialogue between interlocutors (elsewhere called pragmatic, illocutionary, or discourse gestures [Kendon 1995]) – and gestures that are representational – that is, gestures that communicate something about the topic or primary content of the utterance. Interactive gestures do not represent the content of the speech with which they co-occur but instead help frame the speech within its discourse context. These include: gestures that regulate turn-taking behavior by indicating when the floor is being ceded or maintained; gestures that show that an idea, proposal, or observation is being presented; and gestures that show that the speaker is seeking feedback from an interlocutor. The genuinely interactive role of these gestures is evidenced by how they behave with respect to the discourse context. Unlike representational gestures, interactive gestures are less frequent when the interlocutor is absent or not visible than when he or she is present (Bavelas et al. 1992). Interactive gestures sometimes appear to rely on a conduit metaphor (McNeill 1992), wherein communication is treated as an act of content transmission; this transmission is embodied in how we gesture about receiving, giving, and otherwise handling our speech and its contents (Streeck 2009). For example, when making a proposal (“how about…”), the speaker may move an open palm toward the interlocutor, as if he or she is actually offering this proposal for consideration. Finally, listeners themselves can, and do, gesture. The gestures produced by a conversational partner, such as head nodding, can function interactively in their own right as a backchannel signal to the speaker (Kendon 1972; Kita and Ide 2007; McClave 2001).
The lion’s share of research, however, has been devoted to representational gestures – gestures that communicate the topic of the utterance. Representational gestures “mean” in different ways. First, they can mean by directly pointing to objects or locations in space. Such gestures are often called “deictic gestures.” Second, they can mean by depicting properties of an object, scene, or action, as when a speaker uses gesture to describe a memorable event that she witnessed. Gestures can also represent metaphorical properties, as when a speaker uses gesture to display a hierarchy in terms of a series of vertical positions (hierarchies are not literally vertical positions, except on Olympic podia). When gestures depict concrete imagery, they are often called “iconic gestures,” and when they depict abstract imagery, they are often called “metaphoric gestures” (McNeill 1992). Third, gestures can mean what they mean because of a convention in the community, in the same way that the word “dog” means what it means to English speakers because of a convention. These conventionalized gestures, such as the “thumbs up,” the thumb-and-index-finger “okay,” and the circling index finger “crazy,” as well as many others across speech communities, are called “emblems” (Ekman and Friesen 1969; McNeill 1992). Note, however, that emblems may also have deictic or iconic properties. It is probably no accident, for instance, that the “crazy” gesture is produced near the head, or that the “thumbs up” gesture points up rather than down. Indeed, as discussed by Enfield (2009) and others, gestures often mean through some combination of indexicality, iconicity, and conventionality – the three principle types of meaning described by Peirce (1932 [1895]). For this reason, although many researchers continue to use labels that imply discrete categories or types of representational gestures (“iconic gestures,” “deictic gestures,” and so on), others emphasize that gestures have different meaning dimensions that often blend together (Kendon 2004; McNeill 2005).
3. Gesture–language relationship
The intimate relationship between speech and gesture has two broad dimensions: timing and meaning. With respect to timing, a definitional characteristic of co-speech gesture is that it is co-produced with a linguistic message as part of a communicative act. Of course, people do not gesture the entire time they are speaking. Nor is it the case that each and every gesture is accompanied by speech. The important point, rather, is that when people produce co-speech gestures, those gestures are almost always temporally aligned in some meaningful way with a spoken utterance. With respect to meaning, gesture and speech have been argued to share an underlying conceptual message and to collaborate as two mechanisms for communicating this message (McNeill 1992). In this sense, gesture and speech are considered to be co-expressive, although the contributions of these communicative channels may be supplementary to, or redundant with one another (de Ruiter, Bangerter, and Dings 2012; Goldin-Meadow 2003a). These two fundamental dimensions – timing and meaning – frame the broader study of the relationship between gesture and language. The co-timing of gesture and speech has import for the prosodic integration of gesture and language (Section 3.1), whereas their co-expressivity has import for the meaningful integration of gesture and language (Section 3.2).
3.1. TEMPORAL AND PROSODIC INTEGRATION OF SPEECH AND GESTURE
Considering gesture as but one component of a multi-channel and multi-modal communicative system, we draw an analogy to the prosodic structure of language. The pitch excursuses and phrasing (via pausing and other boundary markers) of the prosodic system would lose their import if they were not temporally anchored to the segmental stream in a meaningful way. The prosodic focus of a constituent, for example, has its intended effect because of its meaningful, temporal alignment with that constituent. Temporal alignment is also a basic property of co-speech gesture, one that is likely necessary for gesture to be understood by listeners. The stroke of an iconic gesture, for example, is produced in temporal alignment with the linguistic unit whose meaning it iconically represents or supplements, sometimes called the “lexical affiliate” (Butterworth and Beattie 1978; Kendon 1972; McNeill 1992; Morrel-Samuels and Krauss 1992; Nobe 2000; Schegloff 1984), although it is often impossible to associate a gesture with a single lexical meaning as its meaning may be associated with a larger linguistic unit. Moreover, this synchronization is not always perfect, with gestures very often slightly preceding the part of speech with which they are associated. Interestingly, these slight misalignments do not seem to pose trouble for listeners (de Ruiter 2000; McNeill 2005).
The analogy between prosody and gesture becomes even more apparent in light of the evidence that these two communicative channels, in addition to integrating with the segmental speech stream, also integrate with and influence each other (Swerts and Krahmer 2008). Indeed, early research in prosody, now a productive subfield of linguistics in its own right, drew explicit connections between gesture and prosody (Bolinger 1983). Beat gestures, for example, integrate with the prosodic and rhythmic structure of language (hence, “beat”) and have been found to align with prosodic peaks (Leonard and Cummins 2011, Roustan and Dohen 2010) and stressed syllables (McClave 1994). Numerous studies have also found that the presence and position of co-speech gesture can influence perceived prominence (Bull and Connelly 1985). Interestingly, Krahmer and Swerts (2007) have found that the relationship between gesture and prosody is bi-directional – hearers perceive increased prosodic prominence when a meaningless beat gesture is present, and speakers increase the prosodic prominence of linguistic units when those units are co-produced with a beat gesture.
There are also correlations between specific gestural forms and the information structural role of the speech they accompany. This provides another dimension of similarity between gesture and prosody as specific prosodic melodies have also been found to correlate with, and potentially signal, specific information structural interpretations (Hirschberg and Ward 1995, Pierrehumbert and Hirschberg 1990). As discussed by Jackendoff (1972), for example, the sentence “Fred ate the beans” has a specific prosodic melody ( fall–rise) on Fred when uttered in response to the question “What about Fred?” This melody marks Fred as the contrastive topic of the utterance and changes if the information structural role of Fred changes (as is the case when the question under discussion is instead “What about the beans?”). Gesture, too, may provide a cue to the information structural properties of speech, and listeners may be sensitive to this information. For example, Kendon (1995) has found that the topic (vs. comment) portion of an utterance in Southern Italy frequently co-occurs with a grasp-like closure of the hands (“Finger Bunch”), whereas the focus (vs. theme) portion of the utterance frequently co-occurs with a precision gesture in which the thumb and index finger form a circle (“Ring”) (see Seyfeddinipur 2004 for kindred observations from Iran, and Lempert 2011 for related observations about political speeches). Moreover, like prosody, gesture has effects on the meaning of a given string. For example, Prieto et al. (2013) found that both prosody and gesture can influence whether Catalan ningú and Spanish nadie receive a negative concord (“nobody”) or double negative (“everybody”) interpretation. Along the same lines, Harrison (2010) found that the scope of a negator like not or n’t in English – that is, the string of words that the negator negates – may co-occur with a negative gesture held in space (post-stroke hold), whereas the negator itself co-occurs with the gestural stroke.
3.2. SEMANTIC INTEGRATION OF SPEECH AND GESTURE
A fundamental difference between speech and gesture is that their representational formats are different and, as a result, the two modalities are suited to expressing different kinds of information: speech is categorical and discrete, whereas gesture is gradient and analog. Speech is thus not well-equipped to encode visuo-spatial information, whereas gesture seems to be designed for this task. For example, when a speaker utters the box is near the table, he or she has encoded in speech two objects (box and table) and a relation between them (near). However, the co-speech gesture produced along with this utterance is likely to encode fine-grained information about the objects (the size of the box and the height of the table) and their relation (how far apart the two are and how they are arranged) that does not appear in speech. In this case (as in most instances of co-speech gesture), the gesture functions as a semantically supplementary channel to the spoken language: the gesture contributes information that is not fully specified in the speech. Recent formal work has debated the nature of the supplementary semantic information that gesture can contribute. For example, Ebert and Ebert (2014) and Schlenker (2014) explore how gesture may be subsumed under existing analyses of components of semantic meaning that are not part of the asserted content of an utterance, such as appositive descriptions and speaker presuppositions. This area of research clearly benefits from, and ties in with, recent formal analyses of the semantics of gestures (Lascarides and Stone 2009).
Although, as just noted, the information conveyed in gesture can, at times, supplement the information conveyed in speech, there are many times when gesture and speech convey information that appears to be redundant: the gesture contributes information that is already fully specified in the speech. We might have imagined that speakers would use their gestures primarily to disambiguate ambiguities in the spoken language (see de Ruiter, Bangerter, and Dings 2012 for discussion). However, and perhaps counter-intuitively, this is not often the case. So, Kita, and Goldin-Meadow (2009) found, for instance, that English speakers are more likely to produce an identifying gesture when a referent is uniquely specified in speech than when it is not fully specified in speech. This kind of redundancy between speech and gesture has also been documented in the domain of lexical and grammatical aspect, where, for example, gestures co-occurring with imperfective and progressive aspect last longer and involve more repetition than gestures co-occurring with perfect or perfective aspect (Duncan 2002; Parrill, Bergen, and Lichtenstein 2013). Another relatively transparent case of this phenomenon is found in iconic gestures that accompany spatial language. For example, a Zinacantecan gesturer discussed by Haviland (2004) described walking a far distance eastward. Concurrent with this spoken description, he produced a gesture that traced an eastward trajectory and moved higher and higher in space to indicate relative distance (here, farther and farther). Patterns such as these support proposals in which speech and gesture are planned together to express a shared underlying concept (Kita & Özyürek, 2003).
As a final point that relates to both co-timing and co-expressivity, one area that remains to be studied is the extent to which speakers have firm intuitions about the forms of gestures and the timing of gesture and speech. Such intuitions would be akin to notions of grammaticality and acceptability that have received significant attention in prosody (Gussenhoven 1983) and linguistics more generally. Even though speakers are typically not aware that they have gestured (Alibali, Flevares and Goldin-Meadow 1997), they may still have intuitions about what would make an acceptable or unacceptable gesture. This is a rich area for future study.
4. Variation in gesture across languages and cultures
Just as language is universal, yet variable, across speech communities, so too is gesture (see Kita 2009 for a review). However, there is, unfortunately, no entrenched typological tradition in the field of gesture studies, as there is for the field of linguistics. Consequently, our current knowledge of variation in human gesture is fragmentary. Nevertheless, it is clear that there are both universal patterns along with variation. An obvious place where we find such variation is in emblems – “frozen” gestural forms that have conventional meanings. Although all cultures seem to have gestural emblems, cultures vary both in the specific emblems they use (including notorious cases where an everyday emblem in one culture is grounds for fisticuffs in another) and in the size of their emblem inventories (McNeill 1992; Morris et al. 1979), with languages such as Italian boasting an especially rich set (Kendon 2004). In this way, emblems are analogous to ideophones in spoken languages – most, if not all, languages have at least some, but they vary dramatically in how prominent a role they play in the language. Interestingly, gesture itself has been shown to have a special affinity with ideophonic language (Dingemanse 2013, Kita 1997, Kunene 1965).
Further, there are gestural forms that seem to be linked to particular communicative functions across all known cultures and languages, for example, using the head to affirm and negate ( Jakobson, 1972, Kendon 2002), or using the index finger to point (Liszkowski et al. 2012). But, within these generalizations, there is also more fine-grained texture. For example, some communities use a side-to-side headshake for negation, whereas others use a backwards head toss (Morris et al. 1979); and some communities, in addition to using the index finger to point, have conventions for pointing with the head and face (as discussed by Sherzer 1973, Enfield, 2009; Cooperrider and Núñez 2012; and contributors to Kita 2003).
Tentatively, it seems that negation and pointing vary areally, not language by language –pointing with the lips appears to be quite common across Central America, for example. But gesture may also vary in direct relation to patterns in the linguistic structure, strengthening the argument that the two form an integrated communicative system. One demonstration of this phenomenon is in how people speak and gesture about motion events. Languages vary in how they linguistically encode the path and manner of motion events (Talmy 2000), and the gestures that speakers of a given language use have been shown to reflect this variation. Speakers of Japanese and Turkish (versus English) are unlikely to tightly package information about both path and manner within a linguistic phrase. Likewise, when gesturing about motion events, speakers of Japanese and Turkish (versus English) are unlikely to encode both path and manner within a single gesture (Kita and Özyürek 2003). A second area where gesture may vary in relation to linguistic structure is in the expression of spatial frames of reference. Speakers of languages that preferentially talk about spatial relationships using cardinal directions (east, west, north, and south) also seem to have a tendency to preserve cardinal relationships in their gestures (Haviland 1993; Levinson 2003). In these and other cases, it remains an interesting and open question as to whether there is a causal relationship between linguistic differences and gestural differences (or vice versa) or whether differences in these two communicative channels reflect a third, as yet undiscovered, underlying factor.
One area where gestural typology may prove especially useful is in explaining areal typologies of signed languages. Because users of signed languages are frequently deaf and without natural access to a spoken language, the emergence and evolution of signed languages can proceed in relative independence from ambient spoken languages. But it may not be independent of influence from ambient gesture – after all, deaf signers can easily see the gestures produced by the hearing community. Moreover, in language innovation, signers and homesigners1 frequently lexicalize and grammaticalize gestures of the ambient hearing community as part of their emerging linguistic system (Franklin, Giannakidou, and Goldin-Meadow 2011; Frishberg 1975; Janzen and Shaffer 2002; Nyst 2015; Zeshan 2000). For example, signers of Nicaraguan Sign Language, as well as Nicaraguan homesigners, use emblem-like gestures from the surrounding Spanish-speaking community as the basis for lexicalized signs like COME and CHILD (Spaepen et al. 2013). Needless to say, much more work is needed to fully understand typological patterns in gesture variation, how typologically defined gesture groups may or may not overlap with typologically defined language groups (and, relatedly, the extent to which “gesture typologies” and “language typologies” can be unified), and how patterns in co-speech gesture may influence sign language development.
5. Gesture and cognition
The discussion thus far has been devoted to establishing and exploring the intimate relationship between language and gesture in human communication. But in addition to its communicative side, gesture also has a no less interesting cognitive side (Kelly, Manning, and Rodak 2008 provide a more comprehensive overview of this aspect of gesture). One clue that gesture plays a role in cognition is the fact that people continue to gesture even when those gestures cannot possibly be communicative. People gesture when their interlocutor is not visible (Alibali, Heath, and Myers 2001) – for instance, when on the telephone (Bavelas et al. 2008) or when completing tasks alone at a computer (Chu and Kita 2011) – and blind people gesture, even when talking to others they know to be blind as well (Iverson and Goldin-Meadow 1998). This feature of gesture, too, is analogous to language. It has been independently proposed that language communicates, but may not fundamentally, nor solely, serve a communicative purpose (Chomsky 1966, 2010). Thus, both speech and gesture serve a dual purpose as both communicative and cognitive channels. Here, we turn our attention to the cognitive functions of gestures.
Gesture provides a window onto the speaker’s thoughts and can reveal thoughts that are not conveyed in speech. Speakers will, at times, describe one explanation for their beliefs in speech, while conveying a different explanation in gesture (Goldin-Meadow 2003a). A child who has not yet mastered Piagetian conservation may, for example, believe that the amount of water has changed when poured from a tall thin glass into a short wide dish. In justifying this belief, one child said, “It’s different because this one is high and this one is low,” while indicating with her hands the skinny diameter of the glass and then the wider diameter of the dish. Note that, while this is clearly not a redundant use of gesture, it is not a supplementary use of gesture either as the information conveyed in the gesture introduces a new dimension (width) not mentioned at all in speech. Importantly, although the information conveyed in gesture is different from that conveyed in the accompanying speech (a gesture-speech mismatch), the two pieces of information are not contradictory and have the potential to be integrated: height (in the child’s speech) and width (in the child’s gesture) are both necessary components to understanding conservation of quantity.
Interestingly, speakers who produce gesture-speech mismatches on a task are more likely to profit from instruction on that task than speakers who do not produce mismatches, suggesting that gesture and speech share a single underlying system even though the information expressed in the two modalities does not always match (Church and Goldin-Meadow 1986, Goldin-Meadow, Alibali and Church 1993). Gesture can tell us who is ready to learn. Moreover, gesture can propel learning – encouraging learners to gesture as they speak (Beaudoin-Ryan and Goldin-Meadow 2014; Broaders et al. 2007), or teaching them to use specific gestures (Goldin-Meadow, Cook, and Mitchell 2009; Novack et al 2014), can give them new ideas on topics as varied as Piagetian conservation of quantity, moral reasoning, and mathematical equivalence.
Along the same lines, gesture both provides a window onto the approaching linguistic milestones of the child (Iverson and Goldin-Meadow 2005; Cartmill, Hunsicker and Goldin-Meadow 2014; see Goldin-Meadow 2014 for a recent overview), and offers a tool for increasing vocabulary size (LeBarton, Goldin-Meadow, and Raudenbush 2015). Moreover, Alibali, Evans, Hostetter, Ryan, and Mainela-Arnold (2009) have shown that child narratives include more non-redundant gesture-speech combinations than those produced by adults, suggesting that gesture may serve as a compensatory communicative device. Gesture can also serve as both a compensatory and facilitatory device in the acquisition of a second language in adult learners (Gullberg 1998, Marcos 1979, McCafferty 2002).
In addition to playing a causal role in learning, gesturing can fulfill a wide range of other cognitive functions. First, gesture may help speakers find the right words. Gestures have been shown to facilitate lexical access and may play a role in the resolution of tip of the tongue states (de Ruiter 2000; Frick-Horbury and Guttentag 1998; Krauss 1998), although it may be movement per se that boosts lexical access (Ravizza 2003). Second, gestures may help speakers talk about, and think about, space. Gesture is particularly frequent when speech includes spatial content (see Alibali 2005 for a review and Hostetter and Alibali 2011 for a discussion of how this may relate to individual differences in cognitive skills), and prohibiting gesture leads to a decrease in speech rate in utterances with spatial content (Rauscher, Krauss and Chen 1996). More direct evidence for gesture’s role in spatial understanding comes from encouraging people to gesture on a spatial task such as mental rotation and finding improvement in their performance (Chu and Kita 2011). Third, speakers gesture more on problems that are conceptually difficult, even when lexical demands are equated (Alibali, Kita and Young 2000; Hostetter, Alibali, and Kita 2007; Kita and Davies 2009; Melinger and Kita 2007). Finally, gesturing can reduce demand on a speaker’s working memory. When asked to remember an unrelated list of items while explaining how they solved a math problem, speakers are able to maintain more items in verbal working memory (and thus recall more items) when they gesture during the explanation than when they do not gesture. This effect has been found in both children and adults (Goldin-Meadow, Nusbaum, Kelly and Wagner 2001).
An interesting avenue of research has also explored the comparison between gesture and action. This research has found that gesture is unique both in its coordination with speech and in its cognitive and communicative functions (see Kelly, Manning, and Rodak 2008 for discussion, as well as more recent work by Goldin-Meadow and Beilock 2010; Church, Kelly, and Holcombe 2014; Kelly et al. 2015; and Novack et al. 2014). This, too, confirms the special relationship between language and gesture and reveals that this relationship cannot be reduced to the attention or activation associated with increased motoric activity.
6. Conclusion
Since the birth of linguistics as an independent discipline, gesture has been considered a marginal topic, if it has been considered a topic at all. Yet as more and more connections between language and gesture are discovered, scholars in the cognitive sciences are beginning to acknowledge the fundamental unity of language and gesture and to study the two together as a tightly integrated system for communication. This step does not necessarily mean that gesture is part of the “grammar” inside speakers’ heads (see Goldin-Meadow and Brentari 2015 for discussion). It may mean, however, that gesture merits consideration in those hefty written grammars that we mentioned in the introduction (see Seyfeddinipur 2012 for practical information on how to go about including gesture in linguistic fieldwork). Exactly what the role of gesture will be in linguistics in the coming decades remains to be seen. We suggest that this role can be determined only after careful consideration of the structure of gesture and its ties to language. Our bet is that the rich structure of gesture and its multi-faceted, intimate relations to language will compel future generations of linguists to keep their eyes, and not just their ears, open.
Acknowledgments
We thank our colleagues for fruitful discussion of these and other topics as well as the editors and anonymous reviewers who provided valuable feedback and suggestions. Research reported in this publication was supported by NICHD of the National Institutes of Health under award numbers R01HD47450 and P0140605. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Biographies
Natasha Abner researches the linguistic structure of signed languages, including American Sign Language, French Sign Language, Nicaraguan Sign Language, and Nicaraguan homesign. A major focus of her work has been exploring how the linguistic properties of signed languages can be understood as grammatical properties common to all human languages, signed or spoken. She also examines how linguistic properties unique to signed languages may have their origins in the visual-gestural modality of these languages and, consequently, may be shared with patterns observed in co-speech gesture. Since completing her postdoctoral work in the Psychology Department at the University of Chicago, Abner has served as an Assistant Professor of Linguistics at Montclair State University. She holds a PhD in Linguistics from the University of California, Los Angeles, and a BA in Linguistics from the University of Michigan, Ann Arbor.
Kensy Cooperrider’s research explores relationships between language, gesture, and cognition through an interdisciplinary lens. A major focus of Cooperrider’s work has been how humans talk and think about space and, further, how they use space to talk and think about more elusive concepts, like time. His prior work on these topics has involved laboratory experiments, analysis of television interviews, and collaborative fieldwork carried out in Papua New Guinea and in Mexico. Reports of this research have appeared in publications such as Cognition, Gesture, and Trends in Cognitive Sciences. Before beginning his postdoctoral position at the University of Chicago, Cooperrider served as a Visiting Assistant Professor in the Case Western Reserve University department of Cognitive Science. He holds a PhD in Cognitive Science from the University of California, San Diego and a BA in Linguistics from Duke University.
Susan Goldin-Meadow researches language creation in deaf children and adults and the role of gesture in thinking and communicating. A major focus of her research has been studying the impact of linguistic environment on language learning by observing children who lack access to conventional linguistic input (i.e., deaf children whose hearing losses prevent them from learning a spoken language and whose parents have not exposed them to sign language). She has also explored the spontaneous gestures of hearing children and their parents, the insights early gesture offers into language development, and language learning in typically developing and brain-injured children. Goldin-Meadow is the Beardsley Ruml Distinguished Service Professor at the University of Chicago, a fellow of the American Academy of Arts and Sciences, and the 2015 recipient of the Williams James Award for Lifetime Achievement in Basic Research from the Association for Psychological Science. She holds a PhD in Psychology from the University of Pennsylvania and a BA in Psychology from Smith College.
Footnotes
Homesigners are deaf individuals who are unable to learn spoken language and have not been exposed to sign language. These individuals invent gestures, called homesigns, to communicate with the hearing individuals who surround them (Goldin-Meadow 2003b).
Work cited
- Alibali MW. Gesture in spatial cognition: expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation. 2005;5(4):307–31. [Google Scholar]
- Alibali MW, Evans JL, Hostetter AB, Ryan K, Mainela-Arnold E. Gesture–speech integration in narrative: are children less redundant than adults? Gesture. 2009;9(3):290–311. doi: 10.1075/gest.9.3.02ali. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Alibali MW, Flevares L, Goldin-Meadow S. Assessing knowledge conveyed in gesture: do teachers have the upper hand? Journal of Educational Psychology. 1997;89:183–93. [Google Scholar]
- Alibali MW, Heath DC, Myers HJ. Effects of visibility between speaker and listener on gesture production: some gestures are meant to be seen. Journal of Memory and Language. 2001;44:169–88. [Google Scholar]
- Alibali MW, Kita S, Young AJ. Gesture and the process of speech production: we think, therefore we gesture. Language and Cognitive Processes. 2000;15:593–613. [Google Scholar]
- Bavelas JB, Chovil N, Lawrie DA, Wade A. Interactive gestures. Discourse processes. 1992;15(4):469–89. [Google Scholar]
- Bavelas JB, Gerwing J, Sutton C, Prevost D. Gesturing on the telephone: independent effects of dialogue and visibility. Journal of Memory and Language. 2008;58(2):495–520. [Google Scholar]
- Beaudoin-Ryan L, Goldin-Meadow S. Teaching moral reasoning through gesture. Developmental Science. 2014;17(6):984–90. doi: 10.1111/desc.12180. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bolinger D. Intonation and gesture. American speech. 1983:156–74. [Google Scholar]
- Broaders SC, Cook SW, Mitchell ZA, Goldin-Meadow S. Making children gesture brings out implicit knowledge and leads to learning. Journal of Experimental Psychology: General. 2007;136(4):539–50. doi: 10.1037/0096-3445.136.4.539. [DOI] [PubMed] [Google Scholar]
- Bull Peter, Connelly G. Body movement and emphasis in speech. Journal of Nonverbal Behavior. 1985;9(3):169–87. [Google Scholar]
- Butterworth Brian, Beattie G. Gesture and silence as indicators of planning in speech. In: Campbell RN, Smith PT, editors. Recent advances in the psychology of language 4: Formal and experimental approaches. New York, NY: Plenum; 1978. pp. 347–60. [Google Scholar]
- Cartmill EA, Hunsicker D, Goldin-Meadow S. Pointing and naming are not redundant: children use gesture to modify nouns before they modify nouns in speech. Developmental Psychology. 2014;50(6):1660–6. doi: 10.1037/a0036003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chomsky N. Cartesian linguistics: a chapter in the history of rationalist thought. New York, NY: Harper and Row; 1966. [Google Scholar]
- Chomsky N. Some simple evo-devo theses: how true might they be for language? In: Larson RK, Déprez VM, Yamakido H, editors. The evolution of human language: biolinguistic perspectives. Cambridge, UK: Cambridge University Press; 2010. pp. 45–62. [Google Scholar]
- Chu M, Kita S. The nature of gestures’ beneficial role in spatial problem solving. Journal of Experimental Psychology: General. 2011;140(1):102–16. doi: 10.1037/a0021790. [DOI] [PubMed] [Google Scholar]
- Church RB, Goldin-Meadow S. The mistmatch between gesture and speech as an index of transitional knowledge. Cognition. 1986;23:43–71. doi: 10.1016/0010-0277(86)90053-3. [DOI] [PubMed] [Google Scholar]
- Church RB, Kelly S, Holcombe D. Temporal synchrony between speech, action and gesture during language production. Language, Cognition and Neuroscience. 2014;29(3):345–54. [Google Scholar]
- Cooperrider K, Núñez R. Nose-pointing: notes on a facial gesture of Papua New Guinea. Gesture. 2012;12(2):103–29. [Google Scholar]
- de Ruiter JP. The production of gesture and speech. In: McNeill D, editor. Language and gesture. Cambridge, UK: Cambridge University Press; 2000. pp. 248–311. [Google Scholar]
- de Ruiter JP, Bangerter A, Dings P. The interplay between gesture and speech in the production of referring expressions: investigating the tradeoff hypothesis. Topics in Cognitive Science. 2012;4:232–48. doi: 10.1111/j.1756-8765.2012.01183.x. [DOI] [PubMed] [Google Scholar]
- Dingemanse M. Ideophones and gesture in everyday speech. Gesture. 2013;13(2):143–65. [Google Scholar]
- Duncan S. Gesture, verb aspect, and the nature of iconic imagery in natural discourse. Gesture. 2002;2(2):183–206. [Google Scholar]
- Ebert C, Ebert C. Handout of a talk given at Semantics and Philosophy in Europe (SPE 7) Berlin, Germany: 2014. Jun 28, Gestures, Demonstratives, and the attributive/referential distinction. ▪▪. [Google Scholar]
- Efron D. Gesture, race and culture: a tentative study of the spatio-temporal and linguistic aspects of the gestural behavior of Eastern Jews and Southern Conditions. Berlin, Germany: Mouton de Gruyter; 1972. [Google Scholar]
- Ekman P, Friesen WV. The repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica. 1969;1:49–98. [Google Scholar]
- Ekman P, Friesen WV. Hand movements. The Journal of Communication. 1972;22:353–74. [Google Scholar]
- Enfield NJ. The anatomy of meaning: speech, gesture, and composite utterances. Cambridge, UK: Cambridge University Press; 2009. [Google Scholar]
- Franklin A, Giannakidou A, Goldin-Meadow S. Negation, questions, and structure building in a homesign system. Cognition. 2011;118(3):398–416. doi: 10.1016/j.cognition.2010.08.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frick-Horbury D, Guttentag RE. The effects of restricting hand gesture production on lexical retrieval and free recall. The American Journal of Psychology. 1998;111(1):43–62. [Google Scholar]
- Frishberg N. Arbitrariness and iconicity: historical change in American Sign Language. Language. 1975;51(3):696–719. [Google Scholar]
- Goldin-Meadow S. Hearing gestures: how our hands help us think. Cambridge, MA: Harvard University Press; 2003a. [Google Scholar]
- Goldin-Meadow S. Resilience of language: what gesture creation in deaf children can tell us about how all children learn language. New York, NY: Psychology Press; 2003b. [Google Scholar]
- Goldin-Meadow S. How gesture helps children learn language. Language in interaction. In: Arnon I, Casillas M, Kurumada C, Estigarribia B, editors. Studies in honor of Eve V. Clark. The Netherlands: John Benjamins Publishing Company; 2014. pp. 157–71. [Google Scholar]
- Goldin-Meadow S, Alibali MW, Church RB. Transitions in concept acquisition: using the hand to read the mind. Psychological Review. 1993;100(2):279–97. doi: 10.1037/0033-295x.100.2.279. [DOI] [PubMed] [Google Scholar]
- Goldin-Meadow S, Beilock SL. Action’s influence on thought: the case of gesture. Perspectives on Psychological Science. 2010;5(6):664–74. doi: 10.1177/1745691610388764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldin-Meadow S, Brentari D. Gesture, sign and language: the coming of age of sign language and gesture studies. 2015 doi: 10.1017/S0140525X15001247. Under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldin-Meadow S, Cook SW, Mitchell ZA. Gesturing gives children new ideas about math. Psychological Science. 2009;20(3):267–72. doi: 10.1111/j.1467-9280.2009.02297.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Goldin-Meadow S, Nusbaum H, Kelly S, Wagner S. Explaining math: gesturing lightens the load. Psychological Science. 2001;12:516–22. doi: 10.1111/1467-9280.00395. [DOI] [PubMed] [Google Scholar]
- Gullberg M. Gesture as a communication strategy in second language discourse: a study of learners of French and Swedish. Lund, Sweden: Lund University Press; 1998. [Google Scholar]
- Gussenhoven C. Testing the reality of focus domains. Language and Speech. 1983;26(1):61–80. [Google Scholar]
- Harrison S. Evidence for node and scope of negation in coverbal gesture. Gesture. 2010;10(1):29–51. [Google Scholar]
- Haviland JB. Anchoring, iconicity, and orientation in Guugu Yimithirr pointing gestures. Journal of Linguistic Anthropology. 1993;3(l):3–45. [Google Scholar]
- Haviland JB. Gesture. In: Duranti A, editor. A companion to linguistic anthropology. Malden, MA, Oxford, UK: Blackwell Publishing; 2004. pp. 197–221. [Google Scholar]
- Hirschberg J, Ward G. The interpretation of the high-rise question contour in English. Journal of Pragmatics. 1995;24(4):407–12. [Google Scholar]
- Hirschberg J, Ward G. Cognitive skills and gesture–speech redundancy: formulation difficulty or communicative strategy? Gesture. 2011;11(1):40–60. [Google Scholar]
- Hostetter AB, Alibali MW, Kita S. I see it in my hands’ eye: representational gestures reflect conceptual demands. Language and Cognitive Processes. 2007;22:313–36. [Google Scholar]
- Iverson JM, Goldin-Meadow S. Why people gesture when they speak. Nature. 1998;396(6708):228–228. doi: 10.1038/24300. [DOI] [PubMed] [Google Scholar]
- Iverson JM, Goldin-Meadow S. Gesture paves the way for language development. Psychological Science. 2005;16:367–71. doi: 10.1111/j.0956-7976.2005.01542.x. [DOI] [PubMed] [Google Scholar]
- Jakobson R. Motor signs for ‘yes’ and ‘no’. Language in Society. 1972;1:91–6. [Google Scholar]
- Janzen T, Shaffer B. Gesture as the substrate in the process of ASL grammaticization. In: Meier RP, Cormier K, Quinto-Pozos D, editors. Modality and structure in signed and spoken Languages. Cambridge, UK: Cambridge University Press; 2002. pp. 199–223. [Google Scholar]
- Kelly S, Healey M, Özyürek A, Holler J. The processing of speech, gesture, and action during language comprehension. Psychonomic Bulletin & Review. 2015;22(2):517–23. doi: 10.3758/s13423-014-0681-7. [DOI] [PubMed] [Google Scholar]
- Kelly S, Manning SM, Rodak S. Gesture gives a hand to language and learning: perspectives from cognitive neuroscience, developmental psychology and education. Language and Linguistics Compass. 2008;2(4):569–88. [Google Scholar]
- Kendon A. Some relationships between body motion and speech. Studies in dyadic communication. 1972;7:177–216. [Google Scholar]
- Kendon A. Gesticulation and speech: Two aspects of the process of utterance. In: Key MR, editor. The relationship of verbal and nonverbal communication. The Hague, The Netherlands: Mouton; 1980. pp. 207–227. [Google Scholar]
- Kendon A. Sign languages of aboriginal Australia: cultural, semiotic and communicative perspectives. Cambridge, UK: Cambridge University Press; 1988. [Google Scholar]
- Kendon A. Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics. 1995;23(3):247–79. [Google Scholar]
- Kendon A. Some uses of the head shake. Gesture. 2002;2(2):147–82. [Google Scholar]
- Kendon A. Gesture: visible action as utterance. Cambridge, UK: Cambridge University Press; 2004. [Google Scholar]
- Kita S. Two-dimensional semantic analysis of Japanese mimetics. Linguistics. 1997;35(2):379–415. [Google Scholar]
- Kita S, editor. Pointing: where language, culture, and cognition meet. Mahwah, NJ: Erlbaum; 2003. [Google Scholar]
- Kita S. Cross-cultural variation of speech-accompanying gesture: a review. Language and Cognitive Processes. 2009;24(2):145–67. [Google Scholar]
- Kita S, Davies TS. Competing conceptual representations trigger co-speech representational gestures. Language and Cognitive Processes. 2009;24:761–75. [Google Scholar]
- Kita S, Ide S. Nodding, aizuchi, and final particles in Japanese conversation: how conversation reflects the ideology of communication and social relationships. Journal of Pragmatics. 2007;39(7):1242–54. [Google Scholar]
- Kita S, Özyürek A. What does cross-linguistic variation in semantic coordination of speech and gesture reveal? Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language. 2003;48(1):16–32. [Google Scholar]
- Kita S, Van Gijn I, Van der Hulst H. Movement phase in signs and co-speech gestures, and their transcriptions by human coders. Gesture and Sign Language in Human-Computer Interaction. Bielefeld Gesture Workshop; September 17–19; 1998. pp. 23–35. [Google Scholar]
- Krahmer E, Swerts M. The effects of visual beats on prosodic prominence: acoustic analyses, auditory perception and visual perception. Journal of Memory and Language. 2007;57(3):396–414. [Google Scholar]
- Krauss RM. Why do we gesture when we speak? Current Directions in Psychological Science. 1998;7(2):54–60. [Google Scholar]
- Kunene DP. The ideophone in Southern Sotho. Journal of African Languages. 1965;4:19–39. [Google Scholar]
- Lascarides A, Stone M. A formal semantic analysis of gesture. Journal of Semantics. 2009;26(4):393–449. [Google Scholar]
- LeBarton ES, Goldin-Meadow S, Raudenbush S. Experimentally-induced increases in early gesture lead to increases in spoken vocabulary. Journal of Cognition and Development. 2015;16(2):199–220. doi: 10.1080/15248372.2013.858041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lempert M. Barack Obama, being sharp: indexical order in the pragmatics of precision-grip gesture. Gesture. 2011;11(3):241–70. [Google Scholar]
- Leonard T, Cummins F. The temporal relation between beat gestures and speech. Language and Cognitive Processes. 2011;26(10):1457–71. [Google Scholar]
- Levinson SC. Space in language and cognition. Cambridge, UK: Cambridge University Press; 2003. [Google Scholar]
- Liszkowski U, Penny B, Callaghan T, Takada A, de Vos C. A prelinguistic gestural universal of human communication. Cognitive Science. 2012;36(4):698–713. doi: 10.1111/j.1551-6709.2011.01228.x. [DOI] [PubMed] [Google Scholar]
- Lyons J. Semantics. I. Cambridge, UK: Cambridge University Press; 1977. [Google Scholar]
- McCafferty SG. Gesture and creating zones of proximal development for second language learning. The Modern Language Journal. 2002;86(2):192–203. [Google Scholar]
- McClave E. Gestural beats: the rhythm hypothesis. Journal of Psycholinguistic Research. 1994;23(1):45–66. [Google Scholar]
- McClave E. The relationship between spontaneous gestures of the hearing and American Sign Language. Gesture. 2001;1(1):51–72. [Google Scholar]
- McNeill D. Hand and mind: what gestures reveal about thought. Chicago, IL: University of Chicago Press; 1992. [Google Scholar]
- McNeill D. Gesture and thought. Chicago, IL: University of Chicago Press; 2005. [Google Scholar]
- Marcos LR. Hand movements and nondominant fluency in bilinguals. Perceptual and Motor Skills. 1979;48(1):207–14. doi: 10.2466/pms.1979.48.1.207. [DOI] [PubMed] [Google Scholar]
- Meissner M, Philpott SB. The sign language of sawmill workers in British Columbia. Sign Language Studies. 1975;9:291–308. [Google Scholar]
- Melinger A, Kita S. Conceptualisation load triggers gesture production. Language and Cognitive Processes. 2007;22(4):473–500. [Google Scholar]
- Morrel-Samuels P, Krauss RM. Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition. 1992;18(3):615–22. [Google Scholar]
- Morris D, Collet P, Marsh P, O’Shaughnessy M. Gestures, their origins and distribution. New York, NY: Stein and Day; 1979. [Google Scholar]
- Nobe S. Where do most spontaneous representational gestures actually occur with respect to speech. In: McNeill D, editor. Language and gesture. Cambridge, UK: Cambridge University Press; 2000. pp. 186–98. [Google Scholar]
- Novack MA, Congdon EL, Hemani-Lopez N, Goldin-Meadow S. From action to abstraction using the hands to learn math. Psychological Science. 2014;25(4):903–10. doi: 10.1177/0956797613518351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nyst V. The sign language situation in Mali. Sign Language Studies. 2015;15(2):126–50. [Google Scholar]
- Parrill F, Bergen BK, Lichtenstein PV. Grammatical aspect, gesture, and conceptualization: using co-speech gesture to reveal event representations. Cognitive Linguistics. 2013;24(1):135–58. [Google Scholar]
- Peirce CS. The icon, index, and symbol. In: Hartshorne C, Weiss P, editors. Collected Papers of Charles Sanders Peirce, Vol II: Elements of Logic. Cambridge, MA: Harvard University Press; 1932. pp. 156–73. 1895. [Google Scholar]
- Pierrehumbert JB, Hirschberg J. The meaning of intonational contours in the interpretation of discourse. In. Intentions in Communication. 1990:271–311. [Google Scholar]
- Prieto P, Borràs-Comes J, Tubau S, Teresa Espinal M. Prosody and gesture constrain the interpretation of double negation. Lingua. 2013;131:136–50. [Google Scholar]
- Rauscher F, Krauss RM, Chen Y. Gesture, speech, and lexical access: the role of lexical movements in speech production. Psychological Science. 1996;7:226–31. [Google Scholar]
- Ravizza S. Movement and lexical access: do noniconic gestures aid in retrieval? Psychonomic Bulletin & Review. 2003;10:610–5. doi: 10.3758/bf03196522. [DOI] [PubMed] [Google Scholar]
- Roustan B, Dohen M. Gesture and speech coordination: the influence of the relationship between manual gesture and speech. Proceedings of Interspeech 2010, 11th annual conference of the international speech communication association.2010. [Google Scholar]
- Schegloff EA. On some gestures’ relation to talk. In: Atkinson JM, editor. Structures of social action: studies in conversation analysis. Cambridge, UK: Cambridge University Press; 1984. pp. 266–96. [Google Scholar]
- Schlenker P. Gestural presuppositions. 2014 To appear in Snippets. [Google Scholar]
- Seyfeddinipur M. Meta-discursive gestures from Iran: some uses of the ‘Pistol Hand’. In: Müller C, Posner R, editors. The semantics and pragmatics of everyday gestures; Proceedings of the Berlin Conference; Berlin, Germany: Weidler Buchverlag; 2004. pp. 205–16. [Google Scholar]
- Seyfeddinipur M. Reasons for documenting gestures and suggestions for how to go about it. In: Thieberger Nicholas., editor. Oxford Handbook of Linguistic Fieldwork. Oxford, UK: Oxford University Press; 2012. pp. 147–65. [Google Scholar]
- Sherzer J. Verbal and nonverbal deixis: the pointed lip gesture among the San Blas Cuna. Language in Society. 1973;2(1):117–31. [Google Scholar]
- So WC, Kita S, Goldin-Meadow S. Using the hands to identify who does what to whom: gesture and speech go hand-in-hand. Cognitive Science. 2009;33(1):115–25. doi: 10.1111/j.1551-6709.2008.01006.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spaepen E, Coppola M, Flaherty M, Spelke E, Goldin-Meadow S. Generating a lexicon without a language model: do words for number count? Journal of Memory and Language. 2013;69(4):496–505. doi: 10.1016/j.jml.2013.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Streeck J. Gesturecraft: the manufacture of meaning. The Netherlands: John Benjamins Publishing; 2009. [Google Scholar]
- Swerts M, Krahmer E. Facial expression and prosodic prominence: effects of modality and facial area. Journal of Phonetics. 2008;36(2):219–38. [Google Scholar]
- Talmy L. Typology and process in concept structuring. II. Cambridge, MA: MIT Press; 2000. Toward a cognitive semantics. [Google Scholar]
- Zeshan U. Sign language in Indo-Pakistan: a description of a signed language. The Netherlands: John Benjamins Publishing; 2000. [Google Scholar]