A preview of: Cued Speech and Cued Language for Deaf and Hard of Hearing Children
Cued Speech for Enhancing Speech Perception of Individuals with Cochlear Implants
by Jacqueline Leybaert, Cécile Colin, & Catherine Hage
Introduction
This volume documents how deaf children who have been provided with Cued Speech successfully use language representations in major cognitive activities like reading, spelling, remembering, and rhyming without auditory input. The main source of improvement in these cognitive skills is the advantage provided by Cued Speech for speech perception which leads to the natural acquisition of English and other traditionally-spoken languages.
In one of the first studies addressing the issue of spoken language perception, Nicholls and Ling (1982) studied a group of Australian profoundly deaf children educated with Cued Speech at school with for at least three years. They found that speech reception scores of these children increased from about 30 percent for both syllables and words in the lipreading condition to more than 80 percent in the lipreading + cues condition. They emphasized that the children’s average scores in the lipreading + cues condition were within the range of normal hearing listeners’ reception scores of similar material from audition.
Périer, Charlier, Hage and Alegria (1988) studied the advantage provided by the addition of cues to French sentence comprehension. They found an increase from 39 percent correct responses in the lipreading condition to 72 percent in the lipreading + cues condition for a group of children exposed early to Cued Speech, and from 37 to 53 percent for those who were exposed to Cued Speech later and only at school, suggesting a variability related to experience in perceiving and discriminating the phonetic structure of Cued Speech.
Now that most children born profoundly deaf are fitted with a cochlear implant during the early language learning years (Spencer & Marschark, 2003), the need for using Cued Speech might be less apparent. Improvement in children’s hearing via cochlear implants is impacting on strategies of perception of oral language (Geers, 2006). That is, with auditory training, many children with cochlear implants may understand speech sufficiently without having to look at the speaker. However, even for normally hearing people, speech detection and intelligibility are influenced by a speaker’s face. From the seminal work of Sumby & Pollak (1954), it is known that visual speech information dramatically enhances the identification of speech when the auditory information is degraded by noise. Auditory and visual modalities are complementary in the transmission of phonetic features. While voicing and manner of articulation are quite resistant to noise, place of articulation is not. Information about place of articulation, in contrast, is transmitted well via the visual modality (Summerfield, 1987). This multimodal nature of speech reception has been shown through the well-known and commonly cited McGurk effect (McGurk & MacDonald, 1976).
Another compelling reason for considering the multimodal nature of speech reception through a cochlear implant and the benefit of visual integration in speech perception is the fact that the signal delivered by the cochlear implant remains imprecise and incomplete. Recent advances in psychoacoustic research have clarified the role of two types of temporal information in speech perception: (1) frequency information and (2) temporal fine structure. “The auditory system performs a limited-resolution spectral analysis of sounds using an array of overlapping ‘auditory filters’ with center frequencies spanning from 50 to 15,000 Hz. The output of each filter is like a bandpass filtered version of the sound, which contains two forms of information: fluctuations in the envelope (the relatively slow variations in amplitude over time) and fluctuations in the temporal fine structure (the rapid oscillations with rate close to the center of the frequency of the band). The temporal fine structure is often described as a ‘carrier’ while the envelope is described as ‘an amplitude modulator applied to the carrier’” (Lorenzi et al., 2006). Currently, cochlear implants typically use 16-22 electrodes placed along the tono-topic axis of the cochlea, each electrode being designed to provoke a frequency-specific neural activation; however, within each region of stimulated neurons, the temporal fine structure of neural response is quite different from that occurring in a normal cochlea (Shannon, 2007). Modern cochlear implants provide good information about the slow variations in amplitude of the envelope; however, they are poor at transmitting frequency information and information about temporal fine structure (Glasberg & Moore, 1986; Grosgeorges, 2005; Lorenzi et al., 2006).
The lack of temporal fine structure in cochlear implants has consequences on the perception of phonetic features, on degradation of speech perception by noise, and on the perception of musical pitch. At the phonetic level, place of articulation and voicing are mostly impaired, whereas the transmission of manner is well-preserved. Consequently, individuals with a cochlear implant confound minimal word pairs that differ only by place of articulation, such as buck/duck (Giraud, Price, et al., 2001), which create confusions in acquisition of meanings by children. Due to the fragility of the transmission of phonetic features, speech perception through a cochlear implant is dramatically impaired in noisy listening environments (Fu & Nogaki, 2004; Lorenzi et al., 2006). Individuals with a cochlear implant also have difficulties in perceiving musical information related to pitch, while the information about rhythm is relatively well preserved (Fearn & Wolfe, 2000; Frère & Leybaert, 2007). These problems are currently being addressed by the companies who develop cochlear implant technology, and will certainly be reduced in the future (see for example Research with Cochlear Implants). Until that time, however, these problems might best be addressed via visual support.
Given these limitations of cochlear implants, it is reasonable to believe that speechreading and manual cues of Cued Speech remain of valuable use for speech perception by children with a cochlear implant who are in the process of language development. In the following sections, we will discuss research related to the positive effect of visual speech information on language perception at the level of: (1) phonemic syllables; (2) word and pseudoword identification, and (3) morpho-syntactical development.
Integration of Auditory and Speechread Information on the Phonetic Perception of Syllables
Deaf children fitted with a cochlear implant have been found to perform better on speech recognition tasks when visual information is available conjointly with the auditory information rather than when only the auditory information is available (Lachs, Pisoni, & Kirk, 2001; Rouger, Lagleyre, Fraysse, Deneve, Deguine, & Barine, 2007). Given their limited auditory experience, individuals with a cochlear implant might rely more on speechreading than normally hearing children (Clarke, 2003; Rouger et al., 2007).
If audio-visual integration mainly depends on the balance between the weight devoted to the processing of auditory and visual information, it is likely that the way the cortex integrates auditory and visual signals is different in children with a cochlear implant than it is in normally hearing children. A critical variable in the development of audiovisual integration might be the precocity of implantation. Auditory speech perception scores after implantation are better when children have been fitted before the age of three, and even two years old (Baumgartner, Pok, Egelierler et al., 2002; Tyler, Fryauf-Bertschy, Kelsay et al., 1997; Snik, Makhdoum, Vermeulen et al., 1997; Svirsky, Teoh, & Neuburger, 2004). Early implantation would allow auditory networks to maintain more of their initial functionality. Children fitted early with a cochlear implant could more readily exploit the phonetic relations between auditory and visual signals, and, thus, develop audio-visual processing mechanisms earlier and more efficiently...
In order to test the effect of exposure to Cued Speech, Colin et al. (2008) administered the same experiment to a group of deaf children fitted with a cochlear implant who had not been exposed to Cued Speech. These children showed the same reliance on speechreading than the Cued Speech-users when the visual syllable did not correspond with the auditory syllable.
Taken as a whole, these findings suggest that when faced with conflicting audiovisual stimuli, children fitted with a cochlear implant seem to rely mostly on visual speech information. Their auditory speech skills, which appear to be moderate in the AO condition, may be too fragile to resist when they are put into competition with visual processing. It must be noted that the McGurk experiment mimics fairly well the watching of a dubbed film on television; that is, the auditory information is not congruent with the information they could read on the lips. Given that many of the children have confidence in what they read on the lips, without perceiving the sound, it means that they should have problems watching dubbed films. Many of the participants reported this was the case.
Children with cochlear implants rely more on speechreading than normally-hearing children for different reasons. First, they might assign more weight to the visual speech information because the auditory information is degraded. This is evident in the case of normally hearing participants who must recognize stimuli consisting of spectrally reduced speech (SRS). The cochlear implant is that, in both cases, the speech information conveyed by the high frequencies, which is important to perceive the place of articulation, is degraded. Therefore, it is not surprising that children with a cochlear implant who have only partial access to place of articulation information through the auditory channel rely more on speechreading to process place of articulation. When auditory and visual information are put into conflict, their perception of speech is captured by visual information.
Second, the visual predominance of cochlear implant users might also be explained in terms of reorganization of neural resources in the case of deafness followed by cochlear implantation. Early deprivation of auditory information, as in congenital hearing impairment, can lead to a reorganization of neural resources, with a potentially larger involvement of auditory cortex in the processing of visual stimuli (Neville, Schmidt, & Kutas, 1983; Neville & Lawson, 1987). It has been found that the auditory cortex of deaf persons, once reorganized by cross-modal plasticity after years of deafness, can no longer respond to signals from a cochlear implant (Champoux, Lepore, Gagné & Théoret, 2009; Doucet, Bergeron, Lassonde, Ferron, Lepore, 2006; , Lee, Lee, Oh, Kim, Kim, Chung, Lee, Kim, 2001). Children and adults implanted at later ages are at a relative disadvantage compared to children implanted early, because the auditory cortex has already been appropriated by visual modality. As Shannon (2007) notes, the auditory system of children implanted at early ages competes for cortical real estate whereas late implantation may be unable to dislodge existing cortical ‘squatters.’ The results of Schorr et al. (2005), which showed that children implanted later than 30 months of age fail to integrate visual cues with the auditory cues, is compatible with this view.
In summary, speech perception through a cochlear implant presents some important differences in speech perception compared to speech perception in normally hearing individuals. Children with cochlear implants rely more on visual speech information, probably because this information is more reliable than the auditory information, particularly in noisy environments. From the available research, there is no reason to discourage children with a cochlear implant from employing speechreading or using the visible manual cues of Cued Speech in addition to the auditory information available via the cochlear implant. Rather, research suggests that they need both for maximum speech perception.
Conclusions
Data collected in the 1980s and the 1990s demonstrated that the use of Cued Speech can be a powerful tool for language development and subsequent formal reading achievement by profoundly deaf children equipped with hearing aids. Cued Speech enhances speech perception through the visual modality, the acquisition of vocabulary and morphosyntax, and metalinguistic development, as well as the acquisition of reading and spelling (see Chapter 11 in this volume). More recent data seem to indicate that children who received cochlear implants benefit from previous exposure to Cued Speech; however, use of Cued Speech before implantation is likely to become increasingly more rare. Indeed, most children are now fitted with a cochlear implant around the age of one year. During the first months or years of cochlear implant use, speech perception of an implanted child remains imperfect. Oral comprehension does not develop exclusively by the auditory channel but necessitates audiovisual integration. Therefore, the addition of Cued Speech to the signal delivered by the cochlear implant might help deaf children in identifying new words. Children fitted early with a cochlear implant, thus, would benefit from multimodal input during the development of phonological representations, which would serve as the platform from which subsequent phonological awareness, reading, and spelling acquisition could be launched (see Chapter 11 in this volume).
The use of Cued Speech by children with a cochlear implant is not an automatic solution to language development of deaf children. Children may not reliably look at a speaker’s lips and hands, and they may tend to rely on auditory information alone. Some parents may lose their motivation to cue, feel discouraged, or simply abandon coding with the hands. Therefore, it would be important for educators and related service providers to regularly assess whether cueing remains necessary, and under what circumstances after implantation. It is likely that after some period of auditory habilitation, children fitted with a cochlear implant would be capable of learning new words by auditory means and reading alone. Continued attention, nonetheless, should be devoted to the development of delicate, but vital, aspects of language, such as morphosyntax. This domain of language acquisition is particularly important and sensitive to a lack of precise input, as Szagun’s (2004) data show. The capacity to develop morphosyntax easily in response to a well-specified input also tends to diminish with age, although the limits of a precise “sensitive period” cannot be fixed at the present time (Szagun, 2001). In short, the benefit and limits of the use of Cued Speech with children with a cochlear implant remain to be investigated more extensively. In particular, data from languages other than French are urgently needed.
http://www.cuedspeech.org/PDF/Winter2009_OnCue.pdf