Not so! I have a link here that points this out...
A Critical Period for the Development of the Human Central Auditory Pathways
Here is another excerpt...
Critical periods also appear to exist in the auditory system. Language development, dependent on the auditory system, is known to have a critical period [for a review see (Skuse, 1993Go)]. Transient expression of acetylcholinesterase, seen in the rat visual and somatosensory cortices during their critical period, is also found in the auditory cortex with a similar time-course (Robertson, 1987Go; Robertson et al., 1991Go). The cholinergic system is known to be involved in learning effects in the auditory cortex (Juliano, 1998Go; Kilgard and Merzenich, 1998Go; Weinberger, 1998Go) and might play a role in the formation of thalamocortical connections. Lesion studies, in the cochlea as well as in the central nervous system, support the idea of a critical period in auditory development (Harrison et al., 1991Go) [for a review of central lesions see e.g. (Wakita and Watanabe, 1997Go)], as do the data from congenitally deaf individuals equipped with cochlear implants (Eddington et al., 1978Go; Busby et al., 1992Go; Fryauf-Bertschy et al., 1997Go). If congenitally deaf patients are implanted during childhood they can gain complete language competence. However, congenitally deaf patients implanted as adults have significantly poorer auditory performance and do not gain comparable language competence. The deficits in the auditory system responsible for their poor performance remain to be explored.
The link for the above is here...
Congenital Auditory Deprivation Reduces Synaptic Activity within the Auditory Cortex in a Layer-specific Manner -- Kral et al. 10 (7): 714 -- Cerebral Cortex
As I have mentioned before there are two developmental issues going on here. One has to have the cognitive development aspect and the auditory development aspect. Without a properly developed auditory system, you can forget ever being able to hear or speak as well as the hearing do.