Corticalprocessingoftemporalmodulations
XiaoqinWang
aa,*,ThomasLua,LiLiang
a,bLaboratoryofAuditoryNeurophysiology,DepartmentofBiomedicalEngineering,JohnsHopkinsUniversitySchoolofMedicine,
720RutlandAvenue,Ross424,Baltimore,MD21205,USA
bHearingCenter,PearRiverHospitalofFirstMedicalUniversity,Guangzhou510282,GuangdongProvince,China
Abstract
Temporalmodulationsarefundamentalcomponentsofhumanspeechandanimalcommunicationsounds.Understandingtheirrepresentationsintheauditorycortexisacrucialsteptowardsourunderstandingofbrainmechanismsunderlyingspeechprocessing.Whilemodulatedsignalshavelongbeenusedasexperimentalstimuli,theircorticalrepresentationsarenotcompletelyunderstood,particularlyforrapidmodulations.Knownphysiologicaldatadonotadequatelyexplainpsychophysicalobservationsontheperceptionofrapidmodulations,largelyduetoslowstimulus-synchronizedtemporaldischargepatternsofcorticalneurons.Inthisarticle,wesummarizerecentfindingsfromourlaboratoryontemporalprocessingmechanismsintheauditorycortex.Thesefindingsshowthattheauditorycortexrepresentsslowmodulationsexplicitlyusingatemporalcodeandfastmodulationsimplicitlybyadischargeratecode.Rapidlymodulatedsignalswithinashort-timewindow($20–30ms)areintegratedandtransformedintoadischargerate-basedrepresentation.Thefindingsalsoindicatethatthereisasharedrepresentationoftemporalmodulationsbycorticalneuronsthatencodesthetemporalprofileembeddedincomplexsoundsofvariousspectralcontents.Ourresultssuggestthatcorticalprocessingofsoundstreamsoperatesona‘‘segment-by-segment’’basiswithatemporalintegrationwindowontheorderof$20–30ms.Ó2002ElsevierScienceB.V.Allrightsreserved.
Keywords:Auditorycortex;Temporalprocessing;Temporalintegration;Amplitudemodulation;Frequencymodulation;Temporalasymmetry;Species-specificvocalization
1.Introduction
Theneuralrepresentationoftemporalmodu-lationsintheauditorycortexisofspecialinteresttoourunderstandingofmechanismsunderlyingspeechprocessing.Temporalmodulationsarefundamentalcomponentsofcommunicationsou-ndssuchashumanspeechandanimalvocaliza-tions,aswellasmusicalsounds.Low-frequency
Correspondingauthor.Tel.:+1-410-614-4547;fax:+1-410-614-9599.
E-mailaddress:xwang@bme.jhu.edu(X.Wang).
*modulationsareimportantforspeechperceptionandmelodyrecognition,whilehigher-frequencymodulationsproduceothertypesofsensationssuchaspitchandroughness(HoutgastandStee-neken,1973;Rosen,1992).Bothhumansandan-imalsarecapableofperceivingtheinformationcontainedintemporallymodulatedsoundsacrossawiderangeoftimescalesfrommillisecondtotensandhundredsofmilliseconds.Howtheau-ditorycortexencodesthiswidedynamicrangeoftemporalmodulationsisnotwellunderstood.Theneuralrepresentationoftemporalmodu-lationsbeginsattheauditoryperipherywhere
0167-6393/02/$-seefrontmatterÓ2002ElsevierScienceB.V.Allrightsreserved.doi:10.1016/S0167-6393(02)00097-3
108X.Wangetal./SpeechCommunication41(2003)107–121
auditory-nervefibersfaithfullyrepresentfinede-tailsofcomplexsoundsintheirtemporaldischargepatterns(Johnson,1980;JorisandYin,1992;Palmer,1982;WangandSachs,1993).Atsubse-quentnucleialongtheascendingauditorypathway(CN––cochlearnucleus,IC––inferiorcolliculusandMGB––auditorythalamus),theprecisionofthistemporalrepresentationgraduallydegrades(e.g.,CN:BlackburnandSachs,1989;Frisinaetal.,1990;WangandSachs,1994;IC:LangnerandSchreiner,1988;MGB:Creutzfeldtetal.,1980;deRibaupierreetal.,1980)duetobio-physicalpropertiesofneuronsandtemporalinte-grationofconverginginputsfromonestationtothenext.Inamodelingstudyofthetransforma-tionoftemporaldischargepatternsfromtheau-ditorynervetothecochlearnucleus,WangandSachs(1995)showedthatthereductionofphase-lockinginstellatecellscanresultfromthreemechanisms:convergenceofsubthresholdinputsonthesoma,inhibition,andthewell-knownden-driticlow-passfiltering(RallandAgmon-Snir,1998).Thesebasicmechanismsmayalsooperateatsuccessivenucleileadingtotheauditorycortex,progressivelyreducingthetemporallimitofstim-ulus-synchronizedresponses.
Ithaslongbeennoticedthatneuronsintheau-ditorycortexdonotfaithfullyfollowrapidlychangingstimuluscomponents(deRibaupierreetal.,1972;Goldsteinetal.,1959;WhitfieldandEvans,1965).Anumberofpreviousstudieshaveshownthatcorticalneuronscanonlybesynchro-nizedtotemporalmodulationsataratefarlessthan
100Hz(BieserandM€u
ller-Preuss,1996;deRib-aupierreetal.,1972;Eggermont,1991,1994;GaeseandOstwald,1995;LuandWang,2000;SchreinerandUrbas,1988),comparedwithalimitof$1kHzattheauditory-nerve(JorisandYin,1992;Palmer,1982).Thelackofsynchronizedcorticalresponsestorapid,butperceivabletemporalmodulationhasbeenpuzzling.Becausemostofthepreviousstudiesinthepastthreedecadesonthissubjectwerecon-ductedinanesthetizedanimals,withafewexcep-tions(BieserandM€u
ller-Preuss,1996;Creutzfeldtetal.,1980;deRibaupierreetal.,1972;EvansandWhitfield,1964;Goldsteinetal.,1959;WhitfieldandEvans,1965),ithasbeenspeculatedthatthereportedlowtemporalresponserateintheauditory
cortexmightbecausedpartiallybyanesthetics,whichhavebeenshowntoalterthetemporalre-sponsepropertiesoftheauditorycortex(Goldsteinetal.,1959;Zuritaetal.,1994).Neuralresponsesobtainedunderunanesthetizedconditionsarethereforeofparticularimportancetoourunder-standingofcorticalrepresentationsoftemporallymodulatedsignals.Thisarticlesummarizesrecentfindingsfromourstudiesinawakeprimatesontheissuesrelatedtocorticalrepresentationsoftempo-ralmodulations.
2.Thelimitonstimulus-synchronizeddischargesintheauditorycortexandatwo-stagemechanismGoldsteinetal.(1959)showedthatclick-fol-lowingratesofcorticalevokedpotentialswerehigherinunanesthetizedcatsthaninanesthetizedones.Inourstudy,wesystematicallyinvestigatedresponsesofsingleneuronsintheprimaryaudi-torycortex(A1)ofawakemarmosetmonkeystorapidsequencesofclicks(Luetal.,2001b).Bothwide-andnarrow-bandclicktrainswithinter-clickintervals(ICIs)rangingfrom3to100mswerestudied.Narrow-bandclickswerecenteredateachneuronÕscharacteristicfrequency(CF).IncontrasttoneuronsstudiedinA1ofanesthetizedanimals,whichrespondedstronglytobothwide-andnar-row-bandclicks(LuandWang,2000),themajorityofneuronsexaminedinA1ofunanesthetizedan-imalsrespondedstronglytonarrow-bandclicks,butwereonlyweaklydrivenor,moreoften,un-responsivetowide-band(rectangular)clicks(Luetal.,2001b).OnetypeofresponsetoclicktrainsisillustratedinFig.1(A).Thisneuronexhibitedsignificantstimulus-synchronizedresponsestoclickstimuliatlongICIs(>25–30ms).Thedis-chargestoclicktrainsbecamenon-synchronizedatmediumICIsanddiminishedatshortICIs(<20–25ms),apparentlyduetoinhibition(Fig.1(A)).Thesecondtypeofneuralresponsedidnotexhibitstimulus-synchronizeddischarges.Theyresponded,however,tochangesinICIwithmonotonicallychangingdischargeratewhentheICIwasshorterthan$20–30ms,asillustratedbytheexampleinFig.1(B).Wehaveshownthatthelimitonstim-ulus-synchronizedresponsesisontheorderof
20–25ms(medianvalueofsampledpopulation)inA1intheunanesthetizedcondition(Luetal.,2001b).TheobservationthatneuronsaresensitivetochangesofshortICIsindicatesthatadischargerate-basedmechanismmaybeinoperationwhenICIsareshorterthan$20–30ms.WehaveidentifiedtwopopulationsofA1neuronsthatdisplayedthesetworesponsetypes,referredtoassynchronizedandnon-synchro-nizedpopulations,respectively(Fig.2).Thetwopopulationsappearedtoencodesequentialstim-uliinverydifferentmanners.Neuronsinthe
synchronizedpopulationshowedstimulus-syn-chronizeddischargesatlongICIs,butfewre-sponsesatshortICIs.Thispopulationofneuronscanthusrepresentslowlyoccurringtemporaleventsexplicitlyusingatemporalcode.Thenon-synchronizedpopulationofneuronsdidnotex-hibitstimulus-synchronizeddischargesateitherlongorshortICIs.Thispopulationofneuronscanimplicitlyrepresentrapidlychangingtemporalin-tervalsbytheiraveragedischargerates.Onthebasisofthesetwopopulationsofneurons,theauditorycortexcanrepresentawiderangeoftimeintervalsinsequential,repetitivestimuli(Fig.2).Thelargenumberofneuronswithnon-synchro-nizedandsustaineddischargesatshortICIsob-servedinA1ofawakeanimals(Luetal.,2001b)wasnotobservedinA1ofanesthetizedanimals(LuandWang,2000).
Theissuesofrateandtemporalcodinghavelongbeenstudiedattheauditoryperipheryandbrainstemregardingspeechsounds(Sachsetal.,
1992).Bothtemporal-placeandrate-placerepre-sentationsappeartobeadequatetorepresentavowelÕsspectrumbytheauditory-nervefibers(SachsandYoung,1979;YoungandSachs,1979).Atthecochlearnucleus,differenttypesofneuronsbegintoshowspecificityinutilizingrateandtemporalinformationinrepresentingavowelÕsspectrum,withthetemporal-placecodepreservedbybushycellsbutdegradedbychoppercells(BlackburnandSachs,1990).Asdemonstratedbyfindingsfromourstudies,rateandtemporalin-formationarefurthersegregatedamongpopula-tionsofneuronsintheauditorycortex,aprocessingstagethatisseveralsynapsesawayfromthecochlearnucleus.
3.Temporalintegrationwindowoftheprimaryauditorycortex
Thelimitedstimulus-synchronizedresponsesobservedinneuralresponsestoclicktrains(Figs.1and2)suggestthatcorticalneuronsintegratese-quentialorcontinuousstimulioverabrieftimewindowthatwedefineoperationallyasthetem-poralintegrationwindow.Twosequentialacousticeventsfallingwithinthetemporalintegrationwindowarenotdistinguishedasseparateeventsattheoutputofaneuron.Animplicationofthetemporalintegrationwindowisthatitshouldre-sultinamaximumresponseofaneuronwhentheintegrationofsequentialorcontinuingstimuluseventsisperformedoverthedurationofthiswindow.Wehavefurtherinvestigatedtheseno-tionsthroughaseriesofexperimentsinwhichcorticalneuronsweretestedwithavarietyoftemporalmodulations(Liangetal.,1999,2002).Intheseexperiments,temporalmodulationswerein-troducedbysinusoidallymodulatingtoneornoisecarriersinamplitudeorfrequency.Foramplitude-modulatedtones(sAM),thecarrierfrequencyremainedconstantwhiletheamplitudewassinu-soidallymodulated.Inthecaseofamplitude-modulatednoises(nAM),theamplitudeofanoisecarrier(narrow-bandorbroad-band)wasmodu-latedbyasinusoid.Forfrequency-modulatedtones(sFM),theamplituderemainedconstant
whilethecarrierfrequencywassinusoidallymod-ulated.
Fig.3showsresponsesofarepresentativeau-ditorycorticalneurontosAMandsFMstimuliatvariousmodulatingfrequencies.Atamodulationfrequencyof16Hz,thedischargeratereachedthemaximuminthisneuronforbothsAMandsFMstimuli(Fig.3(B)).Thismodulationfrequencyisconventionallyreferredtoasthedischargerate-basedbestmodulationfrequency(rBMF).DatashowninFig.3(B)aregenerallyreferredtoasthedischargerate-basedmodulationtransferfunction(rMTF).ThemajorityofneuronsinA1ofawakemarmosetsrespondedmaximallytomodulatedtonesataparticularmodulationfrequencywithsustainedfirings.Ingeneral,rMTFsderivedfromresponsesofaneurontosAMandsFMstimulihadsimilarshapesandcloselymatchedrBMF(Fig.3(B)).Thissimilaritywasalsoobservedbe-tweencorticalresponsestoamplitude-modulatedtoneornoise,asshownbyanexampleneuroninFig.4(A)and(B).Thisneurondischargedmaxi-mallyatmodulationfrequencyof64Hz,regard-lessofwhetherthetemporalmodulationwasintroducedinatoneornoisecarrier(Fig.4(C)).Becauseamplitudeandfrequencymodulationsareproducedalongdifferentstimulusdimensions,thematchofrBMFsinvariousstimulusconditionssuggestsaninherenttemporalselectivityincorticalneuronsthatisapplicabletoawiderangeoftime-varyingstimuli.ThefactthatauditorycorticalneuronsshowedsimilarrMTFandrBMF(Fig.4(C))inresponsetoamplitude-modulatedtonesandbroad-bandnoisesindicatesthattheobserved
modulationselectivitywasindeedatemporalin-steadofaspectralphenomenon.Together,thesedatashowthatneuronsintheauditorycortexofawakemarmosetshaveapreferredtemporalmod-ulationfrequencythatisrelativelyindependentofhowthetemporalmodulationisintroduced(inthetimeorfrequencydomain,bytoneornoisecar-rier).ThemostfrequentlyencounteredrBMFsinA1ofawakemarmosetsrangedfrom8to64Hz.Forthepopulationofneuronswestudied,thedistributionsofrBMFswerecenterednear30HzforbothsAMandsFMstimuli(Fig.5(A)).Infact,A1ismaximallyexcitednearthisparticulartem-poralmodulationfrequency(Fig.5(B)).Thissug-geststhattheoptimaltemporalintegrationwindowforA1asawholeisintheorderof$30ms.
TheexamplegiveninFig.3alsoshowsthatdischargesofcorticalneurons,inresponsetosAMandsFMstimuli,couldexhibitstimulus-synchro-nizedtemporalpatterns(Fig.3(A)).DischargepatternsthataresynchronizedtothemodulationwaveformofansAMorsFMstimuluscanbequantifiedbythevectorstrength(GoldbergandBrown,1969)andtheRayleighstatistic(MardiaandJupp,2000).ThevaluesoftheRayleighsta-tisticgreaterthan13.8areconsideredasstatisti-callysignificant(p<0:001)(MardiaandJupp,2000).Fig.3(C)plotsRayleighstatisticsasafunctionofmodulationfrequency,alsoreferredtoasthedischargesynchrony-basedmodulationtransferfunctionortemporalmodulationtrans-ferfunction(tMTF).ThemodulationfrequencycorrespondingtothemaximumofatMTFiscon-ventionallyreferredtoasthedischargesynchrony-basedbestmodulationfrequency(tBMF).ThisneuronrespondedtosAMstimuliwithwell-syn-chronizeddischargesatmodulationfrequenciesupto128Hz(Fig.3(C)).ThetemporaldischargepatternsinresponsetosFMstimuli(Fig.3(A),lower)inthesameneuron,however,differedmarkedlyfromthosetosAMstimuli(Fig.3(A),upper)inthatthereweretwoclustersoffiringswithineachmodulationperiod.Thiswasbecauseboththeupwardanddownwardtrajectoryofthemodulationwaveformexcitedthisneuron.Asaresult,thedischargesynchronywasmuchstronger
X.Wangetal./SpeechCommunication41(2003)107–121113
Fig.5.Populationpropertiesfordischargerate-basedmodu-lationfrequencyselectivity(Liangetal.,2002).(A)OverlappinghistogramsshowingdistributionsofrBMFderivedfromsAM(open)andsFM(shaded)stimuli,respectively.Theneuronsincludedinthehistogramexhibitedband-passtMTF.Thebin-widthsofthehistogramsareonabase-2logarithmicscale.ThedistributionsofrBMFsAMandrBMFsFMarenotstatisticallydifferentfromeachother(Wilcoxonranksumtest,p¼0:1).Themeansofthetwodistributionsare25.9Hz(rBMFsAM)and19.2Hz(rBMFsFM)onthebase-2logarithmicscaleand48.8Hz(rBMFsAM)and35.1Hz(rBMFsFM)onthelinearscale,re-spectively.Themediansofthetwodistributionsare22.6Hz(rBMFsAM)and18.1Hz(rBMFsFM),respectively.(B)DischargeratesaveragedovertheentirepopulationofsampledneuronsareplottedversusmodulationfrequencyforsAM(solidline)andsFM(dashedline)stimuli.Spontaneousrateswerenotsubtractedinthecalculations.
whenmeasuredattwicethemodulationfrequencythanatthemodulationfrequencyofthestimulus
(Fig.3(C)).Ingeneral,dischargescouldbesyn-chronizedatarateapproximatelyequaltothemodulationfrequencywhensAMstimuliwereused,whereasforsFMstimuli,responsesynchro-nizationcouldoccurataratetwiceaslargeasthemodulationfrequency.Moreover,stimulus-inducedsynchronizationwassometimesproducedbyonetypeofmodulatedsound,butnotbyan-othertypeinanindividualneuron.Temporalmodulationinastimulusis,therefore,notalwaysaccuratelyreflectedinthedischargesynchronyofcorticalneurons.
AcrosspopulationsofA1neurons,tBMFislowerthanrBMF(comparingFig.6(A)withFig.5(A)).ThedistributionsoftBMF,likethoseofrBMF,aresimilarforsAMandsFMresponses.Thisobservationagainsupportsthepropositionthatauditorycorticalneuronshaveapreferredtemporalmodulationfrequency,regardlessofhowthemodulationisintroduced(inthetimeorfre-quencydomain).Moreover,theselectivityforaparticulartemporalmodulationfrequencywasof-tenobservedinaveragedischargerateintheabsenceofstimulus-synchronizeddischarges,in-dicatingatemporal-to-ratetransformationinthecorticalcodingoftemporalmodulations(Liangetal.,2002).
Anotherusefulmeasureofdischargesynchronyinaneuronisthemaximumsynchronizationfre-quency(fmax),whichisdefinedasthehighestmodulationfrequencyatwhichsignificantdis-chargesynchronyexists.Thedistributionoffmaxiscenteredbetween32and64Hz(Fig.6(B)),whichisconsistentwiththedistributionofthesynchro-nizationboundaryofcorticalneuronsdeterminedbyclicktrainstimuli(Fig.2).Fig.2showsthatthepercentageofneuronswiththesynchronizationboundarylessthan$25ms(equivalentto>40Hzmodulationfrequency)dropsrapidly.
InFig.6(C),weplotthepercentofsampledneuronsthatexhibitedstatisticallysignificantdis-chargesynchronyasafunctionofmodulationfrequency.ThisfigureshowsthatA1ismaximallysynchronizedtothetemporalmodulationatamodulationfrequencyof$8Hz,whichislowerthanthemodulationfrequency($30Hz)thatelicitsthemaximumdischargerate(Fig.5(B)).
114X.Wangetal./SpeechCommunication41(2003)107–121
Fig.6.Populationpropertiesfordischargesynchrony-basedmodulationfrequencyselectivity(Liangetal.,2002).(A)OverlappinghistogramsshowingdistributionsoftBMFderivedfromsAM(open)andsFM(shaded)stimuli,respectively.Thebin-widthsofthehistogramsareonabase-2logarithmicscale.ThedistributionsoftBMFsAMandtBMFsFMarenotstatisticallydifferentfromeachother(Wilcoxonranksumtest,p¼0:82).Themeansofthetwodistributionsare9.7Hz(tBMFsAM)and9.2Hz(tBMFsFM)onthebase-2logarithmicscaleand15.6Hz(tBMFsAM)and14.2Hz(tBMFsFM)onthelinearscale,respectively.Themediansofthetwodistributionsare9.6Hz(tBMFsAM)and10.0Hz(tBMFsFM),respectively.(B)Overlappinghistogramsshowingdistributionsofmaximumsyn-chronizationfrequency(fmax)derivedfromsAM(open)andsFM(shaded)stimuli,respectively.Thebin-widthsofthehistogramsareonabase-2logarithmicscale.Thedistributionsoffmax(sAM)andfmax(sFM)arenotstatisticallydifferentfromeachother(Wilcoxonranksumtest,p¼0:97).Themeansofthetwodistributionsare34.2Hz(sAM)and32.9Hz(sFM)onthebase-2logarithmicscaleand58.9Hz(sAM)and57.4Hz(sFM)onthelinearscale,respectively.Themediansofthetwodistributionsare34.2Hz(sAM)and39.4Hz(sFM),respectively.(C)PercentageofneuronswithstatisticallysignificantRayleighstatistic(>13.8)overtheentirepopulationofsampledneuronsisplottedversusmodulationfrequencyforsAM(solidline)andsFM(dashedline)stimuli.
Theseobservationsfurthersupportthetwo-stagemechanismproposedonthebasisofcorticalre-sponsestoclicktrains(Luetal.,2001b).
Dotheresponsepropertiesofcorticalneuronsrevealedbytemporallymodulatedsignalsbearanyimplicationsfortheprocessingofcomplexsoundssuchasspeechorspecies-specificvocalizations?Asweknow,speechandmusicalsoundscontainprominentmodulationsinbothamplitudeandfrequencydomains.Inparticular,low-frequency(<30Hz)modulationsareimportantformelodyrecognitionwhich,assuggestedbyourdata,maybeencodedonthebasisoftemporaldischargepatternsofcorticalneurons.Onepossiblepsycho-
X.Wangetal./SpeechCommunication41(2003)107–121115
physicalcorrelateofthemaximumsynchronizationfrequencyisthelowerlimitofpitch,whichisde-finedasthelowestrepetitionratethatevokesasensationofpitchandhasbeenfoundnear30Hz(Krumbholzetal.,2000;Pressnitzeretal.,2001).Thetemporalintegrationwindowofcorticalneu-ronsontheorderof20–30mssuggeststhatspectro-temporaltransients,suchasformanttransitions,maybeintegratedintheauditorycortexandim-plicitlycodedbydischargerates.Formodulationfrequenciesbelow$30Hz,modulationperiodscanberesolvedtemporallybecausetherearesignificantsynchronizeddischargestothemodulationperiods(Liangetal.,2002).Forhighermodulationfre-quenciesthatcorrespondpsychophysicallytothesensationofroughness(ZwickerandFastl,1999),theyarelikelyrepresentedbydischargerateinsteadoftemporaldischargepatterns.TherBMFcanbeashighas256Hzinunanesthetizedauditorycortex
(BieserandM€u
ller-Preuss,1996;Liangetal.,2002).Ourrecentstudyusingclicktrainstimulishowedrate-codingforevenhigherrepetitionfrequencies(Luetal.,2001b).Theprimatespeciesstudied,thecommonmarmoset,producestwotypesofvocal-izations(trillandtrillphee)thatdisplayprominentsinusoidalfrequencymodulations(Wang,2000).Thedistributionsofthemodulationfrequencyinbothtypesofvocalizationsarecenteredcloseto30Hz(AgamaiteandWang,1997),nearthetemporalmodulationfrequencypreferredbymostcorticalneurons(Liangetal.,2002).Temporalmodulationfrequencyinthesevocalizationsdifferedamongindividualmarmosets,suggestingthatthisparam-etermaybeusedincalleridentification.Behav-iorally,trillandtrillpheecallsareconsideredcontactcallsthatmarmosetsuseduringintra-spe-ciesvocalexchanges.HavingthecallsÕmodulationfrequenciesclosetothetemporalmodulationfre-quencythatmaximallyexcitestheauditorycortexshouldfacilitatecorticalprocessingofthesevo-calizations.
4.Corticalresponsestostimulustransientswithinthetemporalintegrationwindow
TheexperimentsdiscussedabovesuggestthatA1neuronsintegratestimuluscomponentswithin
atimewindowof$30msandtreatcomponentsoutsidethiswindowasdiscreteacousticevents.HumansandanimalsareknowntodiscriminatechangesinacousticsignalsattimescalesshorterthanthetemporalintegrationwindowofA1neu-rons,suggestingthatcorticalneuronsmustbeabletosignalsuchrapidchanges.Wehaveinvestigatedsensitivityofcorticalneuronstorapidchangeswithintheputativetemporalintegrationwindow(Luetal.,2001a)usingaclassoftemporallymodulatedsignalstermedrampedanddampedsi-nusoids(Patterson,1994a,b)(Fig.7(A)).Adampedsinusoidconsistsofapuretoneamplitude-modu-latedbyanexponentialfunction.Ithasafastonsetfollowedbyaslowoffset.Therateofam-plitudedecayisdeterminedbytheexponentialhalf-life.Arampedsinusoidisatime-reverseddampedsinusoid.Bothtypesofsoundshaveidenticallong-termFourierspectra.Ourexperi-mentalstimuliconsistedoframpedordampedsi-nusoidsegments,typicallywithaperiodof25ms,repeatedconsecutivelyfor500ms.Foreachneu-ron,thecarrierfrequencywassettoaneuronÕsCFandthehalf-lifewasvariedfrom0.5to32ms.Mostcorticalneuronsthatwestudiedshowedaclearpreferenceforeitherrampedordampedsi-nusoids(i.e.,theyrespondedmorevigorouslytoasinglestimulustype),withagreaterportionofneuronspreferringrampedstimuli(Luetal.,2001a).Someneuronsrespondednearlyexclu-sivelytoonestimulustype.Arepresentativeex-ampleofaneuronpreferringrampedsinusoidstodampedsinusoidsisshowninFig.7(B).Thisneuronrespondedmorestronglytorampedsinu-soidsatmosthalf-lives(Fig.7(B),aandb).Theresponseasymmetrywasobservedinaveragedis-chargerate,butnotinstimulus-synchronizeddis-charges(Fig.7(B),c).Fig.7(C)showsaneuronthatrespondedpreferentiallytodampedsinusoids.Generally,preferenceforstimulustypewascon-sistentacrosshalf-livesasillustratedbytheseex-amples.Theseobservationsdemonstratedthattemporalcharacteristicswithinthetemporalinte-grationwindowcanprofoundlymodulateacor-ticalneuronÕsresponsiveness.
Wefoundthatneuronsinbothsynchronizedandnon-synchronizedpopulations(Fig.2)weresensitivetotemporalasymmetrywithinabrief
116X.Wangetal./SpeechCommunication41(2003)107–121
Fig.7.Corticalresponsestorampedanddampedsinusoidalstimuli(Luetal.,2001a).(A)Rampedanddampedsinusoidalstimuli(Patterson,1994a)withhalf-liferangingfrom0.5to32ms.Theexamplesshownhaveaperiodof25msandcarrierfrequencyof10kHz.Onlythefirst100msofthestimuliareshown.Rampedsinusoidsareshownintheshadedblocks.(B)ArepresentativeexampleofneuronswithapreferenceforrampedsinusoidsrecordedinmarmosetA1(Luetal.,2001a).(a)PSTHsareplottedintheorderofincreasinghalf-lifealongtheordinate,withalternatingresponsestoramped(shaded)anddamped(unshaded)sinusoids.TheheightsofPSTHsarenormalizedtothemaximumbincountoverallstimulusconditions.Stimulusonsetwasat500ms,anddurationwas500ms.(b)Drivendischargeratesareplottedasafunctionofhalf-lifeforramped(thickline)anddamped(thinlinewithopencircles)sinusoids,respectively.(c)Asymmetryindexisshownforcalculationsbasedondischargerate(cross)orvectorstrength,VS(square)andisdefinedasðRrÀRdÞ=ðRrþRdÞ,whereRrandRdareaveragedischargeratesorVStorampedanddampedstimuli,respectively.Non-significantindexvaluesweresettozero(pP0:05,Wilcoxonrank-sum).Significantasymmetrypreferencebasedondischargeratewaspresentovermosthalf-livestestedinthisneuron.Nostatisticallysignificantasymmetryindexwasfoundforstimulus-synchronizedactivityinthisneuron.(C)Arepresentativeexampleofneuronswithpreferencefordampedsinusoids.Theformatisthesameasin(B).
timewindow($25ms)asmeasuredbytheirav-eragedischargerates.Fig.8showsresponsesoftworepresentativeA1neuronstosequencesoframpedanddampedsinusoidsatdifferentinter-stimulusintervalsorrepetitionperiods(3–100ms),onewithstimulus-synchronizeddischarges(Fig.8(A))andtheotherwithnon-synchronizeddis-charges(Fig.8(B)).Atrepetitionperiodslongerthanthepresumedtemporalintegrationwindow(>20–25ms),dischargesoftheneuronshowninFig.8(A)weresynchronizedtoeachperiodoframpedanddampedsinusoidswhiletheneuronshowedstrongerresponsestodampedsinusoids(Fig.8(A),top).Whenrepetitionperiodswereshorterthanthepresumedtemporalintegrationwindow,synchronizeddischargesdisappearedbut
theoverallpreferenceforthedampedsinusoidswasmaintained(Fig.8(A),bottom).Fig.8(B)showsaneuronofthenon-synchronizedpopula-tionthatrespondedmorestronglytoramped
sinusoidsatallrepetitionperiodstested.Theseobservationsdemonstratethatthesensitivitytotemporalasymmetrywithinthetemporalintegra-tionwindowisindependentofacorticalneuronÕsabilitytosynchronizetostimulusevents,suggest-ingarate-basedcodingmechanismfortimescalesshorterthanthetemporalintegrationwindowforA1neurons.
InFig.9wecompareresponseasymmetryofpopulationsofA1neuronswithpsychophysicalperformanceindiscriminatingrampedversusdampedsinusoidsbyhumans(Luetal.,2001a).Theshapeofthecurvebasedonaveragedischargerateisqualitativelysimilartopsychophysicaldatawithbothtonecarriers(Patterson,1994a)andwide-bandnoisecarriers(AkeroydandPatterson,1995).Psychophysicalperformanceacrosshalf-lifeappearstoberelatedtothepercentageofA1neuronsthatshowedsignificantresponseasym-metryintheiraveragedischargerates.Apopula-tionmeasurebasedondischargesynchrony,ontheotherhand,revealsthatonlyaverysmallportionofA1neurons(<5%)showedresponseasymmetry
intheirtemporaldischargepatternsforthestim-ulusperiodused(25ms).
5.Theroleoftemporalselectivityanditsbehavioralrelevanceincorticalprocessingofspecies-specificvocalizations
Theneuralmechanismsinvolvedinproducingthetemporalselectivitydiscussedabove(atlongandshort-timescales)maypossiblycontributetoneuralselectivityofcomplexvocalizations(Esseretal.,1997;Margoliash,1983;Wangetal.,1995a).IthasbeenshownthatnaturalvocalizationsofmarmosetmonkeysproducedstrongerresponsesinA1thandospectrallysimilar,buttemporallyalteredvocalizations(Wangetal.,1995a).Fig.10showsthatasubpopulationofneuronsinA1ofanesthetizedmarmosetsrespondedmorestronglytoanaturaltwittercallthantoitstime-reversedversion.Responsesofthissubpopulationofneu-ronswerefoundtohaveaclearerrepresentationofthespectralshapeofthecallthandidresponsesofthenon-selectiveneurons(Wang,2000;Wangetal.,1995a).Whiletheseobservationsdemonstratearoleoftemporalselectivityincorticalresponsestocomplexvocalizations,theyalsosuggestthatmarmosetauditorycortexmaypreferentiallyre-spondtosoundswithbehavioralsignificance.Tofurthertestthisnotion,wehavedirectlycomparedresponsestonaturalandtime-reversedcallsintheauditorycortexofthecat,anextensivelystudiedmammalianspecies,whoseA1sharessimilarbasicphysiologicalproperties(e.g.,CF,threshold,la-tency,etc.)tothatofthemarmoset(AitkinandPark,1993;Schreineretal.,2000).UnlikeneuronsinA1ofmarmosets,however,neuronsincatA1didnotdifferentiatenaturalmarmosetvocaliza-tionsfromtheirtime-reversedversions(WangandKadia,2001).Together,theseobservationssuggestthattemporalselectivityofcorticalneuronsmaybedependentonthebehavioralrelevanceofacousticsignalsthataspeciesencounters.Apar-ticularformoftemporalselectivityunderonebe-havioralcontextmaynotexistunderotherbehavioralcontexts.Onepotentialmechanismthatmaygiverisetosuchbehavioraldependenceisexperience-dependentcorticalplasticity(Buono-
manoandMerzenich,1998;Merzenichetal.,1984).Infact,ithasbeenshowninthesomato-
sensorycortexthattemporalaspectsofacomplexstimulusarecrucialindeterminingspecificformsoflearning-inducedcorticalreorganization(Wangetal.,1995b).Wehaveargued(Wang,2000)thatthetwocentralissuesinourunderstandingofcorticalrepresentationofcommunicationssoundsare:(a)neuralencodingofstatisticalstructureofcommunicationsoundsand(b)theroleofbehav-ioralrelevanceinshapingcorticalrepresentations.Behaviorallyrelevanttemporalmodulationsareclearlyimportantacousticfeaturestobeconsid-eredintheexplorationofneuralencodingmech-anismsforspeech-likesignalsintheauditorycortex.
6.Summary
Basedonourfindingsfromthestudiesdis-cussedabove,wesuggestatwo-stagemodelforprocessingtemporalmodulationsbytheauditorycortex.Inthismodel,theauditorycortexinte-gratescontinuousacousticstreamsoveratempo-ralintegrationwindowof%30ms.Temporalpatternsthatareseparatedbyintervalslongerthanthisintegrationwindowareexplicitlycodedbytemporaldischargepatternsofcorticalneurons.Rapidtime-varyingcomponentswithinthetem-poralintegrationwindowareinsteadrepresentedimplicitlybyadischargerate-basedcode.Thecombinationofbothtemporalandratecodesshouldsufficientlyencodethewiderangeoftem-poralmodulationsofbiologicallyimportantcom-plexsounds.
Thesignificantreductioninthetemporallimitonstimulus-synchronizeddischargesattheaudi-torycortex,ascomparedwiththeauditoryperi-phery,hasanimportantfunctionalimplication.Itsuggeststhatcorticalprocessingofsoundstreamsoperatesona‘‘segment-by-segment’’basisratherthanona‘‘moment-by-moment’’basisasfoundintheauditoryperiphery.Thisisperhapsnecessaryforcomplexintegrationandcomparisontotakeplaceatthisleveloftheauditorysystem,sincehigher-levelprocessingtasksrequireabroaderviewofacousticeventsprecedingandfollowingaparticulartimeofinterest.Moreover,auditoryinformationisencodedattheperipheryatamuch
120X.Wangetal./SpeechCommunication41(2003)107–121
highertemporalmodulationratethantheratesatwhichthevisualortactileinformationisencoded.Theslow-downoftemporalresponseratealongtheascendingauditorypathwayallowsfast-pacedauditoryinformationtobeintegratedinthecere-bralcortexwithinformationfromothersensorymodalitiesthatisintrinsicallyslower.
Acknowledgements
WewouldliketodedicatethisarticletoDr.MurrayB.Sachs,aregularparticipantinthepre-viousUtrechtSpeechPerceptionworkshops,whoseworkonneuralencodingmechanismsattheauditory-nerveandcochlearnucleushaspavedthewayforourunderstandingoftheirtransformationsathigherauditorycenters.Thisresearchwassup-portedbyWhitakerFoundationResearchGrantRG-96-0268,NIH-NIDCDGrantDC03180andbyaPresidentialEarlyCareerAwardforScientistsandEngineers(X.Wang).WethankDr.RossSniderandSteveEliadesfortechnicalassistance,AshleyPistorioforcolonymanagement,animaltraining,graphicassistanceandproofreadingthemanuscript.Publicationsfromourlaboratoryre-ferredtointhisarticlecanbeobtainedathttp://www.bme.jhu.edu/~xwang/papers.html.
References
Agamaite,J.A.,Wang,X.,1997.Quantitativeclassificationof
thevocalrepertoireofthecommonmarmoset(CallithrixJacchusJacchus).Assoc.Res.Otolaryng.Abstr.20,144.Aitkin,L.,Park,V.,1993.Auditionandtheauditorypathway
ofavocalnewworldprimate,thecommonmarmoset.Prog.Neurobiol.41,345–367.
Akeroyd,M.A.,Patterson,R.D.,1995.Discriminationof
widebandnoisesmodulatedbyatemporallyasymmetricfunction.J.Acoust.Soc.Amer.98,2466–2474.
Bieser,A.,M€u
ller-Preuss,P.,1996.Auditoryresponsivecortexinthesquirrelmonkey:neuralresponsestoamplitude-modulatedsounds.Exp.BrainRes.108,273–284.
Blackburn,C.C.,Sachs,M.B.,1989.Classificationofunittypes
intheanteroventralcochlearnucleus:post-stimulustimehistogramsandregularityanalysis.J.Neurophysiol.62,1303–1329.
Blackburn,C.C.,Sachs,M.B.,1990.Therepresentationsofthe
steady-statevowelsound/e/inthedischargepatternsofcatanteroventralcochlearnucleusneurons.J.Neurophysiol.63(5),1191–1212.
Buonomano,D.V.,Merzenich,M.M.,1998.Corticalplasticity:
fromsynapsestomaps.Annu.Rev.Neurosci.21(4),149–186.
Creutzfeldt,O.,Hellweg,F.-C.,Schreiner,C.,1980.Thalamo-corticaltransformationofresponsestocomplexauditorystimuli.Exp.BrainRes.39,87–104.
deRibaupierre,F.,GoldsteinJr.,M.H.,Yeni-Komshian,G.,
1972.Corticalcodingofrepetitiveacousticpulses.BrainRes.48(1),205–225.
deRibaupierre,F.,Rouiller,E.,Toros,A.,deRibaupierre,Y.,
1980.Transmissiondelayofphase-lockedcellsinthemedialgeniculatebody.HearingRes.3(1),65–77.
Eggermont,J.J.,1991.Rateandsynchronizationmeasuresof
periodicitycodingincatprimaryauditorycortex.HearingRes.56(1–2),153–167.
Eggermont,J.J.,1994.Temporalmodulationtransferfunctions
forAMandFMstimuliincatauditorycortex.Effectsofcarriertype,modulatingwaveformandintensity.HearingRes.74(1–2),51–66.
Esser,K.-H.,Condon,C.J.,Suga,N.,Kanwal,J.S.,1997.
SyntaxprocessingbyauditorycorticalneuronsintheFM–FMareaofthemustachedbatpteronotusparnellii.Proc.Natl.Acad.Sci.USA94,14019–14024.
Evans,E.F.,Whitfield,I.C.,1964.Classificationofunit
responsesintheauditorycortexoftheunanesthetizedandunrestrainedcat.J.Physiol.171,476–493.
Frisina,R.D.,Smith,R.L.,Chamberlain,S.C.,1990.Encoding
ofamplitudemodulationinthegerbilcochlearnucleus:I.Ahierarchyofenhancement.HearingRes.44,99–122.
Gaese,B.H.,Ostwald,J.,1995.Temporalcodingofamplitude
andfrequencymodulationintheratauditorycortex.Eur.J.Neurosci.7,438–450.
Goldberg,J.M.,Brown,P.B.,1969.Responseofbinaural
neuronsofdogsuperiorolivarycomplextodichotictonalstimuli:somephysiologicalmechanismsofsoundlocaliza-tion.J.Neurophysiol.32,613–636.
GoldsteinJr.,M.H.,Kiang,N.Y.-S.,Brown,R.M.,1959.
Responsesoftheauditorycortextorepetitiveacousticstimuli.J.Acoust.Soc.Amer.31,356–364.
Houtgast,T.,Steeneken,H.J.M.,1973.Themodulationtrans-ferfunctioninroomacousticsasapredictorofspeechintelligibility.Acoustica28,66–73.
Johnson,D.H.,1980.Therelationshipbetweenspikerateand
synchronyinresponsesofauditory-nervefiberstosingletones.J.Acoust.Soc.Amer.68,1115–1122.
Joris,P.X.,Yin,T.C.T.,1992.Responsestoamplitude-modu-latedtonesintheauditorynerveofthecat.J.Acoust.Soc.Amer.91(1),215–232.
Krumbholz,K.,Patterson,R.D.,Pressnitzer,D.,2000.The
lowerlimitofpitchasdeterminedbyratediscrimination.J.Acoust.Soc.Amer.108(3),1170–1180.
X.Wangetal./SpeechCommunication41(2003)107–121
121
Langner,G.,Schreiner,C.E.,1988.Periodicitycodinginthe
inferiorcolliculusofthecat.I.Neuronalmechanisms..J.Neurophysiol.60,1799–1822.
Liang,L.,Lu,T.,Wang,X.,1999.Temporalencodingof
amplitudemodulatedsoundswithnoisecarrierinthelateralbeltareasoftheauditorycortexinawakemarmosetmonkeys.Soc.Neurosci.Abstr.29.
Liang,L.,Lu,T.,Wang,X.,2002.Neuralrepresentationsof
sinusoidalamplitudeandfrequencymodulationsintheauditorycortexofawakeprimates.J.Neurophysiol.87,2237–2261.
Lu,T.,Wang,X.,2000.Temporaldischargepatternsevokedby
rapidsequencesofwide-andnarrow-bandclicksintheprimaryauditorycortexofcat.J.Neurophysiol.84,236–246.
Lu,T.,Liang,L.,Wang,X.,2001a.Neuralrepresentationof
temporallyasymmetricstimuliintheauditorycortexofawakeprimates.J.Neurophys.85,2364–2380.
Lu,T.,Liang,L.,Wang,X.,2001b.Temporalandrate
representationsoftime-varyingsignalsintheauditorycortexofawakeprimates.Nat.Neurosci.4,1131–1138.Mardia,K.V.,Jupp,P.E.,2000.DirectionalStatistics.Wiley,
NewYork.
Margoliash,D.,1983.Acousticparametersunderlyingthe
responsesofsong-specificneuronsinthewhite-crownedsparrow.J.Neurosci.3,1039–1057.
Merzenich,M.M.,Nelson,R.J.,Stryker,M.P.,Cynader,M.S.,
Schoppmann,A.,Zook,J.M.,1984.Somatosensorycorticalmapchangesfollowingdigitalamputationinadultmon-keys.J.Comp.Neurol.224,591–605.
Palmer,A.R.,1982.Encodingofrapidamplitudefluctuations
bycochlear-nervefibersintheguinea-pig.Arch.Otorhino-laryngol.236,197–202.
Patterson,R.D.,1994a.Thesoundofasinusoid:Spectral
models.J.Acoust.Soc.Amer.96,1409–1418.
Patterson,R.D.,1994b.Thesoundofasinusoid:Time-interval
models.J.Acoust.Soc.Amer.96,1419–1428.
Pressnitzer,D.,Patterson,R.D.,Krumbholz,K.,2001.The
lowerlimitofmelodicpitch.J.Acoust.Soc.Amer.109(5),2074–2084.
Rall,W.,Agmon-Snir,H.,1998.Cabletheoryfordendritic
neurons.In:Segev,C.K.a.I.(Ed.),MethodsinNeuronalModeling.TheMITPress,Cambridge,MA.
Rosen,S.,1992.Temporalinformationinspeech:acoustic,
auditoryandlinguisticaspects.Philos.Trans.Roy.Soc.Lond.Ser.B,Biol.Sci.336(1278),367–373.
Sachs,M.B.,Young,E.D.,1979.Encodingofsteady-state
vowelsintheauditorynerve:Representationintermsofdischargerate.J.Acoust.Soc.Amer.66,470–479.
Sachs,M.B.,Blackburn,C.C.,Banks,M.I.,Wang,X.,1992.Processingoftheauditory-nervecodeforspeechbypopulationsofcellsintheanteroventralcochlearnucleus.In:Schouten,M.E.H.(Ed.),TheAuditoryProcessingofSpeech:FromSoundstoWords.MoutonDeGruyterPublishers,Berlin,pp.47–60.
Schreiner,C.E.,Urbas,J.V.,1988.Representationofamplitudemodulationintheauditorycortexofthecat.II.Compar-isonbetweencorticalfields.HearingRes.32(1),49–63.Schreiner,C.E.,Read,H.L.,Sutter,M.L.,2000.Modularorganizationoffrequencyintegrationinprimaryauditorycortex.Annu.Rev.Neurosci.23,501–529.
Wang,X.,2000.Oncorticalcodingofvocalcommunicationsoundsinprimates.Proc.Natl.Acad.Sci.USA97,11843–11849.
Wang,X.,Kadia,S.C.,2001.Differentialrepresentationofspecies-specificprimatevocalizationsintheauditorycor-ticesofmarmosetandcat.J.Neurophysiol.86,2616–2620.Wang,X.,Sachs,M.B.,1993.Neuralencodingofsingle-formantstimuliinthecat.I.Responsesofauditory-nervefibers.J.Neurophysiol.70,1054–1075.
Wang,X.,Sachs,M.B.,1994.Neuralencodingofsingle-formantstimuliinthecat.II.Responsesofanteroventralcochlearnucleusunits.J.Neurophysiol.71,59–78.
Wang,X.,Sachs,M.B.,1995.TransformationoftemporaldischargepatternsinaVCNstellatecellmodel:Implica-tionsforphysiologicalmechanisms.J.Neurophysiol.73,1600–1616.
Wang,X.,Merzenich,M.M.,Beitel,R.,Schreiner,C.E.,1995a.Representationofaspecies-specificvocalizationintheprimaryauditorycortexofthecommonmarmoset:tempo-ralandspectralcharacteristics.J.Neurophysiol.74(6),2685–2706.
Wang,X.,Merzenich,M.M.,Sameshima,K.,Jenkins,W.M.,1995b.Remodelingofhandrepresentationinadultcortexdeterminedbytimingoftactilestimulation.Nature378,71–75.
Whitfield,I.,Evans,E.,1965.Responsesofauditorycorticalneuronstostimuliofchangingfrequency.J.Neurophysiol.28,655–672.
Young,E.D.,Sachs,M.B.,1979.Representationofsteady-statevowelsinthetemporalaspectsofthedischargepatternsofpopulationsofauditory-nervefibers.J.Acoust.Soc.Amer.66,1381–1403.
Zurita,P.,Villa,A.E.,deRibaupierre,Y.,deRibaupierre,F.,Rouiller,E.M.,1994.ChangesofsingleunitactivityinthecatÕsauditorythalamusandcortexassociatedtodifferentanestheticconditions.Neurosci.Res.19(3),303–316.
Zwicker,E.,Fastl,H.,1999.Psychoacoustics.Spinger,Berlin.
因篇幅问题不能全部显示，请点此查看更多更全内容